Method and system for prediction or training model based on multi-dimensional time series data

Abstract

A method for prediction based on multi-dimensional time series data is disclosed, including: monitoring a multi-dimensional time series data stream to obtain a current observation point, the current observation point including multi-dimensional feature data; based on the current observation point, applying a trained neural process model for prediction, the neural process model being trained using a plurality of previous observation points, each previous observation point including multi-dimensional feature data and corresponding label data, wherein the neural process model includes an encoder and a decoder, the encoder including a mutual attention module, the mutual attention module assigning weights to the plurality of previous observation points based on the association between the multi-dimensional feature data of the current observation point and the multi-dimensional feature data of one or more previous observation points for ultimately generating a target prediction of the current observation point. The present application also relates to other methods, systems, devices, and computer-readable storage devices. The method of the present application can perform prediction more accurately based on multi-dimensional time series data.

Description

Technical Field

[0001] One or more embodiments of this specification relate to machine learning, and in particular to methods, systems, devices, and computer-readable storage media for prediction or training models based on multi-dimensional time series data. Background Art

[0002] Currently, machine learning has been applied to time series data to model the time series data and use the established model to perform predictions or processing (such as anomaly recognition or risk recognition, etc.).

[0003] For example, in scenarios such as online transactions, time series data associated with users (such as login history, transaction history, etc.) can usually be used for modeling, so that abnormal transactions (such as illegal transactions) or abnormal users (such as users who perform illegal transactions) can be identified.

[0004] However, there may be problems when using time series data in scenarios such as online transactions for modeling. For example, on the one hand, most machine learning models can only process one-dimensional time series data and have difficulties in processing multi-dimensional time series data. On the other hand, for multi-dimensional time series data, existing solutions usually adopt rule-based models (sometimes called baseline models). Such rule-based models often have strong correlations in scoring metrics in different intervals, so there is a problem of low accuracy.

[0005] In addition, existing machine learning models can usually achieve relatively high accuracy only when the data sample size is large when processing multi-dimensional time series data. When the data sample size is low (such as for long-tail users or new users with little data), existing machine learning models usually have difficulty producing satisfactory results.

[0006] Unfortunately, for users' online activities, especially activities such as anomaly recognition in online transaction scenarios, the data to be processed often has strong correlations and often requires processing of less data. For example, malicious entities often register as new users to perform malicious operations, so there are usually fewer available samples for identifying abnormal transactions of such new users.

[0007] Therefore, there is a need for a modeling solution for multi-dimensional time series data that can also achieve high accuracy with less data. Summary of the Invention

[0008] To overcome the defects of the prior art, one or more embodiments of this specification provide a more accurate modeling solution by establishing a private model for each user and provide more perfect privacy protection for users.

[0009] One or more embodiments of this specification achieve their above objects through the following technical solutions.

[0010] In one aspect, a method for prediction based on multi-dimensional time series data is disclosed, including: monitoring a multi-dimensional time series data stream to obtain a current observation point, the current observation point including multi-dimensional feature data xt; based on the current observation point, applying a trained neural process model for prediction, the neural process model being trained using a plurality of previous observation points (xi, yi), each previous observation point including multi-dimensional feature data xi and corresponding label data yi, wherein the neural process model includes an encoder and a decoder, the encoder including a mutual attention module, the mutual attention module assigning weights to the plurality of previous observation points based on the association between the multi-dimensional feature data xt of the current observation point and the multi-dimensional feature data xi of one or more previous observation points for finally generating a target prediction y* of the current observation point.

[0011] Preferably, the encoder includes a deterministic path and a latent path, wherein the deterministic path includes a deterministic encoder for generating a plurality of encoded representations ri based on a plurality of previous observation points (xi, yi), and the mutual attention module generating a single aggregated representation r* specific to the current observation point based on the multi-dimensional feature data xt of the current observation point, the multi-dimensional feature data xi of the one or more previous observation points, and the plurality of encoded representations.

[0012] Preferably, the deterministic encoder uses a self-attention model.

[0013] Preferably, the latent path generates a latent variable z based on the plurality of previous observation points (xi, yi), and the decoder generates the target prediction y* of the current observation point based on the multi-dimensional feature data xt of the current observation point, the representation r* specific to the current observation point, and the latent variable z.

[0014] Preferably, the multi-dimensional time data stream is an online activity data stream of a user, and wherein the method includes:

[0015] Detecting an anomaly in the online activity of the user based on the target prediction y*.

[0016] Preferably, the method further includes:

[0017] Obtaining offline data, wherein detecting the anomaly in the online activity of the user is further based on the offline data.

[0018] Preferably, the method further includes:

[0019] Providing a one-dimensional time series anomaly detection module, and

[0020] Use the one-dimensional time series anomaly detection module together with the neural process model to detect anomalies in the user's online activities.

[0021] Preferably, the method further includes:

[0022] After detecting an anomaly, automatically determine the cause of the anomaly using an attribution module.

[0023] Preferably, the method further includes:

[0024] After detecting an anomaly, output an alarm message using an alarm module.

[0025] On the other hand, a method for training a model based on multi-dimensional time series data is also disclosed, including: obtaining multi-dimensional time series data; generating a plurality of previous observation points (xi, yi) based on the multi-dimensional time series data, each observation point corresponding to a time indication and including multi-dimensional feature data xi and corresponding label data yi; using a subset of the plurality of previous observation points (xi, yi) as training data to train a neural process model, where the neural process model includes an encoder and a decoder, and the encoder includes a mutual attention module, and the mutual attention module is configured to: for each observation point, assign weights to the plurality of previous observation points based on the association between the multi-dimensional feature data xi of this observation point and the multi-dimensional feature data xi of other observation points in the subset, for finally generating the target prediction yip of this observation point, where the target prediction yip is used to generate a loss value with the label data yi of this observation point, and iteratively adjust the neural process model based on the loss value to generate a trained model.

[0026] Preferably, the encoder includes a deterministic path and a latent path, where the deterministic path includes a deterministic encoder, and the deterministic encoder is used to generate a plurality of encoded representations ri based on the subset, and the mutual attention module generates a single aggregated representation r specific to this observation point based on the multi-dimensional feature data xi of this observation point, the multi-dimensional feature data xi of the previous observation points in the subset, and the plurality of encoded representations.

[0027] Preferably, the deterministic encoder uses a self-attention model.

[0028] Preferably, the latent path generates a latent variable z based on the subset, and the decoder generates the target prediction yip of this observation point based on the multi-dimensional feature data xi specific to this observation point, the representation r specific to this observation point, and the latent variable z.

[0029] Preferably, the method further includes:

[0030] Calculate a loss value based on the target prediction yip and the label data yi for one or more observation points; and

[0031] Iteratively update the neural process model based on the loss value to generate a trained neural process model.

[0032] In another aspect, a system for detecting anomalies in online activities is also disclosed, including: a data acquisition module configured to: monitor the data stream of a user's online activities to obtain a current observation point, the current observation point including multi-dimensional feature data xt; a multi-dimensional time series anomaly detection module configured to: based on the current observation point, apply a trained neural process model for prediction, the neural process model being trained using a plurality of previous observation points (xi, yi), each previous observation point including multi-dimensional feature data xi and corresponding label data yi, wherein the neural process model includes an encoder and a decoder, and the encoder includes a mutual attention module that assigns weights to the plurality of previous observation points based on the association between the multi-dimensional feature data xt of the current observation point and the multi-dimensional feature data xi of one or more previous observation points for finally generating a target prediction y* of the current observation point; and detect anomalies in the user's online activities based on the target prediction y*.

[0033] Preferably, the data acquisition module is further configured to acquire offline data, and detecting anomalies in the user's online activities is further based on the offline data.

[0034] Preferably, the system further includes a one-dimensional time series anomaly detection module, wherein the one-dimensional time series anomaly detection module is configured to detect anomalies in the user's online activities together with the multi-dimensional time series anomaly detection module.

[0035] Preferably, the system further includes an attribution module, wherein the attribution module is configured to automatically determine the cause of the anomaly using the attribution module.

[0036] Preferably, the system further includes an alarm module, wherein the alarm module is configured to output an alarm message after detecting an anomaly.

[0037] In yet another aspect, an apparatus for generating a model for a user is disclosed, including: a memory; and a processor configured to execute the method as described in any of the above.

[0038] In yet another aspect, a computer-readable storage medium storing instructions is disclosed, which when executed by a computer, cause the computer to execute the above method.

[0039] Compared with the prior art, one or more embodiments of the present specification can achieve one or more of the following technical effects:

[0040] Can perform predictions more accurately based on multi-dimensional time series data;

[0041] Can process multi-dimensional time series data; and

[0042] Can perform predictions with less data volume. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above invention content and the following detailed implementation manners will be better understood when read in conjunction with the accompanying drawings. It should be noted that the drawings are only examples of the claimed invention. In the drawings, the same reference numerals represent the same or similar elements.

[0044] Figure 1 Shows an overall flowchart of a method for modeling multi-dimensional time series data according to an embodiment of the present specification.

[0045] Figure 2 Shows a schematic diagram of a neural process model according to an embodiment of the present specification.

[0046] Figure 3 Shows a flowchart of an example method for generating predictions of a neural process model according to an embodiment of the present specification.

[0047] Figure 4 Shows a flowchart of an example method for making predictions based on multi-dimensional time series data according to an embodiment of the present specification.

[0048] Figure 5 Shows a flowchart of an example method for detecting anomalies in online activities according to an embodiment of the present specification.

[0049] Figure 6 Shows a block diagram of an example system for detecting anomalies in online activities according to an embodiment of the present specification.

[0050] Figure 7 Shows a schematic block diagram of an apparatus that can be used to perform the methods described above according to an embodiment of the present specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The content of the following detailed implementation manners is sufficient for any person skilled in the art to understand the technical content of one or more embodiments of the present specification and to implement them accordingly. And based on the specification, claims and drawings disclosed in the present specification, those skilled in the art can easily understand the objectives and advantages related to one or more embodiments of the present specification.

[0052] As described above, there may be problems when using time series data in scenarios such as online transactions for modeling. For example, on the one hand, most machine learning models can only process one-dimensional time series data and have difficulties in processing multi-dimensional time series data. On the other hand, for multi-dimensional time series data, existing solutions usually adopt rule-based models (sometimes called baseline models). Such rule-based models often have strong correlations in the scoring metrics in different intervals, so there is a problem of low accuracy.

[0053] In addition, existing machine learning models can usually achieve relatively high accuracy only when the data sample size is large when processing multi-dimensional time series data. When the data sample size is low (for example, for long-tail users or new users with little data), existing machine learning models usually have difficulty producing satisfactory results.

[0054] Attention Neural Process (ANP) is an algorithm that recently emerged to improve the Neural Process (NP). Currently, ANP is usually used in natural language processing and time modeling, but has not been used to process time series data, especially multi-dimensional time series data. This application notices that the ANP model has good applicability to multi-dimensional time series data, especially multi-dimensional time series data in online trading activities (such as for anomaly recognition), and accordingly designs a solution to use the ANP process to model multi-dimensional time series data. For more details about ANP, reference can be made to the paper "ATTENTIVE NEURAL PROCESSES" published by Hyunjik Kim et al. in ICLR 2019 in 2019, and the content of this paper is incorporated herein by reference in its entirety.

[0055] Reference Figure 1 , which shows the overall flowchart of method 100 for modeling multi-dimensional time series data according to an embodiment of this specification.

[0056] Method 100 may include: at operation 102, multi-dimensional time series data can be obtained. Time series data is a type of data that usually includes a sequence of data points arranged in chronological order. In one example, the time interval of this group of time series is a constant value (such as 1 second, 5 minutes, 1 hour, 12 hours, 7 days, 1 month, etc.). In another example, the time interval of this group of time series may not be a constant value but may include real-time events. Preferably, the multi-dimensional time series data includes the timestamp of each data point.

[0057] Multi-dimensional time series data is time series data involving data of multiple different dimensions.

[0058] For example, taking the online transaction scenario as an example, over time, events such as user login, user transfer, user transaction, and change of the city where the user is located may occur, and each event forms a data point. For each data point, there may be data in various dimensions. For example, for each transaction, there may be transaction time, transaction amount, transaction subject, merchant name, device identifier of the user client that executes the transaction, transaction location, and so on. There are also some time-related statistical data, such as the number of logins within 3 days, the number of times in the city within a week, the transaction amount within a month, the proportion of historical false transactions, and so on. These data can also be associated with a specific time (such as the time when the statistics are performed).

[0059] The data associated with the user may also include data that does not change over time (or basically does not change) or data that is not related to events. An example is user basic attribute data, such as data on the user's age, gender, city where the user is located, etc. Another example is the user's rights and interests characteristics, such as the amount of the user's rights and interests, types of rights and interests, etc. Yet another example is the user's payment ability characteristics, such as the user's account balance, loan balance, etc. According to needs, these data can also be used as part of the multi-dimensional time series data, and in this case, timestamps (such as the latest sampling timestamp) can be added to these data. In other examples, these data may not be used as part of the multi-dimensional time series data. For example, these data can be provided to the model separately as offline data.

[0060] These data collected as prediction inputs can be processed in subsequent steps to generate multi-dimensional feature data.

[0061] The multi-dimensional time series data may also include data associated with the variable to be predicted, such as whether the current transaction is an illegal transaction, whether the current user is a malicious user, and so on. These data can be processed in subsequent steps to generate label data.

[0062] It should be understood that the above description of the multi-dimensional time series data is only exemplary. Designers can choose any other suitable time series data according to needs.

[0063] The multi-dimensional time series data of the user in multiple dimensions can be obtained, for example, from a data store for storing user data. This data store can be the local storage of the server or server cluster that executes method 100, or it can be a remote storage that can be accessed by the server or server cluster, such as cloud storage.

[0064] This data store can collect multi-dimensional time series data of the user from multiple sources, for example. For example, this data store can obtain the user's account data from the user account server, obtain the user's transaction data from the transaction server, obtain the user's location information from the user's client device, and so on.

[0065] Method 100 may further include: at operation 104, generating a plurality of previous observation points based on the multi-dimensional time series data. For example, each previous observation point may be represented as (xi, yi). Wherein, xi may be the multi-dimensional feature data xi of the observation point, and yi may be the corresponding label data yi of the observation point. The multi-dimensional feature data is a variable used to predict other variables, and the label data refers to the variable to be predicted.

[0066] Preferably, each observation point may correspond to a time indication. In one example, the time indication may be represented by a time point. For example, the time indication may be represented as a timestamp. For example, the time indication may represent data of an event occurring at the current time point, or may represent data of an event occurring during the period between the current time point and the corresponding time point of the previous observation point. For the data type associated with the time point, the data value may be the relevant data of the event occurring at that time point (such as the transaction amount of the current transaction, etc.). For the data type associated with the time period, the data value may be the data statistically obtained during the period between the current time point and the corresponding time point of the previous observation point (such as the total transaction amount within a certain time period).

[0067] In another example, the time indication may be represented by a time period. For example, the time indication may represent a time interval represented by two timestamps. For example, the time indication may represent data of an event occurring within that time interval. For example, if a transaction has occurred during that time interval, the data value may be the amount of that transaction. If multiple transactions have occurred during that transaction interval, the data value may be the total transaction amount of those multiple transactions.

[0068] Assume that the sample data obtained at each time indication is x t , and the feature data of each observation point is represented as xi. In one example, the feature data of each observation point may be the sample data obtained at the time indication t, that is, xi = x t . For example, x1 = x 1 , x2 = x 2 , x3 = x 3 , and so on.

[0069] In another preferred example, the feature data of each observation point may be a tensor of historical data obtained before the time indication t, that is, xi = [x 1 , x 2 , …, x t . For example, x1 = [x 1 , x2 = [x 1 , x 2 , x3 = [x 1 , x 2 , x 3, and so on.

[0070] Preferably, after obtaining time series data of multiple dimensions, the multi-dimensional time series data can be processed to generate previous observation points, and the processing may include preprocessing. For example, data cleaning, data integration, data augmentation, data transformation, etc. can be performed on the time series data of multiple dimensions. Preferably, dimensionality reduction can be performed on the time series data of the multiple dimensions.

[0071] Subsequently, as needed, operations such as feature extraction and feature selection can be performed on the preprocessed data to generate the multiple previous observation points. A reshape operation can also be performed to make the dimensions of the corresponding variables meet the requirements. Other suitable operations known to those skilled in the art can also be performed.

[0072] Method 100 may further include: at operation 106, a subset of the multiple previous observation points (xi, yi) can be used as training data to train a neural process model. For example, after obtaining multiple previous observation points, a part of the multiple previous observation points can be selected as training data. For example, other data can be used as validation data or test data for validation or testing during or after the model training process.

[0073] Reference Figure 2 , which shows a schematic diagram of a neural process model 200 according to an embodiment of the present specification. As Figure 2 shown, the neural process model 200 may include an encoder 202 and a decoder 204.

[0074] The encoder 202 may include two paths: a deterministic path and a latent path. In Figure 2 , the deterministic path is represented by a solid line, and the latent path is represented by a dashed line.

[0075] Generally, the deterministic path includes a deterministic encoder 206 for generating a representation of each input-output pair (xi, yi). In the embodiments of the present specification, there is also a mutual attention module in the deterministic path for generating a single aggregated representation.

[0076] The latent path may include a latent encoder 208 and an average aggregation module (represented by m and ~ in Figure 2 ) for generating a latent variable z.

[0077] The decoder 204 may include a decoding module. In Figure 2 , the decoding module is shown as an MLP (multi-layer perceptron) for generating predictions based on the output from the encoder.

[0078] Next, in combination with Figure 3 , a detailed description will be givenFigure 2 The specific operation mode of the neural process model 200 in Figure 3 , which shows a flowchart of a method 300 for generating predictions of a neural process model according to an embodiment of the present specification. Specifically, the method 300 uses multiple previous observation points (xi, yi) to generate a prediction of the feature data (shown as x* in Figure 2 ) for one observation point (shown as y* in Figure 3 ). Herein, Figure 2 The observation points (xi, yi) in can also be referred to as context points, which can be a subset of the training data (e.g., a batch of training data). x* can also be referred to as the target query. When applying the model to generate a prediction, x* can represent the feature data of the current observation point (i.e., the observation point to be predicted), and the current observation point can come from a real-time data stream, etc.; when training the model, x* can be taken from the previous observation points in the training data. y* can be referred to as the target prediction, which is the prediction generated using this neural process model; when training the model, the target prediction can be compared with the actual label to generate a loss value.

[0079] Specifically, the method 300 may include: at operation 302, passing multiple previous observation points (xi, yi) in the training data into the deterministic encoder 206 in the encoder 202 to generate multiple representations ri. The previous observation points (xi, yi) are, for example, Figure 2 The (x1, y1), (x2, y2), and (x3, y3) shown in . Of course, during training, the passed-in observation points can be a set of previous observation points in a batch, and the neural process model can be iterated over multiple batches to continuously update the neural process model, thereby generating a trained neural process model.

[0080] Specifically, the previous observation points are respectively passed into the deterministic encoder 206 and the latent encoder 208 in the encoder 202.

[0081] In some examples, the deterministic encoder 206 and the latent encoder 208 can adopt an MLP (Multi-Layer Perceptron) model.

[0082] In a preferred example, the deterministic encoder 206 and the latent encoder 208 can adopt a self-attention model. The self-attention model can enable the interaction between different previous observation points (xi, yi) to be reflected. By using the self-attention model, the embodiments of the present specification can reflect the association between different previous observation points, so as to be able to utilize the explicit or implicit correlation between each observation point to generate a prediction.

[0083] The deterministic encoder 206 can generate a corresponding representation ri for each previous observation point (xi, yi). For example, inFigure 2 Among them, the deterministic encoder 206 can generate r1, r2, and r3 for the previous observation points (x1, y1), (x2, y2), and (x3) respectively.

[0084] Method 300 may include: at operation 304, the multiple representations ( Figure 2 r1, r2, and r3 in it), the multi-dimensional feature data xi of the multiple previous observation points ( Figure 2 x1, x2, and x3 in it), and the multi-dimensional feature data of the current observation point (i.e., the target query) x* are input into the mutual attention module 210 to generate a single aggregated representation r* specific to the current observation point.

[0085] Depending on the specific situation, the mutual attention module 210 can adopt any one of the Laplace model, dot product model, and multi-head model. The specific details of these models will not be described herein. However, it should be understood that these models can enable the generated single aggregated representation r* to pay attention to the association between the target query (i.e., the feature data x* of the current observation point) and the feature data of other observation points (such as r1, r2, r3). By paying attention to this association, the present model can reflect the changes in data in the time dimension, so that the present model has high accuracy when processing time series data.

[0086] Method 300 may include: at operation 306, the multiple previous observation points (xi, yi) in the training data are input into the hidden path to generate a hidden variable z. Specifically, first, the multiple previous observation points (xi, yi) are input into the hidden encoder 208 in the encoder 202 to generate multiple hidden encoder outputs. Subsequently, the multiple hidden encoder outputs are input into the average aggregation module for mean-aggregation to generate the hidden variable z.

[0087] Method 300 may further include: at operation 308, the multi-dimensional feature data x* of the current observation point, the representation r* specific to the current observation point, and the hidden variable z are input into the decoder 204 to generate the target prediction y* of the current observation point.

[0088] For example, the decoder can adopt an MLP model. The MLP model is a commonly used model in neural process models. The specific details of the MLP model will not be further described herein.

[0089] After understanding how to use the neural process model 200 to generate a target prediction for a specific observation point, one can know how to train the neural process model 200.

[0090] For example, when training a neural process model, the feature data xi in each previous observation point (xi, yi) in the training data can be sequentially used as the target query x*, and the neural process model can be used to generate the target prediction y* of the previous observation point, and the target prediction y* can be compared with the actual label data yi of the previous observation point to generate a loss value, and the neural process model can be iteratively updated using the loss value. After meeting certain conditions (such as reaching a set number of iterations or the loss value reaching a target threshold), the training can be stopped and a trained neural process model can be generated.

[0091] Of course, the trained neural process model can be verified and tested so as to improve or retrain the neural process model when needed to improve the performance of the neural process model.

[0092] Reference Figure 4 , which shows a flowchart of an example method 400 for prediction based on multi-dimensional time series data according to an embodiment of the present specification.

[0093] As Figure 4 shown, method 400 may include: at operation 402, a multi-dimensional data stream may be monitored to obtain a current observation point, the current observation point including multi-dimensional feature data xt. For example, preferably, a multi-dimensional data stream may be received in real time and the data in the multi-dimensional data stream may be converted into observation points. The current observation point may only include multi-dimensional feature data xt, and the multi-dimensional feature data xt may be used to generate the target prediction of the current observation point.

[0094] Method 400 may include: at operation 404, based on the current observation point, a trained neural process model may be applied for prediction. The neural process model may be trained using a plurality of previous observation points (xi, yi), each previous observation point including multi-dimensional feature data xi and corresponding label data yi. For example, the method described above with reference to Figures 1 to 3 may be used to perform the training of the neural process model to generate a trained neural process model (such as the neural process model 200 of Figure 2 ). During the prediction process, part or all of the plurality of previous observation points (xi, yi) may also be used.

[0095] As described above with reference to Figure 2 shown, the neural process model 200 may include an encoder 202 and a decoder 204. The encoder 204 may include a deterministic path and a latent path. The deterministic path may include a deterministic encoder 206, while the latent path may include a latent encoder 208. In one example, the deterministic encoder 206 and the latent encoder 208 may adopt an MLP model. And in a preferred example, the deterministic encoder 206 and the latent encoder 208 may adopt a self-attention model.

[0096] The deterministic encoder 206 can be used to generate a plurality of encoded representations ri based on a plurality of previous observation points (xi, yi). The encoder 202 may further include a mutual attention module 210. The mutual attention module 210 can assign weights to the plurality of previous observation points based on the association between the multi-dimensional feature data xt of the current observation point and the multi-dimensional feature data xi of one or more previous observation points, for finally generating the target prediction y* of the current observation point. Specifically, the mutual attention module 210 can generate a single aggregated representation r* specific to the current observation point based on the multi-dimensional feature data xt of the current observation point, the multi-dimensional feature data xi of the one or more previous observation points, and the plurality of encoded representations.

[0097] The latent path can generate a latent variable z based on the plurality of previous observation points (xi, yi). The decoder 204 can generate the target prediction y* of the current observation point based on the multi-dimensional feature data xt of the current observation point, the representation r* specific to the current observation point, and the latent variable z.

[0098] As described above, the multi-dimensional time data stream can be the online activity data stream of a user, and the method can be used to detect anomalies in the user's online activities based on the target prediction y*, as described below with reference to Figure 5 as described.

[0099] Reference Figure 5 , which shows a flowchart of an example method 500 for detecting anomalies in online activities according to an embodiment of the present specification. Below, the method 500 will be described in conjunction with Figure 6 the block diagram of an example system 600 for detecting anomalies in online activities.

[0100] The method 500 may include: at operation 502, the online activity data stream of the user may be monitored to obtain a current observation point. This operation may be performed, for example, by Figure 6 the data acquisition module 602. For example, the data acquisition module 602 may receive the online activity data stream in real time. As described above, the online activity data stream may come from various sources, such as a transaction system, a payment system, a user client, etc. The online activity data stream may include various time series data as introduced above.

[0101] In a preferred example, the data acquisition module 602 may also receive an offline data stream. The offline data stream may include, for example, non-real-time data. For example, the offline data stream may include user basic attribute data, payment ability data, etc. as described above. In some other examples, the offline data stream may also include non-real-time time series data. As mentioned above, the offline data stream may also be used by the neural process model.

[0102] After obtaining the data, observation points can be generated based on the obtained data, as described above in step 104 or 402. The operation of generating observation points can be performed by Figure 6 the data acquisition module 602 or the anomaly detection module 604. The data (or observation points) obtained by the data acquisition module 602 can be transmitted to the anomaly detection module 604.

[0103] Method 500 may include: At operation 504, a trained neural process model can be applied based on the current observation point for prediction to obtain a target prediction.

[0104] Method 500 may further include: At operation 506, anomalies in the user's online activities can be detected based on the target prediction. For example, the target prediction can directly indicate the existence of an anomaly in the user's online activities (e.g., the target prediction indicates an abnormal transaction). Alternatively, the target prediction can be compared with a threshold to determine whether there is an anomaly in the user's online activities.

[0105] The above operations of obtaining the target prediction and detecting anomalies can be performed, for example, by Figure 6 the neural process model in the anomaly detection module 604. For example, the anomaly detection module 604 can include a multi-dimensional time series anomaly detection module or a single-dimensional time series anomaly detection module. The multi-dimensional time series anomaly module can adopt, for example, the neural process model as described above. In this case, the operations described above (e.g., in step 404 of method 400) can be used to generate the target prediction based on the current observation point. The single-dimensional time series anomaly detection module can include, for example, a robust time-frequency model, an ESD model, a CNN model, a siamese network model, etc. Any other appropriate model can also be adopted to perform anomaly detection.

[0106] Method 500 may further include: Preferably, at operation 508, after detecting an anomaly, the cause of the anomaly can be automatically determined. For example, this operation can be performed by Figure 6 the attribution analysis module 606. The attribution analysis module 606 can adopt, for example, a contribution degree down-probing model (e.g., a multi-layer contribution degree down-probing model) or a Drillup model, etc., to perform. Any other suitable attribution analysis module can also be adopted to automatically determine the cause of the anomaly.

[0107] Method 500 may further include: Preferably, at operation 510, an alarm message can be output. For example, a rule engine can be used to output the alarm message based on alarm rules. For example, the rule engine can determine whether to execute an alarm based on a predetermined rule, and how to execute the alarm, etc.

[0108] The method of the present invention skillfully utilizes the consistency and correlation of user behaviors in different time frames by applying a mutual attention model, and at the same time, by applying a self-attention model, it skillfully utilizes the correlation between different-dimensional data within the same time frame, thereby leveraging the internal associations of time series data in both the time and user dimensions and improving the accuracy of model recognition.

[0109] In addition, the neural process model described in this article is particularly suitable for anomaly detection in cases where the amount of historical data is small or there is no historical data because it can fully utilize time series data in multiple dimensions and the associations between various observation points.

[0110] Reference Figure 7 , which shows a schematic block diagram of a device 700 that can be used to execute the method described above according to an embodiment of this specification.

[0111] The device may include a processor 710 and a memory 715. The processor is configured to execute any of the methods described above. The memory may store, for example, the acquired data (such as the online data stream or offline data stream described above). The processor may execute any of the methods described above.

[0112] The device may include a network connection device 725, which may include, for example, a network connection device that connects to other devices (such as a server can connect to a user's client and / or other servers, and a client can connect to a server) through a wired connection or a wireless connection. The wireless connection may be, for example, a WiFi connection, a Bluetooth connection, a 3G / 4G / 5G network connection, etc.

[0113] The device may further include other peripheral components 720, such as a keyboard and a mouse, etc.

[0114] Each of these modules may communicate directly or indirectly with each other, for example, via one or more buses (such as bus 705).

[0115] Moreover, this application also discloses a computer-readable storage medium storing computer-executable instructions thereon. When the computer-executable instructions are executed by a processor, the processor is caused to execute the methods of the various embodiments described herein.

[0116] In addition, this application also discloses a device that includes a processor and a memory storing computer-executable instructions. When the computer-executable instructions are executed by the processor, the processor is caused to execute the methods of the various embodiments described herein.

[0117] In addition, this application also discloses a system that includes a device for implementing the methods of the various embodiments described herein.

[0118] It will be understood that the methods according to one or more embodiments of the present specification can be implemented in software, firmware, or a combination thereof.

[0119] It should be understood that the various embodiments in this specification are described in a progressive manner, and for the same or similar parts between the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the description of the method embodiments.

[0120] It should be understood that the specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] It should be understood that an element described herein in the singular form or shown as only one in the drawings does not represent limiting the quantity of that element to one. Additionally, a module or element described or shown herein as separate may be combined into a single module or element, and a module or element described or shown herein as a single one may be split into multiple modules or elements.

[0122] It should also be understood that the terms and expressions used herein are for descriptive purposes only, and one or more embodiments of the present specification should not be limited to these terms and expressions. Using these terms and expressions does not mean excluding any equivalent features of the illustration and description (or parts thereof), and it should be recognized that various modifications that may exist should also be included within the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be regarded as covering all such equivalents.

[0123] Similarly, it should be noted that although reference has been made to the current specific embodiments for description, those of ordinary skill in the art in this technical field should recognize that the above embodiments are only used to illustrate one or more embodiments of the present specification, and various equivalent changes or substitutions can be made without departing from the spirit of the present invention. Therefore, as long as the changes and modifications to the above embodiments are within the scope of the spirit of the present invention, they will fall within the scope of the claims of this application.

Claims

1. A method for prediction based on multi-dimensional time series data, comprising: Monitoring a multi-dimensional time series data stream to obtain a current observation point, the current observation point including multi-dimensional feature data xt, wherein the multi-dimensional time series data stream is an online activity data stream of a user; Based on the current observation point, applying a trained neural process model for prediction, the neural process model being trained using a plurality of previous observation points (xi, yi), each previous observation point including multi-dimensional feature data xi and corresponding label data yi, wherein the neural process model includes an encoder and a decoder, the encoder including a mutual attention module, the mutual attention module assigning weights to the plurality of previous observation points based on the association between the multi-dimensional feature data xt of the current observation point and the multi-dimensional feature data xi of the plurality of previous observation points for ultimately generating a target prediction y* of the current observation point, wherein the encoder includes a deterministic path and a latent path, wherein the deterministic path includes a deterministic encoder, the deterministic encoder being configured to generate a plurality of encoded representations ri based on the plurality of previous observation points (xi, yi), the mutual attention module generating a single aggregated representation r* specific to the current observation point based on the multi-dimensional feature data xt of the current observation point, the multi-dimensional feature data xi of the plurality of previous observation points, and the plurality of encoded representations, wherein the latent path generates a latent variable z based on the plurality of previous observation points (xi, yi), and the decoder generates the target prediction y* of the current observation point based on the multi-dimensional feature data xt of the current observation point, the representation r* specific to the current observation point, and the latent variable z.

2. The method according to claim 1, wherein the deterministic encoder uses a self-attention model.

3. The method according to claim 1, wherein the method comprises: Detecting an anomaly in the online activity of the user based on the target prediction y*.

4. The method according to claim 3, wherein the method further comprises: Obtaining offline data, wherein detecting the anomaly in the online activity of the user is further based on the offline data.

5. The method according to claim 3, wherein the method further comprises: Providing a one-dimensional time series anomaly detection module, and Using the one-dimensional time series anomaly detection module together with the neural process model to detect the anomaly in the online activity of the user.

6. The method according to claim 3, wherein the method further comprises: After detecting an anomaly, automatically determining the cause of the anomaly using an attribution module.

7. The method according to claim 3, wherein the method further comprises: After detecting an anomaly, outputting an alarm message using an alarm module.

8. A method for training a model based on multi-dimensional time series data, comprising: Obtaining multi-dimensional time series data, wherein the multi-dimensional time series data is online activity data of a user; Generating a plurality of previous observation points (xi, yi) based on the multi-dimensional time series data, each observation point corresponding to a time indication and including multi-dimensional feature data xi and corresponding label data yi; Training a neural process model using a subset of the plurality of previous observation points (xi, yi) as training data, wherein the neural process model includes an encoder and a decoder, the encoder includes a mutual attention module, and the mutual attention module is configured to: assign weights to the plurality of previous observation points for each observation point based on the association between the multi-dimensional feature data xi of this observation point and the multi-dimensional feature data xi of other observation points in the subset for finally generating the target prediction yip of this observation point, wherein the target prediction yip is used to generate a loss value with the label data yi of this observation point, iteratively adjusting the neural process model based on the loss value to generate a trained model, wherein the encoder includes a deterministic path and a latent path, the deterministic path includes a deterministic encoder, the deterministic encoder is used to generate a plurality of encoded representations ri based on the subset, the mutual attention module generates a single aggregated representation r specific to this observation point based on the multi-dimensional feature data xi of this observation point, the multi-dimensional feature data xi of the previous observation points in the subset, and the plurality of encoded representations, the latent path generates a latent variable z based on the subset, and the decoder generates the target prediction yip of this observation point based on the multi-dimensional feature data xi specific to this observation point, the representation r specific to this observation point, and the latent variable z.

9. The method according to claim 8, wherein the deterministic encoder uses a self-attention model.

10. The method according to claim 8, wherein the method further comprises: Calculating a loss value based on the target prediction yip and the label data yi of one or more observation points; And Iteratively updating the neural process model based on the loss value so as to generate a trained neural process model.

11. A system for detecting anomalies in online activities, comprising: A data acquisition module configured to: monitor a data stream of a user's online activities to obtain a current observation point, the current observation point including multi-dimensional feature data xt; A multi-dimensional time series anomaly detection module configured to: Based on the current observation point, a trained neural process model is applied for prediction. The neural process model is trained using multiple previous observation points (xi, yi), where each previous observation point includes multi-dimensional feature data xi and corresponding label data yi. The neural process model includes an encoder and a decoder. The encoder includes a cross-attention module that assigns weights to the multiple previous observation points based on the association between the multi-dimensional feature data xt of the current observation point and the multi-dimensional feature data xi of the multiple previous observation points for finally generating the target prediction y* of the current observation point. The encoder includes a deterministic path and a latent path. The deterministic path includes a deterministic encoder that is used to generate multiple encoded representations ri based on the multiple previous observation points (xi, yi). The cross-attention module generates a single aggregated representation r* specific to the current observation point based on the multi-dimensional feature data xt of the current observation point, the multi-dimensional feature data xi of the multiple previous observation points, and the multiple encoded representations. The latent path generates a latent variable z based on the multiple previous observation points (xi, yi). The decoder generates the target prediction y* of the current observation point based on the multi-dimensional feature data xt of the current observation point, the representation r* specific to the current observation point, and the latent variable z; and Detect an anomaly in the user's online activity based on the target prediction y*.

12. The system according to claim 11, wherein the data acquisition module is further configured to acquire offline data, and detecting an anomaly in the user's online activity is further based on the offline data.

13. The system according to claim 11, wherein the system further includes a one-dimensional time series anomaly detection module, and the one-dimensional time series anomaly detection module is configured to detect an anomaly in the user's online activity together with the multi-dimensional time series anomaly detection module.

14. The system according to claim 11, the system further includes an attribution module, and the attribution module is configured to automatically determine the cause of the anomaly using the attribution module.

15. The system according to claim 11, wherein the system further includes an alarm module, and the alarm module is configured to output alarm information after detecting an anomaly.

16. An apparatus for generating a model for a user, comprising: A memory; And A processor configured to execute the method according to any one of claims 1-7.

17. An apparatus for generating a model for a user, comprising: A memory; And A processor configured to execute the method according to any one of claims 8-10.

18. A computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to execute the method according to any one of claims 1-7.

19. A computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to execute the method according to any one of claims 8-10.

Images