Abnormality detection method and device, equipment, storage medium and program product

Abstract

The invention provides an anomaly detection method and device, equipment, a storage medium and a program product, and relates to the field of artificial intelligence. According to the method, a transaction log of a target object is obtained, the transaction log comprises a plurality of transactions, and each transaction comprises a plurality of record items and value information corresponding to each record item; according to the method, firstly, based on an association rule mining algorithm, record item information in a transaction log is mined to obtain a plurality of frequent item set combinations, a single frequent item set combination comprises at least one frequent item set, the data size needing to be processed by a subsequent model is reduced, and the detection efficiency is improved; according to the multiple frequent item set combinations and the timestamps in the transaction logs, a frequent item set combination sequence is constructed, the frequent item set combination sequence indicates the dynamic relation of the frequent item set combinations changing along with time, and the dynamic relation is accurately recognized through a time sequence model; and the abnormity existing in the transaction log is accurately detected according to the dynamic relationship, so that the detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an anomaly detection method, apparatus, device, storage medium, and program product. Background Art

[0002] Anomaly data detection is used to identify patterns in data that differ from the majority of data points. It is often used to detect potential errors, fraud, failures, or security threats, and is widely used in finance, fault monitoring, and network security. Current traditional methods rely on manually defined rules or statistical analysis, but they struggle to cope with complex data patterns and are prone to misjudgments and missed detections. Artificial intelligence and machine learning technologies have improved detection accuracy, but they also face challenges with computational complexity and efficiency. Summary of the Invention

[0003] The present application provides an anomaly detection method, apparatus, device, storage medium, and program product to improve the accuracy and efficiency of abnormal data detection.

[0004] In a first aspect, the present application provides an anomaly detection method, comprising:

[0005] Obtaining a transaction log of a target object; wherein the transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp;

[0006] Based on an association rule mining algorithm, mining the record item information in the transaction log to obtain a plurality of frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set;

[0007] Constructing a frequent item set combination sequence according to the multiple frequent item set combinations and the timestamps in the transaction log; wherein the frequent item set combination sequence is used to indicate the dynamic relationship of the frequent item set combinations changing over time;

[0008] Based on the time series model and according to the frequent item set combination sequence, anomaly detection is performed on the transaction log of the target object to determine an anomaly detection result.

[0009] In a second aspect, the present application provides an anomaly detection device, comprising:

[0010] An acquisition module, configured to acquire a transaction log of a target object; wherein the transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp;

[0011] A mining module, configured to mine the record information in the transaction log based on an association rule mining algorithm to obtain a plurality of frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set;

[0012] A construction module, configured to construct a frequent item set combination sequence according to the multiple frequent item set combinations and the timestamps in the transaction log; wherein the frequent item set combination sequence is used to indicate a dynamic relationship of the frequent item set combinations changing over time;

[0013] The detection module is configured to perform anomaly detection on the transaction log of the target object based on a time series model and according to the frequent item set combination sequence, and determine anomaly detection results.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0015] The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0017] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0018] The present application provides an anomaly detection method, apparatus, device, storage medium and program product. The method obtains a transaction log of a target object, wherein the transaction log includes multiple transactions, the transactions include multiple record items and value information corresponding to each record item, and the transactions carry a timestamp; first, based on an association rule mining algorithm, the record item information in the transaction log is mined to obtain multiple frequent item set combinations, wherein a single frequent item set combination includes at least one frequent item set, which reduces the amount of data that needs to be processed by a subsequent model and improves detection efficiency; then, based on the multiple frequent item set combinations and the timestamps in the transaction log, a frequent item set combination sequence is constructed, wherein the frequent item set combination sequence indicates the dynamic relationship of the frequent item set combinations over time, uses a time series model to accurately identify this dynamic relationship, and accurately detects anomalies in the transaction log based on this dynamic relationship, thereby improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] Figure 1 A flowchart of an anomaly detection method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of another anomaly detection method provided in an embodiment of the present application;

[0022] Figure 3 A structural diagram of a Bi-LSTM provided in an embodiment of the present application;

[0023] Figure 4 A structural diagram of an LSTM unit provided in an embodiment of the present application;

[0024] Figure 5 An overall structural diagram of an anomaly detection method is provided for an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of an anomaly detection device provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0027] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0031] It should be noted that the anomaly detection method, apparatus, equipment, storage medium and program product provided in this application can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence. This application does not limit the application field of the anomaly detection method, apparatus, equipment, storage medium and program product.

[0032] Anomaly data detection is a technique for identifying patterns or behaviors in a dataset that differ from most other data points. It is often used to uncover potential errors, fraud, faults, or other security threats within the data. It has widespread application in many fields, such as finance, fault monitoring, and network security. For example, in the financial industry, anomaly detection of bank user transaction data can identify anomalous transaction data. Based on this anomalous transaction data, relevant technicians can analyze user transaction behavior and determine whether the user has engaged in unusual trading activities, such as fraud, attempts to legalize illegal gains, or account theft. Similarly, in fault monitoring, anomaly detection of device performance data can identify anomalous performance data. Based on this anomalous transaction data, relevant technicians can locate or analyze the cause of the anomaly. Across various fields, efficient and accurate anomaly data detection can improve the reliability and security of systems within those fields and reduce potential losses.

[0033] Among current methods for detecting abnormal data, traditional approaches often rely on manually defined rules (such as thresholds, ranges, etc.) to determine whether data is abnormal. While this approach is simple and easy to implement, it struggles to adapt to complex or changing data patterns. Manually defined rules often struggle to cover all potential abnormal behaviors, especially when the data volume is large and diverse. The limited coverage of the rules may not capture potential complex anomalies in a timely manner. Statistical methods (such as those based on mean and standard deviation) cannot adapt to the nonlinearity and heterogeneity of data. Due to the simplification of rules and models, these traditional and statistical methods can lead to misjudgments (normal data being identified as abnormal) and missed detections (failure to identify truly abnormal data). Methods based on artificial intelligence or other advanced data analysis technologies often face high computational complexity, challenging the efficiency and speed of abnormal data detection.

[0034] In order to solve the above technical problems, an embodiment of the present application proposes an anomaly detection method, which obtains a transaction log of a target object, wherein the transaction log includes multiple transactions, the transactions include multiple record items and value information corresponding to each record item, and the transactions carry a timestamp; first, based on an association rule mining algorithm, the record item information in the transaction log is mined to obtain multiple frequent item set combinations, wherein a single frequent item set combination includes at least one frequent item set, which reduces the amount of data that needs to be processed by subsequent models and improves detection efficiency; then, based on the multiple frequent item set combinations and the timestamps in the transaction log, a frequent item set combination sequence is constructed, wherein the frequent item set combination sequence indicates the dynamic relationship of the frequent item set combination over time, uses a time series model to accurately identify this dynamic relationship, and accurately detects anomalies in the transaction log based on this dynamic relationship, thereby improving detection accuracy.

[0035] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0036] Figure 1 This is a flow chart of an anomaly detection method provided in an embodiment of the present application. The execution subject of this method can be an electronic device for implementing the anomaly detection method, which is implemented through software and hardware, and can specifically be a server deployed locally or in the cloud. The specific steps of this method are as follows:

[0037] S101. Obtain a transaction log of a target object. The transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp.

[0038] The target object refers to an object whose transaction log needs to be detected for anomalies, and may be a bank user, a physical device, etc., which is not limited in this embodiment.

[0039] A transaction log refers to log data that records the record information of a target object at each moment in chronological order. Each log data entry is recorded at each moment, and each log data entry is considered a transaction. A transaction includes multiple record items and the corresponding value information for each record item. The recording time of a transaction is the timestamp carried by the transaction. The record items obtained can be set according to actual application needs and are not limited in this embodiment.

[0040] For example, for a bank user, a transaction log may be the user's transaction log in the bank's transaction database. In the transaction log, multiple transaction information items (i.e., record items) of the user in the transaction are recorded in chronological order, such as transaction amount, transaction type, transaction account, transaction location, transaction device, and transaction status.

[0041] For example, for a physical device, a transaction log can be a monitoring log of the physical device. In the monitoring log, multiple monitoring information items (i.e., record items) of the physical device during operation are recorded in chronological order, such as the central processing unit (CPU) usage, memory usage, disk usage, and network traffic.

[0042] S102 : Mining the record item information in the transaction log based on an association rule mining algorithm to obtain a plurality of frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set.

[0043] The association rule mining algorithm may be any algorithm for mining data and obtaining frequent itemsets corresponding to the data, which is not limited in this embodiment, such as the Apriori algorithm, the FP-growth (Frequent Pattern Growth) algorithm, and the like.

[0044] In association rule mining algorithms, an itemset is a collection of record item values. The support of an itemset refers to the frequency of occurrence of the itemset in a transaction log. Specifically, for any itemset, the ratio of the number of transactions containing the itemset to the total number of transactions is used as the support of the itemset.

[0045] For example, for a transaction log, when the record item is a transaction type, if the transaction type includes category one and category two, that is, the value information of the transaction type includes deposits and withdrawals, then the set composed of the value information of the transaction type, that is, the item set, includes the following three: {deposit}, {withdrawal}, and {deposit, withdrawal}.

[0046] Assume that a total of 10 transactions are recorded in the transaction log, among which the number of transactions with {deposit} is 4, the number of transactions with {withdrawal} is 6, and the number of transactions with {deposit, withdrawal} is 0. Then the support of the item set {deposit} is the ratio of the number of transactions with {deposit} (4) to the total number of transactions (10), that is, 2 / 5; the support of the item set {withdrawal} is the ratio of the number of transactions with {deposit} (6) to the total number of transactions (10), that is, 3 / 5; the support of the item set {deposit, withdrawal} is the ratio of the number of transactions with {deposit, withdrawal} (6) to the total number of transactions (10), that is, 0.

[0047] For any itemset, if the support of the itemset is greater than or equal to the minimum support threshold, it indicates that the record information in the itemset appears frequently in all transactions in the transaction log, and the itemset can be determined as a frequent itemset. The minimum support threshold can be set according to actual application needs and is not limited in this embodiment.

[0048] In this step, a time window is used to slide the transaction log across the time dimension, resulting in multiple sub-transaction logs. For each sub-transaction log, an association rule mining algorithm is used to mine the sub-transaction log's record information, yielding one or more frequent item sets. These one or more frequent item sets are then used as the frequent item set combinations corresponding to the sub-transaction log. This yields multiple frequent item set combinations. These frequent item set combinations reflect the frequent patterns of record item information within different time windows—that is, which frequent item sets exist in the sub-transaction data within a given time window? These frequent item sets reflect which record item information appears frequently.

[0049] The time window size and sliding step size can be fixed or variable, and can be set according to actual application needs. This embodiment does not limit this. It should be noted that there is no necessary connection between the time window size and the sliding step size. That is, when using a larger time window, a smaller sliding step size can be used; when using a smaller time window, a larger sliding step size can also be used.

[0050] For example, for transaction logs, use a larger time window and sliding step size during periods with less user-generated transaction data, and a smaller time window and sliding step size during periods with more user-generated transaction data. For example, for physical device monitoring logs, use a smaller time window during periods with more frequent physical device use, and a larger time window with the same sliding step size during periods with less frequent physical device use.

[0051] S103 , constructing a frequent itemset combination sequence according to the multiple frequent itemset combinations and the timestamps in the transaction log; wherein the frequent itemset combination sequence is used to indicate the dynamic relationship of the frequent itemset combinations changing over time.

[0052] In this step, for any frequent item set combination, a timestamp is selected from the transaction log corresponding to the frequent item set combination, or the average of all timestamps is calculated as the timestamp corresponding to the frequent item set combination; then, according to the order of the timestamps corresponding to each frequent item set combination, each frequent item set combination is arranged into a sequence to obtain a frequent item set combination sequence.

[0053] In this embodiment, a frequent itemset combination includes multiple frequent itemsets corresponding to a certain time window. Frequent itemsets reflect which records frequently appear together. The frequent itemset combination sequence can reflect the changing trend of the frequent itemset combination over time. For example, at different moments in time, which itemsets are frequent, which itemsets change from frequent to infrequent, and which itemsets change from infrequent to frequent. From the temporal trend of the frequent itemset combination sequence, it is possible to determine which changes are normal and which are abnormal.

[0054] S104 : Based on the time series model and according to the frequent item set combination sequence, anomaly detection is performed on the transaction log of the target object to determine anomaly detection results.

[0055] The time series model can be trained using any basic model for processing time series. This embodiment does not limit the basic model. For example, the basic model can be a bidirectional long short-term memory (Bi-LSTM) network, a Transformer, or the like. The time series model can be obtained by training the basic model using a sequence of frequent item sets and corresponding anomaly labels.

[0056] In this step, the frequent item set combination sequence is input into the time series model. The time series model identifies the relationship in the frequent item set combination sequence that indicates the change of frequent item set combinations over time, thereby performing anomaly detection on the transaction log of the target object and obtaining the anomaly detection results of each frequent item set combination in the frequent item set combination.

[0057] If the transaction log corresponding to a certain frequent itemset combination in the frequent itemset combination sequence is normal, the anomaly detection result of the frequent itemset combination is normal. If the transaction log corresponding to a certain frequent itemset combination in the frequent itemset combination sequence is abnormal, the anomaly detection result of the frequent itemset combination is abnormal.

[0058] In an embodiment of the present application, a transaction log of a target object is obtained, wherein the transaction log includes multiple transactions, the transactions include multiple record items and value information corresponding to each record item, and the transactions carry a timestamp; first, based on an association rule mining algorithm, the record item information in the transaction log is mined to obtain multiple frequent item set combinations, wherein a single frequent item set combination includes at least one frequent item set, which reduces the amount of data that needs to be processed by subsequent models and improves detection efficiency; then, based on the multiple frequent item set combinations and the timestamps in the transaction log, a frequent item set combination sequence is constructed, wherein the frequent item set combination sequence indicates the dynamic relationship of the frequent item set combinations over time, and a time series model is used to accurately identify this dynamic relationship, and based on this dynamic relationship, anomalies in the transaction log are accurately detected, thereby improving detection accuracy.

[0059] Figure 2 This is a flow chart of another anomaly detection method provided in an embodiment of the present application. Figure 2 As shown, the specific implementation steps of this method are as follows:

[0060] S201. Obtain a transaction log of a target object; wherein the transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp.

[0061] The specific implementation of this step is similar to the implementation principle of the aforementioned S101 and will not be repeated here.

[0062] S202: Based on the timestamps carried by each transaction in the transaction log, use a time window of a preset size to slide extract the transaction log in the time dimension to obtain multiple sub-transaction logs; wherein the sub-transaction log includes at least one transaction.

[0063] The preset size can be set according to actual application needs, and this embodiment does not limit this.

[0064] In this step, each transaction in the transaction log is arranged in the order of timestamps. A preset large time window is used to slide on the timestamp to extract the transactions within the time window. The transactions within a time window are treated as a sub-transaction log, thereby obtaining multiple sub-transaction logs.

[0065] S203 , for each sub-transaction log, mining the record information of the sub-transaction log based on the association rule mining algorithm to obtain the frequent item set combinations corresponding to the sub-transaction log, and taking the frequent item set combinations corresponding to each sub-transaction log as multiple frequent item set combinations.

[0066] The association rule mining algorithm may be any algorithm for mining data and obtaining frequent itemsets corresponding to the data, which is not limited in this embodiment, such as the Apriori algorithm, the FP-growth (Frequent Pattern Growth) algorithm, and the like.

[0067] The following uses the Apriori algorithm as an example to explain in detail how to mine the record information of sub-transaction logs.

[0068] Specifically, the Apriori algorithm is based on the principle of "prior knowledge" and, according to the following properties: "If an item set is frequent, then all its subsets are also frequent" and "If an item set is infrequent, then all its supersets are also infrequent item sets", it mines the record item information of the sub-transaction log and generates frequent item sets.

[0069] First, the Apriori algorithm generates a candidate 1-item set for the sub-transaction log. Specifically, each different record item information in the sub-transaction log is used as a different candidate 1-item set. For example, a sub-transaction log of a bank user contains two transactions, and each transaction includes two record items: transaction type and transaction amount. Among them, the record item information of the first transaction is: deposit, 100 yuan. The record item information of the second transaction is withdrawal, 100.1 yuan. Then the candidate 1-item set is: {deposit}, {withdrawal}, (100 yuan), {100.1 yuan}. The support of the candidate 1-item set is calculated, and the candidate 1-item set with support greater than or equal to the minimum support threshold is used as a frequent 1-item set. Among them, the minimum support threshold can be set according to the needs of the actual application, and this embodiment does not limit this.

[0070] Then, a candidate 2-item set is generated for the sub-transaction log. Specifically, a set consisting of any two items of different record information from all record information is a candidate 2-item set. Based on the properties "if an item set is frequent, then all its subsets are also frequent" and "if an item set is infrequent, then all its supersets are also infrequent item sets," we determine which candidate 2-item sets are frequent and which are infrequent. For example, if the 1-item set {A} is frequent, {B} is frequent, and {C} is infrequent, then the 2-item set {A, B} is frequent and {A, C} is infrequent.

[0071] This process is repeated until three candidate sets of items are generated and the three frequent items are determined based on their properties until no higher-order candidate sets can be generated. Finally, the frequent item sets of all orders are used as the frequent item set combinations corresponding to the sub-transaction logs. Different sub-transaction logs correspond to different frequent item set combinations.

[0072] Optionally, for each sub-transaction log, if there is a record item in the sub-transaction log whose value range is a continuous numerical space, the value information of the record item is converted into discrete data to obtain a converted sub-transaction log; based on the association rule mining algorithm, the record item information of the converted sub-transaction log is mined to obtain a frequent item set combination corresponding to the sub-transaction log; wherein, the elements in the frequent item set are the value information of the record item.

[0073] In practical applications, the value range of some record items is a continuous numerical space, resulting in an excessive number of different record items and a large amount of computation. For example, a transaction amount can be accurate to two decimal places, but 100.11 yuan and 100.12 yuan are different record items. When generating candidate itemsets, they will be treated as different candidate itemsets. This means that when determining frequent itemsets, the support of each candidate item set must be calculated. When generating higher-order itemsets, a large number of candidate sets will also be generated, resulting in a large amount of computation.

[0074] To reduce the amount of computation, the record information of record items with a value range of continuous data space can be converted into discrete data. For example, when the transaction amount is between 0 and 1,000 yuan, the transaction amount value information is low; when the transaction amount is between 1,001 and 10,000 yuan, the transaction amount value information is medium; when the transaction amount is greater than 10,000 yuan, the transaction amount value information is high.

[0075] In this embodiment, by converting the record item information of the record item whose value range is a continuous data space into discrete data, the amount of data mining calculation is reduced and the data mining efficiency is improved.

[0076] In this embodiment, the transaction log of the target object is extracted through a time window of preset size, and the transaction logs within each time window are mined to obtain the frequent item set combinations corresponding to each sub-transaction log. The frequent item set combinations are evenly distributed in the time dimension, which can help ensure that the frequent item set combinations in different time periods are stable, consistent and representative, and are not biased towards a specific time period.

[0077] S204 , determining a reference timestamp corresponding to each frequent itemset combination; wherein the reference timestamp corresponding to any frequent itemset combination is the first timestamp in the sub-transaction log corresponding to the frequent itemset combination.

[0078] In this step, for any frequent itemset combination, the timestamp of the first transaction in the sub-transaction log corresponding to the frequent itemset combination is used as the reference timestamp of the frequent itemset combination.

[0079] S205 , sorting the multiple frequent itemset combinations according to the reference timestamps corresponding to the respective frequent itemset combinations to obtain a frequent itemset combination sequence.

[0080] In this step, each frequent item set combination is arranged in the order of the reference timestamps of each frequent item set combination to obtain a frequent item set combination sequence.

[0081] In this embodiment, the timestamp of the first transaction in the sub-transaction log corresponding to the frequent itemset combination is used as the base timestamp of the frequent itemset combination, and then each frequent itemset combination is sorted in the order of the base timestamps to obtain a frequent itemset combination sequence. The frequent itemset combination is anchored to the real business timeline, and the original temporal dependency between the frequent itemset combinations is retained, which can accurately reflect the dynamic relationship of the frequent itemset combinations over time.

[0082] S206 , encoding the frequent item set combination sequence to obtain an encoding sequence corresponding to the frequent item set combination sequence.

[0083] In this step, any encoding method for converting a non-numeric sequence into a coded sequence that can be processed by a time series model can be used to encode the frequent itemset combination sequence, which is not limited in this embodiment. For example, the encoding method can be one-hot encoding, hash encoding, word embedding, set encoding, etc.

[0084] For example, the encoding method can be one-hot encoding. Specifically, for any frequent item set combination, any frequent item set in the frequent item set combination is mapped into a unique binary vector; then, the binary vectors of all frequent item sets are concatenated into one vector to obtain the vector corresponding to the frequent item set combination; finally, the vectors corresponding to each frequent item set combination are arranged in the order of the frequent item set combinations to obtain the encoding sequence corresponding to the frequent item set combination sequence.

[0085] For example, the encoding method can also be hash encoding. Specifically, for any frequent item set combination, a hash function is used to map any frequent item set in the frequent item set combination to a low-dimensional vector of fixed length; then, the low-dimensional vectors of all frequent item sets are concatenated into one vector to obtain the vector corresponding to the frequent item set combination; finally, the vectors corresponding to each frequent item set combination are arranged in the order of the frequent item set combination to obtain the encoding sequence corresponding to the frequent item set combination sequence.

[0086] S207. Based on the time series model, perform anomaly detection on the coding sequence to obtain anomaly labels corresponding to each time step in the coding sequence; wherein the anomaly labels are used to indicate anomalies of the sub-transaction log corresponding to the frequent itemset combination corresponding to the time step, and the anomaly labels include normal and abnormal.

[0087] In this step, the code sequence is input into the time series model, which processes the code sequence to obtain anomaly labels corresponding to each time step in the code sequence. If the transaction log corresponding to a time step in the code sequence is normal, the anomaly detection result for that time step is normal. If the transaction log corresponding to a time step in the code sequence is abnormal, the anomaly detection result for that time step is abnormal.

[0088] In this embodiment, by encoding the frequent item set combination sequence, the frequent item set combination sequence can be better processed by the time series model, thereby improving the anomaly detection accuracy of the time series model.

[0089] Optionally, if the exception label corresponding to any time step in the coding sequence indicates that the abnormal situation of the sub-transaction log corresponding to the time step is abnormal, a prompt information is output; wherein the prompt information is used to indicate that there is an abnormality in the target object, and the prompt information includes the sub-transaction log corresponding to the frequent item set combination corresponding to the time step.

[0090] Specifically, when the anomaly detection result corresponding to a certain time step is abnormal, the anomaly label is output, and the record information and timestamp of the transaction in the sub-transaction log corresponding to the time step are output to prompt relevant staff or technical personnel to further review the data with abnormal detection results or take corresponding measures and processing.

[0091] Figure 3 This is a structural diagram of a Bi-LSTM provided in an embodiment of the present application. Figure 3 As shown in the figure, the Bi-LSTM consists of an input layer, a forward Long Short-Term Memory (LSTM) layer, a backward LSTM layer, and an output layer. The input layer is the time series that needs to be processed by the Bi-LSTM. Figure 3 middle, ~ is the weight.

[0092] Figure 4 A structural diagram of an LSTM unit provided in an embodiment of the present application. Figure 4 In the equation, t represents the current time. Represents the output of the output gate at the previous moment (i.e., moment t-1), is the data information status of the previous moment (i.e. memory information), is the output of the forget gate at the current moment, is the output of the input gate at the current moment, is the data information status at the current moment, Represents information activated by the tanh function, is the output of the output gate at the current moment, is the state of the output gate at the current moment. Specifically, after the input time series is input into the LSTM, its calculation process is shown in the following formulas (1)-(6):

[0093] (1)

[0094] (2)

[0095] (3)

[0096] (4)

[0097] (5)

[0098] (6)

[0099] in, 、 、 、 is the weight matrix, 、 、 、 For bias.

[0100] In an optional implementation, the time series model in the aforementioned embodiment can be a bidirectional long short-term memory (Bi-LSTM) network. The Bi-LSTM includes a forward long short-term memory (LSTM) layer, a reverse long short-term memory (LSTM) layer, and an output layer. Based on the time series model, anomalies in the target object's transaction log are detected according to the frequent itemset combination sequence, and the anomaly detection result is determined. This can be implemented as follows:

[0101] The frequent item set combination sequence is encoded to obtain an encoding sequence of the frequent item set combination sequence.

[0102] The specific implementation principle is the same as that of the aforementioned S206 and will not be repeated here.

[0103] The coding sequence is input into the forward long short-term memory network layer, and the forward long short-term memory network layer processes the data corresponding to each time step in the coding sequence in order from front to back to obtain the first eigenvector.

[0104] Specifically, the eigenvalue of any feature in the first eigenvector is calculated as follows: Figure 3 As shown, the forward long short-term memory network layer forward calculates the output of the encoding sequence from the first time step to the last time step. The calculation process is shown in the following formula (7):

[0105] (7)

[0106] in, Indicates the state of the output gate at time t, and t-1 indicates the previous time.

[0107] The coding sequence is input into the reverse long short-term memory network layer, and the reverse long short-term memory network layer processes the data corresponding to each time step in the coding sequence in order from back to front to obtain the second eigenvector.

[0108] Specifically, the eigenvalue of any feature in the second eigenvector is calculated as follows: Figure 3 As shown, the reverse long short-term memory network layer forward calculates the output of the encoding sequence from the last time step to the first time step. The calculation process is shown in the following formula (8):

[0109] (8)

[0110] in, represents the output gate state at time 1, and t+1 represents the next time.

[0111] According to the first eigenvector and the second eigenvector, the output layer outputs an anomaly detection result corresponding to the transaction log of the target object.

[0112] Specifically, the first eigenvector and the second eigenvector are input to the output layer, and the first eigenvector and the second eigenvector of the output layer are comprehensively calculated to obtain the final eigenvalue. The eigenvalue calculation is shown in formula (9):

[0113] (9)

[0114] in, is the output of the output gate.

[0115] Furthermore, the output layer also classifies each eigenvalue in the final eigenvector to obtain each eigenvalue, that is, the abnormal label corresponding to each time step.

[0116] In this embodiment, the forward LSTM layer and the reverse LSTM layer are used to process the encoded sequence from the forward direction and the reverse direction of the encoded sequence respectively. On the basis of LSTM, a reverse calculation path is introduced. Bi-LSTM can better capture the dynamic relationship of frequent item set combinations over time, thereby improving the accuracy of anomaly detection.

[0117] Figure 5 The overall structure diagram of an abnormality detection method is provided for the embodiment of the present application. Figure 5As shown in the figure, the anomaly detection method first takes the transaction log of the target object as the input of the association rule mining algorithm. The association rule mining algorithm mines the transaction log to obtain a frequent item set combination sequence; the frequent item set combination sequence is then input into the bidirectional long short-term memory network for processing. The bidirectional long short-term memory network identifies the abnormal data in the transaction log according to the frequent item set combination sequence, and finally outputs the abnormal label corresponding to the sub-transaction log in the transaction log.

[0118] Figure 6 This is a schematic diagram of the structure of an abnormality detection device provided in an embodiment of the present application. Figure 6 As shown, the anomaly detection device 600 includes: an acquisition module 601, a mining module 602, a construction module 603 and a detection module 604.

[0119] The acquisition module 601 is used to acquire the transaction log of the target object; wherein the transaction log includes multiple transactions, the transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp.

[0120] The mining module 602 is used to mine the record information in the transaction log based on the association rule mining algorithm to obtain multiple frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set.

[0121] The construction module 603 is used to construct a frequent itemset combination sequence according to multiple frequent itemset combinations and timestamps in the transaction log; wherein the frequent itemset combination sequence is used to indicate the dynamic relationship of the frequent itemset combinations changing over time.

[0122] The detection module 604 is configured to perform anomaly detection on the transaction log of the target object based on the time series model and the frequent itemset combination sequence, and determine anomaly detection results.

[0123] In an optional implementation, when mining the record item information in the transaction log based on the association rule mining algorithm to obtain multiple frequent item set combinations, the mining module 602 is specifically configured to:

[0124] According to the timestamps carried by each transaction in the transaction log, a time window of preset size is used to slide and extract the transaction log in the time dimension to obtain multiple sub-transaction logs; wherein the sub-transaction log includes at least one transaction; for each sub-transaction log, the record item information of the sub-transaction log is mined based on the association rule mining algorithm to obtain the frequent item set combination corresponding to the sub-transaction log, and the frequent item set combination corresponding to each sub-transaction log is used as multiple frequent item set combinations.

[0125] In an optional implementation, for each sub-transaction log, based on an association rule mining algorithm, the record information of the sub-transaction log is mined to obtain a frequent item set combination corresponding to the sub-transaction log. The mining module 602 is specifically configured to:

[0126] For each sub-transaction log, if there is a record item in the sub-transaction log whose value range is a continuous numerical space, the value information of the record item is converted into discrete data to obtain the converted sub-transaction log; based on the association rule mining algorithm, the record item information of the converted sub-transaction log is mined to obtain the frequent item set combination corresponding to the sub-transaction log; among which, the elements in the frequent item set are the value information of the record item.

[0127] In an optional implementation, when constructing a frequent itemset combination sequence according to multiple frequent itemset combinations and timestamps in the transaction log, the construction module 603 is specifically configured to:

[0128] Determine the base timestamp corresponding to each frequent itemset combination; wherein the base timestamp corresponding to any frequent itemset combination is the first timestamp in the sub-transaction log corresponding to the frequent itemset combination; sort multiple frequent itemset combinations according to the base timestamps corresponding to each frequent itemset combination to obtain a frequent itemset combination sequence.

[0129] In an optional implementation, based on a time series model and according to a combination sequence of frequent itemsets, anomaly detection is performed on the transaction log of the target object. When determining the anomaly detection result, the detection module 604 is specifically configured to:

[0130] The frequent item set combination sequence is encoded to obtain the encoding sequence corresponding to the frequent item set combination sequence; based on the time series model, anomaly detection is performed on the encoding sequence to obtain the anomaly label corresponding to each time step in the encoding sequence; wherein the anomaly label is used to indicate the abnormal situation of the sub-transaction log corresponding to the frequent item set combination corresponding to the time step, and the abnormal situation includes normal and abnormal.

[0131] In an optional implementation, the detection module 604 is further configured to:

[0132] If the abnormal label corresponding to any time step in the coding sequence indicates that the abnormal situation of the sub-transaction log corresponding to the time step is abnormal, a prompt information is output; wherein the prompt information is used to indicate that there is an abnormality in the target object, and the prompt information includes the sub-transaction log corresponding to the frequent item set combination corresponding to the time step.

[0133] In an optional implementation, the time series model is a bidirectional long short-term memory (Bi-LSTM) network, which includes a forward long short-term memory (LSTM) network layer, a reverse long short-term memory (LSTM) network layer, and an output layer. Based on the time series model, anomalies in the transaction log of the target object are detected according to the frequent item set combination sequence. When determining the anomaly detection result, the detection module 604 is specifically configured to:

[0134] The frequent item set combination sequence is encoded to obtain the encoding sequence of the frequent item set combination sequence; the encoding sequence is input into the forward long short-term memory network layer, and the forward long short-term memory network layer processes the data corresponding to each time step in the encoding sequence in sequence from front to back in time to obtain the first eigenvector; the encoding sequence is input into the reverse long short-term memory network layer, and the reverse long short-term memory network layer processes the data corresponding to each time step in the encoding sequence in sequence from back to front in time to obtain the second eigenvector; based on the first eigenvector and the second eigenvector, the output layer outputs the anomaly detection result corresponding to the transaction log of the target object.

[0135] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 includes a memory 701 and a processor 702. The memory 701 is used to store a computer program, and when the processor 702 executes the computer program, it implements the method of any of the above embodiments. A communication link is provided between the memory 701 and the processor 702. Exemplarily, the memory 701 and the processor 702 can communicate via a communication bus 703.

[0136] Optionally, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps in the method embodiments disclosed herein may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0137] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, a method in any of the above method embodiments is implemented.

[0138] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method in any of the above method embodiments when the computer program is executed by a processor.

[0139] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0140] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0141] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0143] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting anomalies, characterized in that: include: Obtaining a transaction log of a target object; wherein the transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp; Based on an association rule mining algorithm, mining the record item information in the transaction log to obtain a plurality of frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set; Constructing a frequent item set combination sequence according to the multiple frequent item set combinations and the timestamps in the transaction log; wherein the frequent item set combination sequence is used to indicate the dynamic relationship of the frequent item set combinations changing over time; Based on the time series model and according to the frequent item set combination sequence, anomaly detection is performed on the transaction log of the target object to determine an anomaly detection result.

2. The method according to claim 1, characterized in that The association rule mining algorithm is based on mining the record item information in the transaction log to obtain multiple frequent item set combinations, including: According to the timestamps carried by each transaction in the transaction log, using a time window of a preset size, slidingly extracting the transaction log in a time dimension to obtain multiple sub-transaction logs; wherein the sub-transaction log includes at least one transaction; For each of the sub-transaction logs, the record item information of the sub-transaction log is mined based on an association rule mining algorithm to obtain a frequent item set combination corresponding to the sub-transaction log, and the frequent item set combination corresponding to each of the sub-transaction logs is used as the multiple frequent item set combinations.

3. The method according to claim 2, characterized in that For each of the sub-transaction logs, mining the record item information of the sub-transaction log based on an association rule mining algorithm to obtain a frequent item set combination corresponding to the sub-transaction log includes: For each of the sub-transaction logs, if there is a record item in the sub-transaction log whose value range is a continuous numerical space, converting the value information of the record item into discrete data to obtain a converted sub-transaction log; Based on an association rule mining algorithm, the record item information of the converted sub-transaction log is mined to obtain a frequent item set combination corresponding to the sub-transaction log; wherein the elements in the frequent item set are value information of the record items.

4. The method according to claim 2 or 3, characterized in that The step of constructing a frequent item set combination sequence according to the multiple frequent item set combinations and the timestamps in the transaction log includes: Determine a reference timestamp corresponding to each of the frequent itemset combinations; wherein the reference timestamp corresponding to any of the frequent itemset combinations is the first timestamp in the sub-transaction log corresponding to the frequent itemset combination; The multiple frequent item set combinations are sorted according to the reference timestamps corresponding to the respective frequent item set combinations to obtain the frequent item set combination sequence.

5. The method according to claim 3, characterized in that The performing anomaly detection on the transaction log of the target object based on the time series model and the frequent item set combination sequence to determine the anomaly detection result includes: Encoding the frequent item set combination sequence to obtain a coding sequence corresponding to the frequent item set combination sequence; Based on the time series model, anomaly detection is performed on the coding sequence to obtain anomaly labels corresponding to each time step in the coding sequence; wherein the anomaly labels are used to indicate an anomaly of the sub-transaction log corresponding to the frequent item set combination corresponding to the time step, and the anomaly includes normal and abnormal conditions.

6. The method according to claim 5, characterized in that Also includes: If the exception label corresponding to any time step in the coding sequence indicates that the abnormal situation of the sub-transaction log corresponding to the time step is abnormal, a prompt information is output; wherein the prompt information is used to indicate that there is an abnormality in the target object, and the prompt information includes the sub-transaction log corresponding to the frequent item set combination corresponding to the time step.

7. The method according to any one of claims 1 to 3, characterized in that The timing model is a bidirectional long short-term memory (Bi-LSTM) network, which includes a forward long short-term memory (LSTM) network layer, a reverse long short-term memory (LSTM) network layer, and an output layer. The detecting anomalies in the transaction log of the target object based on the time series model and the frequent item set combination sequence to determine anomaly detection results includes: Encoding the frequent item set combination sequence to obtain an encoding sequence of the frequent item set combination sequence; Inputting the coding sequence into the forward long short-term memory network layer, the forward long short-term memory network layer sequentially processes the data corresponding to each time step in the coding sequence in chronological order from front to back to obtain a first eigenvector; Inputting the coding sequence into the inverse long short-term memory network layer, the inverse long short-term memory network layer sequentially processes the data corresponding to each time step in the coding sequence in chronological order from back to front to obtain a second feature vector; The output layer outputs an anomaly detection result corresponding to the transaction log of the target object according to the first feature vector and the second feature vector.

8. An abnormality detection device, characterized in that: include: An acquisition module, configured to acquire a transaction log of a target object; wherein the transaction log includes multiple transactions, each transaction includes multiple record items and value information corresponding to each record item, and the transaction carries a timestamp; A mining module, configured to mine the record information in the transaction log based on an association rule mining algorithm to obtain a plurality of frequent item set combinations; wherein a single frequent item set combination includes at least one frequent item set; A construction module, configured to construct a frequent item set combination sequence according to the multiple frequent item set combinations and the timestamps in the transaction log; wherein the frequent item set combination sequence is used to indicate a dynamic relationship of the frequent item set combinations changing over time; The detection module is configured to perform anomaly detection on the transaction log of the target object based on a time series model and according to the frequent item set combination sequence, and determine anomaly detection results.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images