Anomaly detection method and device of data, computer device and storage medium

Abstract

The present specification discloses an exception detection method and device of data, computer equipment and storage medium, the method comprises: obtaining current transaction log and historical transaction log;Based on the transaction channel, the historical transaction log is divided, and the first historical log of each transaction channel is obtained;Based on the distribution data of the target error code in the first historical log, the abnormal screening condition of each transaction channel is determined, and the target error code is used to represent any category of error code;Based on the abnormal screening condition of each transaction channel and the current transaction log, it is determined whether the current transaction is abnormal.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and more specifically, to a method, apparatus, computer device, and storage medium for detecting data anomalies. Background Technology

[0002] In financial settings, the stable operation of a transaction system is crucial for user fund security, merchant operational efficiency, and platform credibility. Transaction logs, as data reflecting the system's operational status, are highly sensitive; abnormal fluctuations in these logs often indicate risks such as system failures or channel anomalies. Therefore, detecting anomalies in transaction logs is a key factor in ensuring transaction stability.

[0003] In related technologies, transaction logs are monitored using fixed rules; however, the lack of flexibility in fixed-rule detection makes it difficult to adapt to complex application scenarios, resulting in inconsistent accuracy in detecting anomalies in transaction scenarios. Therefore, improving the accuracy of data anomaly detection in transaction scenarios has become an urgent problem to be solved. Summary of the Invention

[0004] This specification provides a method, apparatus, computer device, and storage medium for detecting data anomalies. The method can detect transaction logs at different granularities to determine whether there are any anomalies in the transactions, thereby improving the accuracy of data anomaly detection.

[0005] Firstly, an anomaly detection method for data is provided. The method includes: acquiring current transaction logs and historical transaction logs; dividing historical transaction logs based on transaction channels to obtain the first historical logs for each transaction channel; determining anomaly screening conditions for each transaction channel based on the distribution data of target error codes in the first historical logs, where the target error code is used to represent any type of error code; and determining whether there is an anomaly in the current transaction based on the anomaly screening conditions of each transaction channel and the current transaction logs.

[0006] The above technical solution divides historical transaction logs based on transaction channels to obtain the first historical log for each transaction channel. Based on the distribution data of target error codes in the first historical log, it determines the anomaly screening conditions for each transaction channel. Based on the anomaly screening conditions of each transaction channel and the current transaction log, it determines whether the current transaction has an anomaly. By finely dividing the historical transaction log into transaction channels and dynamically determining the anomaly screening conditions for each transaction channel based on the distribution data of error codes in the divided first historical log, it can dynamically plan the anomaly screening conditions in conjunction with the actual transaction log, improving the matching between the anomaly screening conditions and the actual transaction. On this basis, the current transaction log is detected through the anomaly screening conditions to determine whether the current transaction corresponding to the current transaction log has an anomaly, thus improving the accuracy of data anomaly detection.

[0007] In one possible implementation, based on the distribution data of the target error codes in the first historical log, the anomaly screening conditions for each transaction channel are determined, including: dividing the first historical log based on log attributes to obtain second historical logs for each log attribute, wherein the log attributes are determined based on at least one of business identifier, product identifier, and event identifier; and determining the anomaly screening conditions for each transaction channel for different log attributes based on the distribution data of the target error codes in the second historical log.

[0008] The above technical solution divides the first historical log based on log attributes to obtain the second historical log for each log attribute. Based on the distribution data of target error codes in the second historical log, the anomaly screening conditions for each transaction channel under different log attributes are determined. By dividing the first historical log into log attributes, the anomaly screening conditions for each transaction channel under different log attributes are determined, enabling more granular detection of transaction logs and improving the accuracy of data anomaly detection.

[0009] In one possible implementation, based on the distribution data of the target error codes in the second historical log, the anomaly filtering conditions for each transaction channel under different log attributes are determined, including: based on the distribution data of the target error codes in the second historical log, determining the number and percentage of target error codes within a preset time window, where the percentage represents the proportion of the number of errors to the number of transaction requests within the preset time window; based on the number of errors of the target log attribute, determining the quantity threshold corresponding to the target log attribute, where the target log attribute represents any log attribute; and based on the quantity percentage of the target log attribute, determining the percentage threshold corresponding to the target log attribute, where the anomaly filtering conditions include the quantity threshold and the percentage threshold.

[0010] The above technical solution determines the number and percentage of target error codes within a preset time window by analyzing the distribution data of target error codes in the second historical log. It then combines this with the number and percentage of errors associated with target log attributes to determine the corresponding quantity and percentage thresholds for each attribute. By analyzing the number and percentage of error codes, the frequency of target error codes occurring within the preset time window can be reflected, thereby determining log trend changes and improving the reliability of anomaly detection.

[0011] In one possible implementation, the preset time window includes a first time window and a second time window. The first time window is used to represent the time window adjacent to the current time window, and the second time window is used to represent the time window corresponding to the current time window within the previous preset period.

[0012] The above technical solution includes a first time window and a second time window. The first time window represents the time window adjacent to the current time window, and the second time window represents the time window corresponding to the current time window within the previous preset period. By capturing the pattern of error codes from both short-term and periodic perspectives through multiple time windows, the reliability of determining the anomaly screening conditions is improved, thereby improving the accuracy of data anomaly detection.

[0013] One possible implementation involves determining the quantity threshold corresponding to the target log attribute based on the number of errors in the target log attribute, including: obtaining the business traffic of the current time window; and determining the quantity threshold corresponding to the target log attribute based on the business traffic and the number of errors.

[0014] The above technical solution determines the quantity threshold corresponding to the target log attribute based on the business traffic and error count in the current time window. It can associate data anomaly detection with business traffic, avoid data fluctuations caused by business traffic, and improve the accuracy of anomaly detection.

[0015] In one possible implementation, determining whether a current transaction is abnormal based on the anomaly screening conditions of each transaction channel and the current transaction log includes: dividing the current transaction log based on the transaction channel and log attributes to obtain the first log to be detected under each transaction channel with different log attributes, where the current transaction log is used to represent the transaction log of the current time window; and determining whether the current transaction with the target log attribute is abnormal based on the first log to be detected under each transaction channel and the anomaly screening conditions.

[0016] The above technical solution divides the current transaction logs based on transaction channels and log attributes to obtain the first logs to be detected under different log attributes in each transaction channel. Based on the first logs to be detected under the target log attributes in each transaction channel and the anomaly filtering conditions, it determines whether there are any anomalies in the current transaction for the target log attributes. By dividing the current transaction logs, it ensures that the comparison logs are in the same range, thereby improving the accuracy of data anomaly detection.

[0017] In one possible implementation, based on the first log to be detected and the anomaly filtering conditions of the target log attribute under each transaction channel, it is determined whether the current transaction of the target log attribute is abnormal, including: based on the first log to be detected, determining the first number of errors and the first percentage of the target error code in the target log attribute; when the first number of errors is greater than the number threshold and the first percentage of the number is greater than the percentage threshold, it is determined that the current transaction of the target log attribute is abnormal; when the first number of errors is less than or equal to the number threshold, or the first percentage of the number is less than or equal to the percentage threshold, it is determined that the current transaction of the target log attribute is not abnormal.

[0018] The above technical solution determines the first number of errors and the first percentage of the target error code in the target log attribute based on the first log to be detected, and detects the error number and the percentage of the target error code respectively, avoiding the error of single-dimensional quantity detection and improving the accuracy of data anomaly detection.

[0019] In one possible implementation, the anomaly screening conditions for each trading channel are determined based on the distribution data of the target error codes in the first historical log, including: determining the historical processing items of the target error codes in the first historical log; and determining the anomaly screening conditions for each trading channel based on the distribution data of the historical processing items and the target error codes.

[0020] The above technical solution determines the anomaly screening conditions for each transaction channel by using the historical processing items and distribution data of the target error codes in the first historical log. It can reflect the impact of the target error codes by combining the historical processing items, and thus determine the screening conditions for transaction data based on the impact of the target error codes. This improves both the accuracy and speed of anomaly detection.

[0021] In one possible implementation, based on the distribution data of historical processed items and target error codes, the abnormal screening conditions for each transaction channel are determined, including: determining an error code whitelist based on historical processed items; determining the error code category and / or error frequency of the target error code based on the distribution data of the target error code in the first historical log; and determining the error code to be filtered based on the error code category and / or error frequency of the target error code. The abnormal screening conditions include the error code whitelist and the error code to be filtered.

[0022] The above technical solution determines an error code whitelist based on historical processing events, and determines the error code category and / or error frequency of the target error code based on the distribution data of the target error code in the first historical log, thereby determining the error code to be filtered. Based on the occurrence frequency of historical processing events and target error codes respectively, it can determine whether the target error code is a high-frequency error code, and thus determine whether it needs to be filtered out, avoiding noise caused by high-frequency error codes in data anomaly detection, and improving the efficiency of data anomaly detection.

[0023] In one possible implementation, determining the error code to be filtered based on the error code category and error frequency of the target error code includes: when the error code category of the target error code is a subcategory, determining the target error code as the first filtered error code; when the error frequency of the target error code is greater than a preset frequency threshold, determining the target error code as the second filtered error code; and determining the error code to be filtered based on the first filtered error code and the second filtered error code.

[0024] The above technical solution determines the target error code as the first filter error code when the error code category of the target error code is a subcategory, and determines the target error code as the second filter error code when the error frequency of the target error code is greater than a preset frequency threshold. Based on the first and second filter error codes, the error code to be filtered is determined. By combining the error code category and error frequency, it is determined whether the error code is an allowed error code, thereby removing the error code to be filtered in the transaction task and improving the data anomaly detection rate.

[0025] One possible implementation involves determining whether a current transaction is abnormal based on the anomaly screening conditions of each transaction channel and the current transaction log. This includes: dividing the current transaction log into sections based on the transaction channel to obtain a second test log for each transaction channel; and detecting the second test log based on an error code whitelist and an error code to be filtered to determine whether a current transaction in each transaction channel is abnormal.

[0026] The above technical solution divides the current transaction logs based on the transaction channels to obtain the second test logs for each transaction channel. Based on the error code whitelist and the error codes to be filtered, the second test logs are checked to determine whether there are any anomalies in the current transactions of each transaction channel. By dividing the current transaction logs and historical transaction logs into logs of the same dimension, the test logs and anomaly screening conditions can be compared on the same dimension, thereby determining whether there are any anomalies in the current transactions of each transaction channel, which improves the accuracy of data anomaly detection.

[0027] In one possible implementation, the second test log is checked based on an error code whitelist and the error codes to be filtered to determine whether there are any anomalies in the current transactions of each transaction channel. This includes: filtering the error codes in the second test log based on the error codes to be filtered to obtain a third test log, in which there are no error codes to be filtered; if a new error code is added to the third test log, it is determined that the current transactions of the corresponding transaction channel are abnormal, and the new error code does not belong to the error code whitelist; if no new error code is added to the third test log, it is determined that the current transactions of the corresponding transaction channel are not abnormal.

[0028] The above technical solution filters error codes in the second test log based on the error codes to be filtered, resulting in a third test log. When a new error code is found in the third test log, it is determined that the current transaction of the corresponding trading channel is abnormal; when no new error code is found in the third test log, it is determined that the current transaction of the corresponding trading channel is not abnormal. By removing high-frequency error codes from the second test log and comparing the error codes in the removed test log with the error code whitelist, it is determined whether a new whitelist has been added. This enables rapid detection and alerting of error codes appearing for the first time, preventing the spread of unknown risks.

[0029] Secondly, an anomaly detection device for data is provided, comprising: an acquisition module for acquiring current transaction logs and historical transaction logs; a segmentation module for segmenting historical transaction logs based on transaction channels to obtain first historical logs for each transaction channel; a determination module for determining anomaly screening conditions for each transaction channel based on the distribution data of target error codes in the first historical logs, wherein the target error code represents any type of error code; and a detection module for determining whether an anomaly exists in the current transaction based on the anomaly screening conditions of each transaction channel and the current transaction logs.

[0030] Thirdly, a computer device is provided, comprising one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the above-mentioned method for detecting abnormal data.

[0031] Fourthly, a computer-readable storage medium is provided, in which at least one computer program is stored, the computer program being loaded and executed by a processor to implement the above-mentioned method for detecting abnormal data.

[0032] Fifthly, a computer program product or computer program is provided, the computer program product or computer program including program code stored in a computer-readable storage medium, a processor of a computer device reading the program code from the computer-readable storage medium, the processor executing the program code, causing the computer device to perform the above-mentioned abnormal data detection method.

[0033] The technical solution provided in the embodiments of this specification divides historical transaction logs based on transaction channels to obtain the first historical log for each transaction channel. Based on the distribution data of target error codes in the first historical log, the anomaly screening conditions for each transaction channel are determined. Based on the anomaly screening conditions of each transaction channel and the current transaction log, it is determined whether the current transaction is abnormal. By finely dividing the historical transaction log into transaction channels and dynamically determining the anomaly screening conditions for each transaction channel based on the distribution data of error codes in the divided first historical log, the anomaly screening conditions can be dynamically planned in conjunction with the actual transaction log, improving the matching between the anomaly screening conditions and the actual transaction. On this basis, the current transaction log is detected through the anomaly screening conditions to determine whether the current transaction corresponding to the current transaction log is abnormal, thus improving the accuracy of data anomaly detection. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the framework of a data anomaly detection system provided in the embodiments of this specification; Figure 2 This is a schematic flowchart of a data anomaly detection method provided in the embodiments of this specification; Figure 3 This is a schematic flowchart of a method for detecting anomalies provided in the embodiments of this specification; Figure 4 This is a schematic flowchart of another method for detecting anomalies provided in the embodiments of this specification; Figure 5 This is a schematic diagram of the structure of a data anomaly detection device provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification; Figure 7 This is a schematic diagram of the structure of a server provided in the embodiments of this specification. Detailed Implementation

[0036] The technical solutions in this specification will now be described clearly and in detail with reference to the accompanying drawings. In the description of the embodiments in this specification, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments in this specification, "multiple" refers to two or more than two.

[0037] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0038] In the specific embodiments of this specification, any data related to the object, such as data involved in the use of the application, is required. When the embodiments of this specification are applied to specific products or technologies, permission or consent from the object is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if any of the aforementioned object-related data is involved in the embodiments of this specification, this data must be obtained with the object's authorization and consent, and in accordance with the relevant laws, regulations, and standards of the country and region.

[0039] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of the embodiments of this specification and the technical effects produced by the technical solutions of this specification will be explained below with reference to the accompanying drawings and through the description of several exemplary embodiments. It should be noted that the following embodiments can be referred to, borrowed from, or combined with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be described again.

[0040] In financial settings, the stable operation of a transaction system is crucial for user fund security, merchant operational efficiency, and platform credibility. Transaction logs, as data reflecting the system's operational status, are highly sensitive; abnormal fluctuations in these logs often indicate risks such as system failures or channel anomalies. Therefore, detecting anomalies in transaction logs is a key factor in ensuring transaction stability.

[0041] In related technologies, transaction logs are monitored using fixed rules; however, due to the poor flexibility of fixed rule detection, it is difficult to adapt to complex application scenarios, making it difficult to guarantee the accuracy of anomaly detection in transaction scenarios.

[0042] Specifically, traditional monitoring methods struggle to comprehensively cover all anomalies, especially in complex and ever-changing scenarios, exhibiting significant blind spot risks. The main reasons are twofold: First, local anomalies are masked by overall metrics. Many anomalies occur only in niche paths or low-traffic scenarios (such as specific user groups, peripheral functional modules, or newly launched interfaces). While they may not significantly impact overall success rates or response times, they could conceal serious logical errors, data inconsistencies, or security vulnerabilities. Because traditional detection methods focus on aggregated metrics, these "silent" anomalies often fail to be detected and alerted in a timely manner. Second, monitoring systems heavily rely on fixed rules and lack generalization capabilities. Existing monitoring mechanisms typically require pre-configuring fixed threshold rules across various dimensions (such as region, channel, product line, and user segmentation). This approach is not only costly to deploy and maintain but also highly susceptible to missed detections due to incomplete rule coverage or unreasonable threshold settings. Once an anomaly pattern not covered by pre-defined rules appears, the system is completely unaware, allowing risks to directly breach the defenses.

[0043] In view of this, the embodiments of this specification disclose a data anomaly detection method, which has a more universal and adaptive anomaly identification capability, and can automatically discover signals that deviate from the normal state based on the distribution characteristics or behavioral patterns of the data itself, thereby improving the accuracy of data anomaly detection.

[0044] The following is combined Figure 1 The architecture of the embodiments described in this specification is explained.

[0045] Figure 1 This is a schematic diagram of the framework of a data anomaly detection system provided in the embodiments of this specification.

[0046] For example, such as Figure 1 As shown, the data anomaly detection system includes a data source access module, a baseline construction module, an anomaly analysis module, an anomaly attribution module, and an anomaly alarm module. The data source access module includes logs and transaction records. By importing logs and / or transaction records into the baseline construction module, the module analyzes the imported log data to determine behavioral benchmarks across different dimensions, i.e., anomaly screening conditions. The anomaly analysis module determines whether an anomaly has occurred based on these behavioral benchmarks. When an anomaly occurs, the anomaly attribution module locates the identified anomaly, determining whether it is caused by business operations, network issues, or organizational structure. Finally, the anomaly alarm module issues an alarm.

[0047] The error code baseline in the baseline construction module involves dividing transaction data into characteristic channels, such as transaction channels, transaction institutions, error codes, and business dimensions, and performing anomaly analysis through different granularities of division. The first feature analysis clustering determines the input parameter distribution by clustering the input parameters of user requests; the second feature analysis clustering determines the returned data clusters by clustering the results returned by different transaction institutions; and the third feature analysis clustering clusters by mapping multiple input parameter combinations to error codes. Therefore, the baseline construction module uses multi-dimensional clustering methods to cluster transaction logs corresponding to different dimensions, determining the data distribution at multiple granularities.

[0048] The anomaly analysis module compares the current transaction logs for any abnormalities. Initial detection is achieved by comparing the error code baseline characteristics to identify new error codes and triggering an alert, preventing the spread of unknown risks. Baseline surge detection identifies a sharp increase in error codes across feature dimensions, effectively recognizing abnormal fluctuations. Baseline clustering can cluster based on specified features or adaptively, using a large language model to identify the distribution of different feature clusters.

[0049] When the anomaly analysis module detects an anomaly, the anomaly attribution module analyzes the cause of the anomaly based on the data distribution in different dimensions of the baseline construction module, and outputs an alarm through the anomaly alarm module, so that users can quickly locate the cause of the anomaly after receiving the anomaly alarm and improve the repair efficiency.

[0050] The aforementioned technical solution first constructs a zero-tolerance baseline layer (i.e., initial occurrence judgment) to provide immediate alerts for any newly added error codes. Secondly, based on transaction channels, transaction institutions, and error codes, it introduces finer-grained log attribute classification (such as product identifiers, event identifiers, etc.) to identify abnormal fluctuations in the baseline through dynamic baseline modeling. Then, it combines input and output parameters to construct a behavioral baseline, enabling anomaly detection and attribution analysis to facilitate users' rapid identification of the cause of anomalies. Finally, it uses a large language model to cluster and identify multi-dimensional temporal features, combining this with the abnormal distribution of data to identify transaction anomalies. Based on this, the sensitivity and accuracy of anomaly detection are improved through the mutual collaboration between multiple layers. By constructing a multi-layered defense system, the ability to intercept known anomalies and detect unknown anomalies is enhanced, constituting a high-precision, low-latency, and highly generalizable anomaly detection solution for financial interaction scenarios.

[0051] Figure 2 This is a schematic flowchart of a data anomaly detection method provided in the embodiments of this specification.

[0052] For example, such as Figure 2 As shown, the data anomaly detection method 200 includes steps S210 to S240.

[0053] S210, retrieve the current transaction log and historical transaction log.

[0054] For example, the current transaction log is the log corresponding to the current transaction, which is generated by a transaction task. The historical transaction log is the log corresponding to transactions from previous times, and can be transaction logs collected through an offline task platform. For instance, the current transaction log is the transaction log for the current time window; the length of the current time window can be 1 minute, 10 minutes, 1 hour, etc. The current transaction log and the historical transaction log are transaction logs for the same time window.

[0055] Optionally, the transaction log can be a receipt transaction log or a payment transaction log, meaning that the transaction items include both receipt transaction items and payment transaction items.

[0056] It should be understood that the transaction log includes the processing results corresponding to the transaction requests. For example, if the transaction log includes error codes, it indicates that the transaction corresponding to the transaction log may have an anomaly; if the transaction log only includes success codes, it indicates that the transaction corresponding to the transaction log was successfully completed.

[0057] S220: Divide the historical transaction logs based on the transaction channels to obtain the first historical logs for each transaction channel.

[0058] For example, historical transaction logs are obtained, and the historical transaction logs are divided based on the transaction channel to obtain the first historical log of each transaction channel. That is, the first historical log is used to represent the transaction log of any transaction channel.

[0059] Trading channels are determined through different trading channels and / or trading institutions. For example, different trading channels may correspond to different trading channels; or different trading institutions may correspond to different trading channels. Of course, the same trading channel and the same trading institution can correspond to one trading channel.

[0060] Optionally, transaction channels can include UnionPay, NetsUnion, Alipay, etc. Transaction institutions are organizations that engage in transaction activities; these can be different banks or different exchanges. The choice of transaction channels or institutions can be determined based on the specific circumstances and is not specifically limited here.

[0061] The trading channel is determined based on the trading channel and trading institution. For example, historical trading logs are divided according to the trading channel and trading institution to obtain the first historical log corresponding to the same trading channel and the same trading institution, which is the first historical log of each trading channel.

[0062] S230, based on the distribution data of target error codes in the first historical log, determine the anomaly screening conditions for each trading channel.

[0063] For example, the target error code is used to represent any category of error code. It should be understood that since the first historical log is obtained by dividing the historical transaction log into transaction channels, the distribution data of the target error code in the first historical log represents the distribution data of the same transaction channel and the same error code.

[0064] To address the issue of error codes being masked by overall success rate metrics in low-traffic scenarios, monitoring error codes can significantly improve sensitivity to anomalies in edge paths.

[0065] For example, the historical processing items of the target error code in the first historical log are determined, and the anomaly screening conditions for each transaction channel are determined based on the distribution data of the historical processing items and the target error code.

[0066] Historical processing items indicate the processing method for target error codes at a historical moment, reflecting whether the target error code is an allowed error code. For example, a historical processing item could indicate that the target error code is ignored, in which case the target error code is a valid error code; or, a historical processing item could indicate that the target error code needs to be processed or reviewed immediately, in which case the target error code is an invalid error code.

[0067] By combining historical processing events to determine the handling methods for target error codes at historical moments, and then combining the distribution data of target error codes to determine the anomaly screening conditions for each trading channel, the trading task can be completed.

[0068] The above technical solution determines the anomaly screening conditions for each transaction channel by using the historical processing items and distribution data of the target error codes in the first historical log. It can reflect the impact of the target error codes by combining the historical processing items, and thus determine the screening conditions for transaction data based on the impact of the target error codes. This improves both the accuracy and speed of anomaly detection.

[0069] For example, an error code whitelist is determined based on historical processing events; the error code category and / or error frequency of the target error code are determined based on the distribution data of the target error code in the first historical log; and the error codes to be filtered are determined based on the error code category and / or error frequency of the target error code, with the anomaly filtering conditions including the error code whitelist and the error codes to be filtered.

[0070] Error code whitelists are used to indicate whether historical processing events reflect the allowed whitelist or the legitimate whitelist.

[0071] Optionally, based on the error code category and / or the error frequency of the target error code, determine whether the target error code is a high-frequency error code, and identify the high-frequency error code as the error code to be filtered.

[0072] The above technical solution determines an error code whitelist based on historical processing events, and determines the error code category and / or error frequency of the target error code based on the distribution data of the target error code in the first historical log, thereby determining the error code to be filtered. Based on the occurrence frequency of historical processing events and target error codes respectively, it can determine whether the target error code is a high-frequency error code, and thus determine whether it needs to be filtered out, avoiding noise caused by high-frequency error codes in data anomaly detection, and improving the efficiency of data anomaly detection.

[0073] In one example, the error code to be filtered is determined based on the error code category and error frequency of the target error code. Specifically, when the error code category of the target error code is a subcategory, the target error code is determined as the first filtered error code; when the error frequency of the target error code is greater than a preset frequency threshold, the target error code is determined as the second filtered error code; and the error code to be filtered is determined based on the first filtered error code and the second filtered error code.

[0074] Optionally, the preset frequency threshold can be 1 time / minute, 2 times / minute, etc., without specific limitations.

[0075] For example, if the target error code is a level 2 error code, or if the target error code has an error frequency greater than 1 time per minute, then the target error code is identified as an error code to be filtered. Since the first historical log is obtained by dividing historical transaction logs through transaction channels, the anomaly filtering conditions corresponding to the transaction channel and error code dimension are determined by judging each target error code (i.e., error codes of each error code category) in the first historical log.

[0076] The above technical solution determines the target error code as the first filter error code when the error code category of the target error code is a subcategory, and determines the target error code as the second filter error code when the error frequency of the target error code is greater than a preset frequency threshold. Based on the first and second filter error codes, the error code to be filtered is determined. By combining the error code category and error frequency, it is determined whether the error code is an allowed error code, thereby removing the error code to be filtered in the transaction task and improving the data anomaly detection rate.

[0077] In another example, the target error code is identified as a filterable error code when its error code category is a subcategory. Alternatively, the target error code can be identified as a filterable error code when its error frequency exceeds a preset frequency threshold.

[0078] For abnormal growth of error codes in specific contexts, baseline fluctuation detection can be used. Examples include abnormalities in the operational baseline of funding channels, user demographics, and internal routing links.

[0079] For example, the first historical log is divided based on log attributes to obtain the second historical log for each log attribute, and the anomaly screening conditions for each transaction channel under different log attributes are determined by the distribution data of the target error codes in the second historical log.

[0080] Log attributes are determined by any one of the following: business type, routing rules, or user characteristics. For example, when log attributes are determined by business type, they refer to the business identifier, product identifier, and event identifier. Thus, if log attributes are determined by the business identifier, product identifier, and event identifier, the second historical log is used to represent transaction logs from the same transaction channel, using the same business identifier, product identifier, and event identifier.

[0081] A business identifier is used to identify a business entity or distinguish different business types. For example, a business identifier can differentiate between services such as utility bill payments and travel ticket purchases. A product identifier is used to indicate the name and characteristics of a product. An event identifier is used to distinguish different events.

[0082] The above technical solution divides the first historical log based on log attributes to obtain the second historical log for each log attribute. Based on the distribution data of target error codes in the second historical log, the anomaly screening conditions for each transaction channel under different log attributes are determined. By dividing the first historical log into log attributes, the anomaly screening conditions for each transaction channel under different log attributes are determined, enabling more granular detection of transaction logs and improving the accuracy of data anomaly detection.

[0083] Specifically, the first historical log is divided based on log attributes to obtain the second historical log for each log attribute. Based on the distribution data of the target error code in the second historical log, the number and percentage of the target error code under the preset time window are determined. Based on the number of errors of the target log attribute, the quantity threshold corresponding to the target log attribute is determined. Based on the quantity percentage of the target log attribute, the percentage threshold corresponding to the target log attribute is determined.

[0084] The quantity percentage represents the proportion of errors in the total number of transaction requests within a preset time window. For example, each transaction request within a preset time window may correspond to an error code or a success code. The proportion of error codes in the transaction requests reflects the probability of the event corresponding to the error code occurring.

[0085] The anomaly filtering criteria include quantity thresholds and percentage thresholds. The quantity threshold represents the number of items within a preset time window, while the percentage threshold represents the percentage of items within the same preset time window. Both the quantity threshold and percentage threshold change simultaneously as the time window length changes.

[0086] The target log attribute is used to represent any log attribute.

[0087] There is a correspondence between the number of errors and the quantity threshold; for example, a quantity range is determined based on the number of errors, and the threshold corresponding to the quantity range is determined as the quantity threshold. Similarly, there is a correspondence between the quantity percentage and the percentage threshold; after determining the range in which the quantity percentage falls, the threshold corresponding to that range is determined as the percentage threshold.

[0088] The above technical solution determines the number and percentage of target error codes within a preset time window by analyzing the distribution data of target error codes in the second historical log. It then combines this with the number and percentage of errors associated with target log attributes to determine the corresponding quantity and percentage thresholds for each attribute. By analyzing the number and percentage of error codes, the frequency of target error codes occurring within the preset time window can be reflected, thereby determining log trend changes and improving the reliability of anomaly detection.

[0089] Optionally, the preset time window can be a single time window or multiple time windows. It should be understood that when there are multiple preset time windows, the duration of each preset time window is the same.

[0090] The preset time window includes a first time window and a second time window. The first time window represents the time window adjacent to the current time window, and the second time window represents the time window corresponding to the current time window within the previous preset period. The first time window is used to capture short-term trends; for example, the first time window is the previous time window. The second time window is used to capture cyclical patterns; for example, the second time window is the time window of the same time on the previous day.

[0091] The above technical solution includes a first time window and a second time window. The first time window represents the time window adjacent to the current time window, and the second time window represents the time window corresponding to the current time window within the previous preset period. By capturing the pattern of error codes from both short-term and periodic perspectives through multiple time windows, the reliability of determining the anomaly screening conditions is improved, thereby improving the accuracy of data anomaly detection.

[0092] To improve the accuracy of quantity and percentage thresholds, the thresholds can be adjusted based on business traffic.

[0093] In one example, the business traffic of the current time window is obtained, and the number of errors of the target error code under the preset time window is determined based on the distribution data of the target error code in the second historical log. Then, based on the business traffic and the number of errors, the quantity threshold corresponding to the target log attribute is determined.

[0094] Optionally, the percentage threshold can also be adjusted according to business traffic. The adjustment method of the percentage threshold based on business traffic can be deduced by analogy, and will not be elaborated here.

[0095] Business traffic represents the amount of data exchanged within the current time window. For example, transaction volume is determined based on business traffic. When the transaction volume exceeds a preset transaction volume threshold, an initial threshold is first determined based on the number of errors, and then adjusted upwards to obtain the quantity threshold. The transaction volume is positively correlated with the upward adjustment amount.

[0096] The above technical solution determines the quantity threshold corresponding to the target log attribute based on the business traffic and error count in the current time window. It can associate data anomaly detection with business traffic, avoid data fluctuations caused by business traffic, and improve the accuracy of anomaly detection.

[0097] S240, based on the anomaly screening criteria of each transaction channel and the current transaction log, determine whether there is an anomaly in the current transaction.

[0098] For example, the historical transaction logs and current transaction logs are divided according to the transaction channel to obtain the first historical logs and the logs to be detected for each transaction channel. Based on the distribution data of error codes of each error code category in the first historical logs, the anomaly filtering conditions corresponding to the error codes of each error code category under each transaction channel are determined. Further, the logs to be detected are checked according to the error codes of each error code category under each transaction channel to determine whether there are any anomalies in the current transactions of each transaction channel.

[0099] The exception filtering criteria include an error code whitelist and error codes to be filtered.

[0100] For example, determining whether there is an anomaly in the current transaction based on the anomaly screening conditions of each transaction channel and the current transaction log includes: dividing the current transaction log based on the transaction channel to obtain the second test log for each transaction channel; and detecting the second test log based on the error code whitelist and the error codes to be filtered to determine whether there is an anomaly in the current transaction of each transaction channel.

[0101] It should be understood that the error code whitelist includes whitelists under each transaction channel.

[0102] The above technical solution divides the current transaction logs based on the transaction channels to obtain the second test logs for each transaction channel. Based on the error code whitelist and the error codes to be filtered, the second test logs are checked to determine whether there are any anomalies in the current transactions of each transaction channel. By dividing the current transaction logs and historical transaction logs into logs of the same dimension, the test logs and anomaly screening conditions can be compared on the same dimension, thereby determining whether there are any anomalies in the current transactions of each transaction channel, which improves the accuracy of data anomaly detection.

[0103] Specifically, the current transaction log is divided based on the transaction channel to obtain the second test log for each transaction channel; the error codes in the second test log are filtered based on the error codes to be filtered to obtain the third test log. If a new error code is found in the third test log, it is determined that the current transaction of the corresponding transaction channel is abnormal; if no new error code is found in the third test log, it is determined that the current transaction of the corresponding transaction channel is not abnormal.

[0104] The third test log does not contain any error codes to be filtered. That is, by removing the error codes to be filtered from the second test log, the third test log is obtained, and therefore, the third test log does not contain any error codes to be filtered.

[0105] The newly added error code is not in the error code whitelist. That is, the newly added error code is used to represent the error code that exists in the third test log and is not in the error code whitelist.

[0106] The above technical solution filters error codes in the second test log based on the error codes to be filtered, resulting in a third test log. When a new error code is found in the third test log, it is determined that the current transaction of the corresponding trading channel is abnormal; when no new error code is found in the third test log, it is determined that the current transaction of the corresponding trading channel is not abnormal. By removing high-frequency error codes from the second test log and comparing the error codes in the removed test log with the error code whitelist, it is determined whether a new whitelist has been added. This enables rapid detection and alerting of error codes appearing for the first time, preventing the spread of unknown risks.

[0107] The anomaly screening criteria include quantity thresholds and percentage thresholds.

[0108] For example, determining whether a current transaction is abnormal based on the anomaly screening conditions of each transaction channel and the current transaction log includes: dividing the current transaction log based on the transaction channel and log attributes to obtain the first log to be detected under different log attributes in each transaction channel; and determining whether the current transaction with the target log attribute is abnormal based on the first log to be detected under the target log attribute in each transaction channel and the anomaly screening conditions.

[0109] It should be understood that the current transaction log is used to represent the transaction log of the current time window; the current time window and the preset time window have the same window length, and comparison is made through the same time window to ensure the accuracy of anomaly detection.

[0110] The above technical solution divides the current transaction logs based on transaction channels and log attributes to obtain the first logs to be detected under different log attributes in each transaction channel. Based on the first logs to be detected under the target log attributes in each transaction channel and the anomaly filtering conditions, it determines whether there are any anomalies in the current transaction for the target log attributes. By dividing the current transaction logs, it ensures that the comparison logs are in the same range, thereby improving the accuracy of data anomaly detection.

[0111] Specifically, the current transaction log is divided based on the transaction channel and log attributes to obtain the first log to be detected under different log attributes in each transaction channel. Based on the first log to be detected, the first number of errors and the first percentage of the first number of errors in the target error code in the target log attribute are determined. When the first number of errors is greater than the number threshold and the first percentage of the first number is greater than the percentage threshold, it is determined that the current transaction of the target log attribute is abnormal. When the first number of errors is less than or equal to the number threshold, or the first percentage of the first number is less than or equal to the percentage threshold, it is determined that the current transaction of the target log attribute is not abnormal.

[0112] The first error count represents the number of errors in the first log to be inspected corresponding to the target log attribute (i.e., any log attribute). The first count percentage represents the percentage of the number of transaction requests in the first log to be inspected corresponding to the target log attribute.

[0113] For example, given a first number of errors and a first percentage of errors, the system determines whether the first number of errors exceeds a quantity threshold and whether the first percentage of errors exceeds a percentage threshold in the anomaly filtering criteria. If the first number of errors exceeds the quantity threshold and the first percentage of errors exceeds the percentage threshold in the anomaly filtering criteria, the current transaction for the target log attribute is determined to be abnormal. An alarm is output to notify the user of the anomaly. If the first number of errors is less than or equal to the quantity threshold, or the first percentage of errors is less than or equal to the percentage threshold in the anomaly filtering criteria, the current transaction for the target log attribute is determined not to be abnormal, and the transaction log is then further inspected.

[0114] The above technical solution determines the first number of errors and the first percentage of the target error code in the target log attribute based on the first log to be detected, and detects the error number and the percentage of the target error code respectively, avoiding the error of single-dimensional quantity detection and improving the accuracy of data anomaly detection.

[0115] To overcome the limitations of traditional rules in complex pattern recognition, a Large Language Model (LLM) can be introduced for anomaly detection. Due to the large volume of raw log data, the high cost of minute-level inference, and uncontrollable latency, the current transaction log is divided into preset time windows as input granularity. For example, a 15-minute aggregation window can be used to divide the current transaction log into multiple aggregation windows. The transaction logs from these multiple aggregation windows are then input into the LLM to predict whether any anomalies exist in the transactions corresponding to each aggregation window.

[0116] The large language model is the trained model. Prompt words are constructed to input two types of samples into the large language model: normal historical window logs and suspected abnormal window logs. The model parameters are adjusted using this sample data. During prediction using the large language model, input features include: the error rate of the current window and the previous 10 windows, and the error rate of the corresponding window from the previous day. By predicting the input data using the large language model, it can determine whether the current window deviates from the normal pattern, thereby identifying whether the current transaction is abnormal.

[0117] In addition, to improve the accuracy of large language models in recognizing scenes with gradual changes or blurred boundaries, high-quality positive and negative samples can be added to adjust the large language model, or a confidence threshold can be introduced to filter low-deterministic judgments.

[0118] By introducing a large language model to perform semantic understanding and pattern discrimination of aggregated temporal features, intelligent identification of irregular and nonlinear anomalies is achieved, which improves the generalization ability to unknown anomalies and reduces the reliance on human experience.

[0119] The above technical solution divides historical transaction logs based on transaction channels to obtain the first historical log for each transaction channel. Based on the distribution data of target error codes in the first historical log, it determines the anomaly screening conditions for each transaction channel. Based on the anomaly screening conditions of each transaction channel and the current transaction log, it determines whether the current transaction has an anomaly. By finely dividing the historical transaction log into transaction channels and dynamically determining the anomaly screening conditions for each transaction channel based on the distribution data of error codes in the divided first historical log, it can dynamically plan the anomaly screening conditions in conjunction with the actual transaction log, improving the matching between the anomaly screening conditions and the actual transaction. On this basis, the current transaction log is detected through the anomaly screening conditions to determine whether the current transaction corresponding to the current transaction log has an anomaly, thus improving the accuracy of data anomaly detection.

[0120] Anomaly detection is a process that uses technical means to continuously monitor and compare data in online transaction flows (such as transaction amount, frequency, time, geographical location, device information, user behavior sequences, etc.) to identify behaviors that significantly deviate from normal patterns or historical baselines, and trigger early warning or intervention mechanisms. It should be understood that data anomaly detection in the embodiments of this specification is equivalent to data anomaly detection; that is, determining that the current transaction is abnormal means identifying that the current transaction deviates from the normal pattern or historical baseline, requiring the triggering of an early warning.

[0121] The following will combine Figure 3 and Figure 4 The method for detecting anomalies in the embodiments of this specification is explained in detail.

[0122] Figure 3 This is a schematic flowchart of a method for detecting anomalies provided in the embodiments of this specification.

[0123] For example, such as Figure 3As shown, the online summary log includes current transaction logs and historical transaction logs. The offline task platform processes historical transaction days to construct an error code whitelist and a list of error codes to be filtered. The error codes to be filtered may include multiple error codes, which are frequently occurring error codes; while the error code whitelist contains allowed or valid error codes. It should be understood that neither the error codes in the whitelist nor the error codes to be filtered will affect the normal operation of transactions.

[0124] For historical transaction logs, batch processing is performed using an offline task platform to divide them into historical logs for transaction channels and transaction institutions (i.e., the first historical logs). These divided historical logs are then cleaned and aggregated to generate an error code whitelist, which is then solidified into baseline rules and fed back to the sentinel system as a comparison benchmark. Furthermore, baseline feature statistics are performed on these divided historical logs, identifying error codes with a frequency exceeding 1 time per minute and secondary error codes as those to be filtered.

[0125] Based on the current transaction, current transaction logs are collected and divided into test logs for transaction channels and transaction institutions. The test logs are filtered using error codes of the same dimension (same transaction channel, same transaction institution), eliminating unfiltered error codes. The filtered error codes in the test logs are then compared with an error code whitelist. If all error codes in the filtered test logs belong to the whitelist, the transaction is confirmed to be normal, and testing continues. If a new error code not belonging to the whitelist appears in the filtered test logs, an anomaly is confirmed, triggering an alarm notification to achieve a "zero-tolerance" response. Based on this, rapid detection and alarming of newly encountered error codes prevent the spread of unknown risks.

[0126] This embodiment constructs and compares a whitelist baseline using a triplet of "transaction channel, transaction institution, and error code". It dynamically generates anomaly filtering conditions offline and combines them with the current transaction logs to quickly respond to the first occurrence of an error code, effectively covering low-frequency and edge-path anomalies.

[0127] Figure 4 This is a schematic flowchart illustrating another method for detecting anomalies provided in the embodiments of this specification.

[0128] For example, historical transaction logs can be online logs, routing logs, and audience characteristic logs. Historical transaction logs are categorized based on transaction channels, transaction institutions, and log attributes to obtain categorized historical logs (i.e., second historical logs). For instance, online logs are categorized based on transaction channels, transaction institutions, business identifiers, event identifiers, and product identifiers; routing logs are categorized based on transaction channels, transaction institutions, and routing rules; and audience characteristic logs are categorized based on transaction channels, transaction institutions, and audience characteristics.

[0129] Optionally, log attributes can be determined by any one of the following: business type, routing rules, or user characteristics. For example, when determining log attributes based on business type, the log attributes refer to the business identifier, event identifier, and product identifier.

[0130] Historical transaction logs are segmented to obtain baseline data with the same transaction channels, transaction institutions, and log attributes. Using a 1-minute time window, the number and percentage of the first error for the target error code in the current transaction log corresponding to the current time window are determined. The initial number and percentage of the target error code in the historical transaction logs of the ten preceding first time windows are determined; the average of the initial number of errors in the ten first time windows is determined as the total number of errors, and the average of the initial percentages in the ten first time windows is determined as the percentage of the total number of errors. Finally, the number and percentage of the target error code in the historical transaction logs of the second time window corresponding to the current time window of the previous day are also determined.

[0131] Furthermore, thresholds are determined based on data from short-term and periodic time windows. Based on the number and percentage of error codes in the first time window, the corresponding quantity and percentage thresholds for that time window are obtained. Similarly, based on the number and percentage of error codes in the second time window, the corresponding quantity and percentage thresholds for that time window are obtained. It should be understood that there is a correspondence between the number of error codes and the quantity thresholds, and a correspondence between the quantity percentage and the percentage thresholds. For example, when the number of errors in the first time window is 0, the quantity threshold can be 200; when the number of errors in the first time window is less than 3, the quantity threshold can be 300. As the number of errors in the first time window increases, the corresponding quantity thresholds determined based on the number of errors in the first time window gradually decrease to avoid frequent false alarms.

[0132] After determining the quantity threshold and the percentage threshold, if the number of first errors in the current time window is greater than the quantity thresholds for the two time windows, and the percentage of the first errors is greater than the percentage thresholds for the two time windows, the transaction in the current time window will be marked as abnormal and an alarm will be triggered.

[0133] In one example, when a transaction corresponding to the current time window is marked as abnormal, manual verification can be triggered. After manual review, it can be determined whether to trigger an alarm, thereby reducing the false trigger rate of anomaly detection and improving the accuracy of anomaly detection.

[0134] By using multi-granular dynamic baseline modeling and adaptive threshold adjustment, minute-level time-series baselines can be constructed in multi-dimensional scenarios. By integrating short-term sliding windows and periodic data as dual references, the sensitivity and anti-interference ability of anomaly detection are improved, while the false alarm rate of anomaly alarms is reduced.

[0135] It should be understood that the above examples are provided to help those skilled in the art understand the embodiments of this specification, and are not intended to limit the embodiments of this specification to the specific values ​​or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or variations based on the above examples, and such modifications or variations also fall within the scope of the embodiments of this specification.

[0136] The above text combined Figures 1 to 4 The methods provided in the embodiments of this specification are described in detail below; the following will be combined with Figure 5 and Figure 7 The apparatus embodiments described in this specification are described in detail. It should be understood that the apparatus in the embodiments of this specification can perform the various methods described in the foregoing embodiments of this specification, that is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0137] Figure 5 This is a schematic diagram of the structure of a data anomaly detection device provided in the embodiments of this specification.

[0138] For example, such as Figure 5 As shown, the data anomaly detection device 500 includes: Module 510 is used to obtain the current transaction log and historical transaction log; The partitioning module 520 is used to partition historical transaction logs based on transaction channels to obtain the first historical logs for each transaction channel; The determination module 530 is used to determine the abnormal screening conditions for each transaction channel based on the distribution data of the target error code in the first historical log. The target error code is used to represent any type of error code. The detection module 540 is used to determine whether there is an anomaly in the current transaction based on the anomaly screening conditions of each transaction channel and the current transaction log.

[0139] In one possible implementation, the determining module 530 is specifically used to divide the first historical log based on log attributes to obtain the second historical log for each log attribute. The log attributes are determined based on at least one of business identifier, product identifier, and event identifier. Based on the distribution data of target error codes in the second historical log, the abnormal filtering conditions of each transaction channel in different log attributes are determined.

[0140] In one possible implementation, the determining module 530 is specifically used to determine the number and percentage of target error codes within a preset time window based on the distribution data of target error codes in the second historical log. The percentage represents the proportion of the number of errors to the number of transaction requests within the preset time window. Based on the number of errors of target log attributes, the module determines the quantity threshold corresponding to the target log attribute, where the target log attribute represents any log attribute. Based on the percentage of the number of target log attributes, the module determines the percentage threshold corresponding to the target log attribute. The anomaly filtering conditions include the quantity threshold and the percentage threshold.

[0141] In one possible implementation, the preset time window includes a first time window and a second time window. The first time window is used to represent the time window adjacent to the current time window, and the second time window is used to represent the time window corresponding to the current time window within the previous preset period.

[0142] In one possible implementation, the determining module 530 is specifically used to obtain the business traffic of the target log attribute in the current time window; based on the business traffic and the number of errors, it determines the quantity threshold corresponding to the target log attribute.

[0143] In one possible implementation, the detection module 540 is specifically used to divide the current transaction log based on the transaction channel and log attributes to obtain the first log to be detected under each transaction channel with different log attributes. The current transaction log is used to represent the transaction log of the current time window. Based on the first log to be detected under the target log attribute under each transaction channel and the anomaly filtering conditions, it is determined whether there is an anomaly in the current transaction of the target log attribute.

[0144] In one possible implementation, the detection module 540 is specifically used to determine, based on the first log to be detected, the first number of errors and the first percentage of the target error code in the target log attribute; when the first number of errors is greater than the number threshold and the first percentage of the number is greater than the percentage threshold, determine that the current transaction of the target log attribute is abnormal; when the first number of errors is less than or equal to the number threshold, or the first percentage of the number is less than or equal to the percentage threshold, determine that the current transaction of the target log attribute is not abnormal.

[0145] In one possible implementation, the determining module 530 is specifically used to determine the historical processing items of the target error code in the first historical log; and to determine the abnormal screening conditions for each transaction channel based on the distribution data of the historical processing items and the target error code.

[0146] In one possible implementation, the determining module 530 is specifically used to determine an error code whitelist based on historical processing items; determine the error code category and / or error frequency of the target error code based on the distribution data of the target error code in the first historical log; and determine the error code to be filtered based on the error code category and / or error frequency of the target error code, wherein the anomaly filtering conditions include the error code whitelist and the error code to be filtered.

[0147] In one possible implementation, the determining module 530 is specifically used to determine the target error code as a first filtered error code when the error code category of the target error code is a subcategory; to determine the target error code as a second filtered error code when the error frequency of the target error code is greater than a preset frequency threshold; and to determine the error code to be filtered based on the first filtered error code and the second filtered error code.

[0148] In one possible implementation, the detection module 540 is specifically used to divide the current transaction log based on the transaction channel to obtain the second test log for each transaction channel; based on the error code whitelist and the error codes to be filtered, the second test log is detected to determine whether there are any anomalies in the current transaction of each transaction channel.

[0149] In one possible implementation, the detection module 540 is specifically used to filter the error codes in the second test log based on the error codes to be filtered, to obtain the third test log, in which there are no error codes to be filtered; when there are newly added error codes in the third test log, it is determined that the current transaction of the corresponding transaction channel is abnormal, and the newly added error code does not belong to the error code whitelist; when there are no newly added error codes in the third test log, it is determined that the current transaction of the corresponding transaction channel is not abnormal.

[0150] It should be noted that the data anomaly detection device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data anomaly detection device and the data anomaly detection method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0151] This specification provides a computer device for performing the above-described method. This computer device can be implemented as a terminal or a server. The structure of a terminal will be described below: Figure 6 This is a schematic diagram of the structure of a terminal provided in the embodiments of this specification.

[0152] Terminal 600 can be a smartphone, tablet, laptop, or desktop computer. Terminal 600 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0153] Typically, terminal 600 includes one or more processors 601 and one or more memories 602.

[0154] Processor 601 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen.

[0155] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one computer program, which is executed by the processor 601 to implement the data anomaly detection method provided in the method embodiments of this specification.

[0156] In some embodiments, the terminal 600 may optionally include a peripheral device interface 603 and at least one peripheral device. The processor 601, memory 602, and peripheral device interface 603 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 603 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, and a power supply 608.

[0157] Peripheral interface 603 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 601 and memory 602. In some embodiments, processor 601, memory 602 and peripheral interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 601, memory 602 and peripheral interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0158] The radio frequency (RF) circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc.

[0159] Display screen 605 is used to display a user interface (UI). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 605 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 601 for processing. In this case, display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard.

[0160] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal.

[0161] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 601 for processing, or input to the radio frequency circuit 604 to realize voice communication.

[0162] Power supply 608 is used to supply power to the various components in terminal 600. Power supply 608 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery.

[0163] In some embodiments, the terminal 600 further includes one or more sensors 609. The one or more sensors 609 include, but are not limited to, an accelerometer 610, a gyroscope 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.

[0164] Accelerometer 610 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established with terminal 600.

[0165] The gyroscope sensor 611 can detect the orientation and rotation angle of the terminal 600. The gyroscope sensor 611 can work in conjunction with the accelerometer sensor 610 to collect the user's 3D movements on the terminal 600.

[0166] The pressure sensor 612 can be installed on the side bezel of the terminal 600 and / or on the lower layer of the display screen 605. When the pressure sensor 612 is installed on the side bezel of the terminal 600, it can detect the user's grip signal on the terminal 600, and the processor 601 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612. When the pressure sensor 612 is installed on the lower layer of the display screen 605, the processor 601 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 605.

[0167] An optical sensor 613 is used to collect ambient light intensity. In one embodiment, a processor 601 can control the display brightness of a display screen 605 based on the ambient light intensity collected by the optical sensor 613.

[0168] The proximity sensor 614 is used to detect the distance between the user and the front of the terminal 600.

[0169] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on terminal 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0170] The aforementioned computer equipment can also be implemented as a server. The structure of a server is described below: Figure 7 This is a schematic diagram of the structure of a server provided in the embodiments of this specification.

[0171] Server 700 can vary considerably due to differences in configuration or performance. It may include one or more Central Processing Units (CPUs) 701 and one or more memories 702, wherein the one or more memories 702 store at least one computer program, which is loaded and executed by the one or more processors 701 to implement the methods provided in the various method embodiments described above. Of course, server 700 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. Server 700 may also include other components for implementing device functions, which will not be elaborated here.

[0172] The apparatus provided in the embodiments of this specification may specifically be a chip, component, or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a data anomaly detection method provided in the above embodiments.

[0173] This specification also provides a computer-readable storage medium storing computer program code, which, when run on a computer, causes the computer to execute the aforementioned method steps to implement the data anomaly detection method provided in the above embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, Digital Video Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), microdrives, and magneto-optical disks, read-only memory (ROMs), random access memory (RAMs), erasable programmable read-only memory (EPROMs), electrically erasable programmable read-only memory (EEPROMs), dynamic random access memory (DRAMs), video random access memory (VRAMs), flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0174] This specification also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the data anomaly detection method provided in the above embodiments.

[0175] The computer-readable storage medium, computer program product, or chip provided in this specification are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0176] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0177] In the embodiments provided in this specification, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0178] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this specification should be determined by the scope of the claims.

Claims

1. A method for detecting data anomalies, characterized in that, The method includes: Get the current transaction log and historical transaction log; The historical transaction logs are divided based on the transaction channels to obtain the first historical logs for each transaction channel; Based on the distribution data of the target error codes in the first historical log, the abnormality screening conditions for each of the transaction channels are determined, and the target error codes are used to represent any type of error code; Based on the anomaly screening criteria of each transaction channel and the current transaction log, it is determined whether the current transaction is abnormal.

2. The method according to claim 1, characterized in that, The determination of anomaly screening conditions for each transaction channel based on the distribution data of target error codes in the first historical log includes: The first historical log is divided based on the log attributes to obtain the second historical log for each of the log attributes; Based on the distribution data of the target error codes in the second historical log, the anomaly filtering conditions for each transaction channel under different log attributes are determined.

3. The method according to claim 2, characterized in that, The step of determining the anomaly filtering conditions for each transaction channel under different log attributes based on the distribution data of the target error codes in the second historical log includes: Based on the distribution data of the target error codes in the second historical log, the number of target error codes and their proportion within a preset time window are determined. The proportion is used to represent the percentage of the number of errors in the number of transaction requests within the preset time window. Based on the number of errors of the target log attribute, a quantity threshold corresponding to the target log attribute is determined, wherein the target log attribute is used to represent any log attribute; Based on the quantity ratio of the target log attribute, a percentage threshold corresponding to the target log attribute is determined, and the anomaly filtering conditions include the quantity threshold and the percentage threshold.

4. The method according to claim 3, characterized in that, The preset time window includes a first time window and a second time window. The first time window is used to represent the time window adjacent to the current time window, and the second time window is used to represent the time window corresponding to the current time window within the previous preset period.

5. The method according to claim 3, characterized in that, Determining the quantity threshold corresponding to the target log attribute based on the number of errors of the target log attribute includes: Get the business traffic for the current time window; Based on the business traffic and the number of errors, determine the quantity threshold corresponding to the target log attribute.

6. The method according to any one of claims 3 to 5, characterized in that, The determination of whether the current transaction is abnormal based on the anomaly screening conditions of each transaction channel and the current transaction log includes: The current transaction log is divided based on the transaction channel and the log attribute to obtain the first log to be detected under each transaction channel with different log attributes. The current transaction log is used to represent the transaction log of the current time window. Based on the first log to be detected under the target log attribute of each transaction channel and the anomaly filtering conditions, it is determined whether the current transaction of the target log attribute is abnormal.

7. The method according to claim 6, characterized in that, The step of determining whether the current transaction of the target log attribute is abnormal based on the first log to be detected under each transaction channel and the anomaly filtering conditions includes: Based on the first log to be detected, determine the first number of errors and the first percentage of the first number of the target error code in the target log attribute; When the number of the first errors is greater than the number threshold and the proportion of the first errors is greater than the proportion threshold, it is determined that the current transaction of the target log attribute is abnormal; When the first number of errors is less than or equal to the number threshold, or the first number percentage is less than or equal to the percentage threshold, it is determined that the current transaction of the target log attribute does not have any anomalies.

8. The method according to claim 1, characterized in that, The determination of anomaly screening conditions for each transaction channel based on the distribution data of target error codes in the first historical log includes: Determine the historical processing details of the target error code in the first historical log; Based on the distribution data of the historical processing items and the target error codes, the anomaly screening conditions for each of the transaction channels are determined.

9. The method according to claim 8, characterized in that, The method for determining the anomaly screening criteria for each transaction channel based on the distribution data of the historical processed items and the target error codes includes: Based on the aforementioned historical processing events, an error code whitelist is determined; Based on the distribution data of the target error codes in the first historical log, determine the error code category and / or error frequency of the target error codes; Based on the error code category and / or error frequency of the target error code, an error code to be filtered is determined, and the anomaly filtering conditions include the error code whitelist and the error code to be filtered.

10. The method according to claim 9, characterized in that, The process of determining the error code to be filtered based on the error code category and error frequency of the target error code includes: When the error code category of the target error code is a subcategory, the target error code is determined as the first filter error code; When the error frequency of the target error code is greater than a preset frequency threshold, the target error code is determined as the second filter error code; The error code to be filtered is determined based on the first filtering error code and the second filtering error code.

11. The method according to claim 9 or 10, characterized in that, The determination of whether the current transaction is abnormal based on the anomaly screening conditions of each transaction channel and the current transaction log includes: The current transaction log is divided based on the transaction channel to obtain the second test log for each transaction channel; Based on the error code whitelist and the error codes to be filtered, the second test log is inspected to determine whether there are any anomalies in the current transactions of each transaction channel.

12. The method according to claim 11, characterized in that, The step of detecting the second test log based on the error code whitelist and the error codes to be filtered, and determining whether there are any anomalies in the current transactions of each of the transaction channels, includes: The error codes in the second test log are filtered based on the error codes to be filtered to obtain the third test log, and the third test log does not contain the error codes to be filtered. When a new error code is found in the third test log, it is determined that the current transaction of the corresponding transaction channel is abnormal, and the new error code does not belong to the error code whitelist. If no new error codes are found in the third test log, it is determined that the current transaction in the corresponding transaction channel is not abnormal.

13. A data anomaly detection device, characterized in that, The device includes: The acquisition module is used to acquire the current transaction log and historical transaction log; The partitioning module is used to partition the historical transaction logs based on the transaction channel to obtain the first historical logs for each transaction channel; The determination module is used to determine the abnormal screening conditions for each of the transaction channels based on the distribution data of the target error codes in the first historical log, wherein the target error codes are used to represent any type of error code; The detection module is used to determine whether there is an anomaly in the current transaction based on the anomaly screening conditions of each transaction channel and the current transaction log.

14. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, the computer program being loaded and executed by the one or more processors to implement the method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the method as described in any one of claims 1 to 12.

Images