Anomaly tracing method, electronic device, and computer readable storage medium

Abstract

The present application provides an anomaly tracing method, an electronic device, and a computer readable storage medium. The method may comprise: acquiring a data stream, wherein the data stream comprises time record-based query data, time record-based anomaly logs, and time record-based raw data; determining whether the query data at each time node matches raw data at the corresponding time node; and if the query data at a first time node does not match the raw data at the first time node, determining an anomaly cause on the basis of the distribution of the anomaly logs in the data stream.

Description

Abnormal tracing method, electronic device and computer-readable storage medium

[0001] This application claims priority to Chinese patent application No. 2024103766548, filed on March 29, 2024, entitled “Abnormality Tracing Method, Electronic Device and Computer-Readable Storage Medium”. The contents disclosed in the above-mentioned Chinese patent application are hereby cited in their entirety as part of this application. Technical Field

[0002] The present application relates to the field of data processing technology, and in particular to an abnormality tracing method, an electronic device, and a computer-readable storage medium. Background Art

[0003] As the number of network users increases, the volume of business generated by them also increases. In such scenarios, it is inevitable that when users query business data, the query results may differ from their expectations. In such cases, users need to verify the cause of the discrepancy through manual offline calculations. However, this implementation method becomes more time-consuming as the amount of business data increases. Summary of the Invention

[0004] The purpose of this application is to provide an abnormality tracing method, an electronic device and a computer-readable storage medium, which can more efficiently determine the cause of the abnormality.

[0005] In a first aspect, the present invention provides an anomaly tracing method, comprising: obtaining a data stream, the data stream including query data of each time node, anomaly records of each time node, and source data of each time node; comparing the query data of each time node with the source data at its corresponding time node to determine whether the query data of each time node is accurate; if the query data of the first time node is inaccurate, analyzing the anomaly records in the data stream to determine the cause of the anomaly of the query data of the first time node.

[0006] Through the above implementation method, it is possible to determine whether there are anomalies in the query data by comparing it with the source data. If there are anomalies, the cause of the anomaly can be determined through pre-recorded anomalies. This implementation method does not require users to manually calculate massive amounts of data, thereby improving the efficiency of anomaly determination. Furthermore, it can also locate the cause of the anomaly more quickly based on the anomaly records.

[0007] In an optional embodiment, the abnormal records in the data stream are analyzed to determine the abnormal cause of the query data at the first time node, including: judging whether there are abnormal records at the first time node and before the first time node based on the abnormal records at each time node; if so, determining the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node.

[0008] In an optional embodiment, the query data includes a cumulative quantity; and the abnormal cause of the query data at the first time node is determined based on the abnormal record at the first time node or before the first time node, including: if the cumulative quantity of the first time node in the query data is less than the cumulative quantity recorded in the source data at the first time node, and there is an update abnormality in the abnormal record at the first time node or before the first time node, then determining that the abnormal cause of the query data at the first time node is a data update abnormality.

[0009] In an optional embodiment, determining the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node also includes: if the query data at the first time node is empty and there is an update abnormality in the abnormal records at the first time node or before the first time node, determining that the abnormal cause of the first time node is a data update abnormality.

[0010] In the above implementation, the cause of the abnormality can be located more quickly based on the previously recorded abnormality records, and the abnormality can be improved more quickly based on the determined abnormality cause.

[0011] In an optional embodiment, the abnormal cause of the query data at the first time node is determined based on the abnormal records at the first time node or before the first time node, including: if the query data at the first time node is empty, and there is no abnormal record at the first time node and before the first time node, determining that the abnormal cause of the query data at the first time node is data clearing.

[0012] In an optional embodiment, determining the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node includes: if there is no abnormal record at the first time node and before the first time node in the data stream, and the query data at the first time node is not empty, determining that the abnormal cause of the query data at the first time node is a data calculation error.

[0013] In the above implementation, even if there is no exception record, the cause of the exception can be determined based on the specific value of the query data, which makes it convenient for users to locate and correct the exception.

[0014] In an optional implementation, the method further includes: determining the occurrence time of the data calculation error based on the data stream of the time node before the first time node in the data stream.

[0015] In the above implementation, the time when the data calculation error occurred can also be determined through the data stream, which can help users more easily locate the error and change the erroneous data to better ensure the accuracy of subsequent data queries.

[0016] In an optional embodiment, determining the time when a data calculation error occurs based on the data stream of the time node before the first time node in the data stream includes: comparing the query data of the time node before the first time node with the source data of its corresponding time node to determine whether the query data of the time node before the first time node is accurate; if the query data of the time node before the first time node are all accurate, determining that the time when the data calculation error occurs is between the first time node and the time node before the first time node; if before the first time node, only the query data of the second time node is inaccurate, determining that the time when the data calculation error occurs is between the second time node and the time node before the second time node.

[0017] In an optional embodiment, the data stream also includes data calculation results for each time node, and the data calculation results include the cumulative number of businesses at each time node determined based on the source data; determining the time when the data calculation error occurs based on the data stream of the time node before the first time node in the data stream includes: comparing the query data of the time node before the first time node with the data calculation results of its corresponding time node to determine whether the data calculation results of the time node before the first time node are accurate; if before the first time node, only the data calculation result of the third time node is inaccurate, then determining that the time when the data calculation error occurs is between the third time node and the time node before the third time node.

[0018] In an optional embodiment, obtaining the data stream includes: determining the data stream associated with the key information based on the key information carried in the abnormal query request, wherein the key information includes one or more of a user identity identifier and an IP address.

[0019] In the above implementation, the reason for abnormality in the data of local users can be queried based on key information such as user identity and IP address.

[0020] In an optional embodiment, for the source data corresponding to the key information, at each time node formed at a preset time interval, the data calculation results of each time node are calculated based on the source data; the abnormal records of the key information at each time node are queried; the query data of the key information at each time node are queried; the key information, the source data, the data calculation results, the abnormal records and the query data are stored in chronological order to form a data stream corresponding to the key information.

[0021] In the above implementation, various types of data can be pre-arranged according to time nodes and time sequence, which can facilitate the rapid location of abnormal causes when it is necessary to locate the abnormal causes.

[0022] In a second aspect, the present invention provides an anomaly tracing device, comprising: an acquisition module for acquiring a data stream, wherein the data stream includes query data of each time node, an anomaly record of each time node, and source data of each time node; a comparison module for comparing the query data at each time node with the source data at its corresponding time node to determine whether the query data at each time node is accurate; and a determination module for analyzing the anomaly record in the data stream to determine the cause of the anomaly of the query data at the first time node if the query data at the first time node is inaccurate.

[0023] In a third aspect, the present invention provides an electronic device comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in any one of the aforementioned embodiments.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which executes the steps of the method described in any one of the aforementioned embodiments when the computer program is executed by a processor.

[0025] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in any one of the aforementioned embodiments when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0027] FIG1 is a schematic diagram of interaction between a server and a user terminal provided in an embodiment of the present application;

[0028] FIG2 is a block diagram of an electronic device provided in an embodiment of the present application;

[0029] FIG3 is a flow chart of an abnormality tracing method provided in an embodiment of the present application;

[0030] FIG4 is a partial flow chart of the abnormality tracing method provided in an embodiment of the present application;

[0031] FIG5 is a schematic diagram of a visualized data flow in an example provided in an embodiment of the present application;

[0032] FIG6 is a schematic diagram of data flow provided in an embodiment of the present application;

[0033] FIG7 is a schematic diagram of the functional modules of the abnormality tracing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0035] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0036] In risk control operations, transactions, queries, and other services can be conducted online. However, as data volumes increase, determining the cause of query data anomalies increases. The current approach to determining the cause of anomalies in query data involves offline calculations on the source data for risk control operations and by collecting and comparing logs generated during the risk control process. This approach results in a relatively long time and workload to determine the cause of the anomaly, which in turn results in a relatively prolonged impact on online risk control operations. Furthermore, the anomalies recorded in the business logs are merely transient anomalies that occur in real time during the operation and may not directly lead to abnormal query data results.

[0037] Based on the above research, the embodiments of the present application can provide an anomaly tracing method, electronic device and computer-readable storage medium, which can achieve relatively rapid determination of anomalies and their causes through comparison of data streams.

[0038] To facilitate understanding of this embodiment, the operating environment of an exception tracing method disclosed in the embodiment of this application is first introduced in detail.

[0039] FIG1 is a schematic diagram illustrating the interaction between a server 110 and a user terminal 120 according to an embodiment of the present application. The server 110 communicates with one or more user terminals 120 via a network for data communication or interaction. The server 110 may be a network server 110, a database server 110, or the like. The user terminal 120 may be a personal computer (PC), a tablet computer, a smartphone, a personal digital assistant (PDA), or the like.

[0040] The user terminal 120 may provide a query interface through which an exception query request is sent to the server 110 to implement exception tracing. Upon receiving the exception query request, the server 110 may be configured to execute the steps of the exception tracing method provided in the embodiment of the present application to determine the cause of the exception in the query data.

[0041] Optionally, the server 110 can also be configured to execute the steps of the anomaly tracing method according to a set time period to determine whether the query data contains an anomaly, and if so, to further determine the cause of the anomaly. For example, the set time period can be every specified time period. The specified time period can be a week, a day, twelve hours, an hour, or the like. Of course, the specified time period can also be set based on actual needs.

[0042] As shown in Figure 2, it is a block diagram of an electronic device. The electronic device 200 may include a memory 211 and a processor 213. Those skilled in the art will understand that the structure shown in Figure 2 is merely illustrative and does not limit the structure of the electronic device 200. For example, the electronic device 200 may also include more or fewer components than those shown in Figure 2, or have a configuration different from that shown in Figure 2. Among them, the server 110 and the user terminal 120 shown in Figure 1 may include the components shown in the electronic device 200 shown in Figure 2. Of course, based on different functions, the server 110 and the user terminal 120 may also include more components than the electronic device 200 shown in Figure 2. For example, the server 110 may also include components such as a positioning system and a communication unit, and the user terminal 120 may also include components such as a display unit and a positioning system.

[0043] The memory 211 and processor 213 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines. The processor 213 is used to execute the executable modules stored in the memory.

[0044] The memory 211 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable read-only memory (PROM), or a programmable read-only memory (PROM).

[0045] (Erasable Programmable Read-Only Memory, referred to as EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. Among them, the memory 211 is used to store programs, and the processor 213 executes the programs after receiving the execution instruction. The method executed by the electronic device 200 defined by the process disclosed in any embodiment of the present application can be applied to the processor 213 or implemented by the processor 213.

[0046] The processor 213 may be an integrated circuit chip with signal processing capabilities. The processor 213 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor.

[0047] The electronic device 200 in this embodiment can be used to execute each step in each method provided in the embodiments of the present application. The following describes in detail the implementation process of the abnormality tracing method through several embodiments.

[0048] Please refer to Figure 3, which is a flow chart of the anomaly tracing method provided in an embodiment of the present application. The anomaly tracing method provided in an embodiment of the present application can be applied to an electronic device, and the steps of the anomaly tracing method are performed by the electronic device. The specific process shown in Figure 3 will be described in detail below.

[0049] Step 310: Acquire data stream.

[0050] The data stream includes query data at each time node, exception records at each time node, and source data at each time node.

[0051] Among them, each time node can be a predefined time node, or a time node formed by initiating a query request corresponding to the query data, a time node formed based on an abnormal record, etc.

[0052] Optionally, each query data item may be stored in association with the query time of the query data when it is generated, so as to form query data for each time node. The source data may be stored according to a plurality of preset time stages, so as to form source data recorded with the latest time point of the time stage as the time node.

[0053] In this embodiment, the source data may represent original business data. For example, the method of the present application may be used in a trading website to determine the cause of anomalies in query data. The original business data may represent order data generated by the trading website.

[0054] Query data may be the result of a query operation indexed by key information and limiting information. Key information may include, but is not limited to, user identification, IP address, and other information. Limiting information may include, but is not limited to, time range, query category, and other information. User identification may include, but is not limited to, user account name, user ID number, and user account number.

[0055] For example, the query data may be a query request initiated based on a need to view source data, and a query result generated based on the query request. For example, the query data may be the cumulative number of orders, cumulative order amount, cumulative sales volume of goods, cumulative sales volume of specified goods, etc.

[0056] Exception records can be generated by associating exceptions that occur during business operations with the time they occurred, creating a time-based exception record. Exception records can also be generated by maintaining source data. For example, this could be a transaction conducted on a trading website. Source data maintenance can include statistical processing and storage of source data.

[0057] For example, an exception record may be an exception generated when storing source data, such as a source data storage failure, a source data storage location access failure, a source data storage request timeout, etc. An exception record may be an exception generated during the computation of the source data, such as a source data access failure when computing the required source data, etc. An exception record may also be an exception generated when storing a result computed from the source data, such as a computation result storage failure, a computation result storage location access failure, a computation result storage request timeout, etc.

[0058] Optionally, the condition for triggering anomaly tracing may be an actively initiated anomaly query request, and anomaly tracing is initiated based on the anomaly query request. Exemplarily, the anomaly query request may be submitted through an interactive interface of an electronic device that executes the method, or may be submitted by a user terminal that is in communication with the electronic device.

[0059] Optionally, the abnormal query request may carry key information. The key information may be information such as a user identity, IP address, etc. The user identity may be an identifier such as a user account, user mobile phone number, user ID number, etc.

[0060] Based on the key information carried in the abnormal query request, the data stream associated with the key information is determined. Subsequent steps 320 and 330 can determine whether the query data of the key information is abnormal based on the data stream associated with the key information.

[0061] Optionally, the condition for triggering exception tracing can also be the occurrence of a monitored query anomaly. Optionally, multiple tracing time nodes are pre-set, and exception tracing is initiated at each tracing time node. The tracing time nodes can be set as needed, for example, three, five, and so on, tracing time nodes can be set per day.

[0062] In this embodiment, after the exception tracing is triggered, the exception tracing may be implemented by executing the method defined in steps 310 to 330 .

[0063] Step 320 : Compare the query data at each time node with the source data at its corresponding time node to determine whether the query data at each time node is accurate.

[0064] If the query data at the first time node is inaccurate, it means that the query data obtained at the current first time node is abnormal, and step 330 may be executed to screen the cause of the abnormality.

[0065] If the query data at each time node is accurate, it means that the query data at each time node is normal, and the exception tracing process can be ended.

[0066] The exact method of determining the query data may vary depending on the query data.

[0067] If the query data is numerical data, it can be determined whether the query data of the time node is equal to the corresponding value of the source data at the time node. If they are equal, it can be determined that the query data of the time node is accurate.

[0068] For example, the query data may include the cumulative number of orders generated from the start time to a specified time point. The cumulative number of orders obtained from the query data at each time point can be checked to see if it is the same as the cumulative number of orders in the source data at that time point. If they are the same, it indicates that the query data matches the source data. The start time can be set as needed, for example, 00:00 every day; another example, 08:00 every day; another example, the first day of each month, etc.

[0069] Exemplarily, the query data may include the cumulative consumption amount generated from the start time to the specified time node. The cumulative consumption amount obtained through the query data of each time node can be determined to be the same as the cumulative consumption amount of the source data of the time node. If they are the same, it can be determined that the query data of the time node is accurate.

[0070] If the query data is original business data, it can be determined whether the number of original business data items contained in the query data at the time node is the same as the number of original business data items in the source data at the time node. If the number of items is the same, it can be determined that the query data at the time node is accurate.

[0071] Step 330: Analyze the abnormal records in the data stream to determine the abnormal cause of the query data at the first time node.

[0072] If there is an abnormal record, it can be determined based on the abnormal record whether the query data at the first time node is inaccurate due to the abnormality recorded in the abnormal record.

[0073] If no abnormal record exists, it is possible to first determine whether there are errors in the data in the source database of the query data, thereby determining the cause of the abnormal query data at the first time point. For example, it is possible to determine whether the data exists in the source database of the query data, or whether there are errors in the data in the source database of the query data.

[0074] Optionally, after determining the abnormal reason for the abnormality in the query data at the first time node, the abnormal reason may be sent to the user terminal that sent the abnormal query request, so as to be displayed on a display interface of the user terminal.

[0075] This method allows you to compare source data with exception records to determine if an exception has occurred. This reduces the time required to locate the cause of an exception compared to manual calculations. Rapidly identifying the cause of an exception also allows for faster resolution of query anomalies and prevention of potential query anomalies, reducing service delays caused by query anomalies in the application scenarios of this anomaly tracing method.

[0076] In one embodiment, the above-mentioned step 330 may include: judging whether there are abnormal records at the first time node and before the first time node based on the abnormal records of each time node; if so, determining the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node.

[0077] In this embodiment, if there are abnormal records before or at the first time node, it is possible to first determine whether the abnormalities represented by these abnormal records will cause abnormalities in the query data. For example, if the abnormal record indicates that some data storage has failed, and this part of the data will be output as query data to respond to the query request, then it can be determined that the abnormal record will cause abnormalities in the query data. For example, if the abnormal record indicates that the source data storage has failed, and the source data will not be output as query data to respond to the query request, then it can be determined that the abnormal record will not cause abnormalities in the query data. For example, if the abnormal record indicates that access to the storage address failed when storing data, and the stored data will also be output as query data, then it can be determined that the abnormal record will cause abnormalities in the query data.

[0078] If the anomaly represented by the anomaly record at or before the first time node causes an anomaly in the query data, it can be determined that the anomaly represented by the anomaly record is the cause of the anomaly in the query data at the first time node.

[0079] Optionally, the query data may include a cumulative quantity; the exception in the exception record may include an update exception. Taking the application scenario of the exception tracing method as a trading system as an example, the cumulative quantity may be the cumulative number of trading orders generated before the time point when the query data is triggered.

[0080] Exemplarily, an update anomaly may represent an anomaly caused when the latest data is stored in the source database of the query data. For example, an update anomaly may include abnormal records such as the latest data stored is empty or the latest data storage fails.

[0081] The above-mentioned determination of the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node includes: if the cumulative number of the first time node in the query data is less than the cumulative number recorded in the source data at the first time node, and there is an update abnormality in the abnormal records at the first time node or before the first time node, then determining that the abnormal cause of the query data at the first time node is a data update abnormality.

[0082] If the cumulative quantity in the query data at the first time node is less than the cumulative quantity recorded in the source data at the first time node, this may indicate that the query data obtained at the first time node was not generated based on the complete first time node and the complete transaction orders before the first time node. This may indicate that there were transaction orders that were not counted before the query data was obtained, or that there was an error in counting the transaction orders at the first time node and the first time node. Based on this, if there is an update anomaly at or before the first time node, this may indicate that the data in the source database for query is not fully included in the transaction orders accumulated at and before the first time node due to the update anomaly, resulting in the query data obtained at the first time node not being generated based on the complete transaction orders accumulated at and before the first time node.

[0083] Optionally, the exception record may include query exceptions and update exceptions.

[0084] The above-mentioned determination of the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node may include: if the query data at the first time node is empty, and there is an update abnormality in the abnormal records at the first time node or before the first time node, then determining that the abnormal cause of the first time node is a data update abnormality.

[0085] The above-mentioned determination of the abnormal cause of the query data at the first time node based on the abnormal records at the first time node or before the first time node may include: if the query data at the first time node is empty, and there is no query update abnormality in the abnormal records at the first time node or before the first time node, but there is an update abnormality in the abnormal records at the first time node or before the first time node, then it can be determined that the abnormal cause of the first time node is data update abnormality and data clearing.

[0086] In this embodiment, if the query data at the first time point is empty, it means that no data can be queried at the first time point, and the reason for this may be that the data does not exist in the source database. The reason for the absence of data in the source database may be that the source database is updated incorrectly, resulting in no data. It may also be that the source database is updated correctly, but the data has expired due to a long storage period, resulting in the data being cleared.

[0087] Furthermore, if the query data before the first time node are normal, but the query data at the first time node is abnormal, it may mean that the latest data has not been updated to the source database in a timely manner, and the historical data has been cleared, resulting in the inability to query data at the first time node.

[0088] In another embodiment, the above step 330 may include: if there is no abnormal record at the first time node and before the first time node, and the query data at the first time node is empty, determining that the cause of the abnormality is data clearing.

[0089] Furthermore, if the query data before the first time node are all normal, it may mean that the data in the source database are all normal before the first time node. However, if the data cannot be queried at the first time node, it may mean that the first time node and the data before the first time node in the source database are cleared, resulting in the inability to query data at the first time node.

[0090] In another embodiment, the above-mentioned step 330 may include: if there is no abnormal record in the data stream at the first time node and before the first time node, and the query data of the first time node is not empty, determining that the abnormal cause of the query data of the first time node is a data calculation error.

[0091] If there are no abnormal records at the first time node and before the first time node, it means that there are no abnormalities in the data management of the source database at the first time node and before the first time node. However, if there are abnormalities in the query data of the first time node, it may mean that there are errors in the statistical data of the first time node of the source database.

[0092] In order to better correct the anomalies in the query data and reduce the probability of anomalies in subsequent queries, the anomaly tracing method in the embodiment of the present application may also include: determining the time when the data calculation error occurred based on the data stream of the time node before the first time node in the data stream.

[0093] Optionally, the query data of the time node before the first time node may be compared with the source data of the corresponding time node to determine whether the query data of the time node before the first time node is accurate, so as to determine the time when the data calculation error occurs.

[0094] The method of determining whether the query data of the time nodes before the first time node is accurate is the same as the method of determining whether the query data of each time node in step 320 described above is accurate, and will not be repeated here.

[0095] Exemplarily, the query data before the first time node may be compared with the source data to determine the time node when the query data anomaly occurs, and then the time when the data calculation error occurs may be determined based on the time node when the query anomaly occurs.

[0096] If the query data of the time node before the first time node is accurate, it can be determined that the time when the data calculation error occurs is between the first time node and the time node before the first time node.

[0097] If, before the first time node, only the query data of the second time node is inaccurate, then it is determined that the time when the data calculation error occurs is between the second time node and a time node immediately before the second time node.

[0098] In more scenarios, there may be multiple time nodes before the first time node where the query data is inaccurate. The time when the data calculation error occurs can be determined based on the multiple time nodes where the query data is inaccurate.

[0099] For example, if the query data of multiple consecutive time nodes before the first time node all have anomalies, the earliest time node among the multiple consecutive time nodes can be determined as the time when the data calculation error occurred; or, the time between the earliest time node among the multiple consecutive time nodes and the previous time node of the earliest time node can be determined as the time when the data calculation error occurred.

[0100] For example, if the query data of multiple time nodes that are not completely continuous before the first time node all have anomalies, it can mean that there is a data calculation error in the earliest time node among the multiple time nodes that are not completely continuous, but the data calculation in the source database has been corrected. In this case, the earliest time node among the continuous time nodes formed by the first time node can be determined as the time when the data calculation error occurred, or the time between the earliest time node among the continuous time nodes formed by the first time node and the previous time node of the earliest time node can be determined as the time when the data calculation error occurred; if the query data of the time nodes adjacent to the first time node do not have anomalies, it can be determined that the time when the data calculation error occurred is between the first time node and the previous time node of the first time node.

[0101] In this embodiment, the data stream may further include data calculation results of the time record, and the data calculation results include the cumulative number of services at different time nodes determined based on the source data.

[0102] Optionally, the query data of the time node before the first time node can be compared with the data calculation results of the corresponding time node to determine whether the data calculation results of the time node before the first time node are accurate, and then determine the time when the data calculation error occurs.

[0103] If, before the first time node, only the data calculation result of the third time node is inaccurate, it is determined that the occurrence time of the data calculation error is between the third time node and a time node immediately preceding the third time node.

[0104] For example, if the data calculation result of the third time node before the first time node obtains a cumulative number of businesses that is not equal to the cumulative number of businesses contained in the source data of the third time node, it may indicate that the data calculation result of the third time node before the first time node is inaccurate.

[0105] The above example describes a case where only a single time point has an inaccurate data calculation result. In some cases, the data calculation results may be inaccurate at multiple time points. In this case, the time when the data calculation error occurred can be determined based on the time points where the multiple data calculation results are inaccurate.

[0106] For example, if the data calculation results of multiple consecutive time nodes before the first time node are inaccurate, the earliest time node among the multiple consecutive time nodes can be determined as the time when the data calculation error occurred; or, the time between the earliest time node among the multiple consecutive time nodes and the previous time node of the earliest time node can be determined as the time when the data calculation error occurred.

[0107] For example, if the data calculation results of multiple incompletely continuous time nodes before the first time node are all inaccurate, it can be said that there is a data calculation error in the earliest time node among the multiple incompletely continuous time nodes, but the data calculation in the source database has been corrected. In this case, the earliest time node among the continuous time nodes formed by the first time node can be determined as the time when the data calculation error occurred, or the time between the earliest time node among the continuous time nodes formed by the first time node and the previous time node of the earliest time node can be determined as the time when the data calculation error occurred; if the data calculation results of the time nodes adjacent to the first time node are accurate, it can be determined that the time when the data calculation error occurred is between the first time node and the previous time node of the first time node.

[0108] Through the above implementation method, not only can the cause of the abnormality in the query data be located, but also the time when the data calculation error occurs can be determined when the data calculation anomaly is determined. In this way, the source database of the data calculation error can be updated more promptly, so that the data obtained from subsequent data queries can be more accurate.

[0109] In this embodiment, data streams may be prepared in advance before executing steps 310 to 330. As shown in FIG4 , the data streams are determined by the following steps 410 to 430.

[0110] Step 410 , for the source data corresponding to the key information, at each time node formed at a preset time interval, calculate a data calculation result for each time node based on the source data.

[0111] For example, the data calculation results may include the cumulative order amount, cumulative order quantity, cumulative order amount, cumulative product sales, cumulative sales of a specified product, etc. The data type may also vary depending on the data calculation result. For example, the data type may include integer, floating point, string, etc.

[0112] Optionally, after obtaining the data calculation results, they can be stored in the source database. The aforementioned query data can also be obtained by querying the source database.

[0113] Optionally, the data calculation result can be achieved through online calculation. The source data can be calculated every first set time period to obtain the data calculation result. For example, the first set time period can be 1 second, 5 seconds, 10 seconds, etc.

[0114] Optionally, the data calculation result can be achieved through offline calculation, and the source data can be calculated every second set time period to obtain the data calculation result. For example, the second set time period can be 1 hour, 5 hours, 1 day, etc.

[0115] Step 420: Query abnormal records of key information at each time point.

[0116] The abnormal record may be generated during the management of the source database of the query data.

[0117] For example, it may be an abnormal record generated when calculating various business data corresponding to the key information before storing it in the source database, or it may be an abnormal record generated during the process of storing the data in the source database.

[0118] The exception records may include data calculation anomalies, data storage anomalies, and data query anomalies. Data calculation anomalies may include missed orders, duplicate orders, order calculation errors, etc. Data storage anomalies may include storage failures, storage location access errors, storage request response timeouts, etc. Data query anomalies may include empty query results, query request failures, query link failures, etc.

[0119] Step 430: Query the query data of key information at each time node.

[0120] The query data may be based on data generated by a query request obtained from a user terminal using key information as an index.

[0121] Exemplarily, the query data may be obtained from a database that records query data.

[0122] In step 440 , the key information, source data, data calculation results, exception records, and query data are stored in chronological order to form a data stream corresponding to the key information.

[0123] For example, the source data, data calculation results, exception records, and query data can be arranged in time sequence to obtain a data stream. For example, for each time node, the source data, data calculation results, exception records, and query data at that time node can be bound. For another example, each data item can be stored in chronological order, with the data at each time node being stored.

[0124] Optionally, the data stream corresponding to each key information is stored separately to construct a data stream for each key information.

[0125] Exemplarily, the data flow for each IP address may be stored separately. Exemplarily, the data flow for each user identification may be stored separately.

[0126] In order to avoid the data volume from becoming larger and larger, a data window can be set for the data stream. For example, the size of the data window of the data stream can be limited by time. Each group of data streams can only contain one data window to obtain data.

[0127] Optionally, the size of the data window can be set as needed. The size of the data window can be determined based on the time period typically targeted by query requests. For example, if a user's query requirements are generally for data for the same day, the size of the data window can be equal to or greater than a day, such as 24 hours, 25 hours, or 42 hours. For example, if a user's query requirements are generally for data within a week, the size of the data window can be equal to or greater than a week, such as one week, eight days, or ten days.

[0128] Through this data stream, you can quickly locate the data calculation results, source data, query data, and exception records at each time point. By locating this data, you can determine the cause of the exception and locate the time when the exception occurred.

[0129] Figure 5 shows a schematic diagram of a visualized data flow in an example. The example shown in Figure 5 includes information such as a timeline, source data for each time node on the timeline, data calculation results, and query data. The following describes the data contained in each node in the data flow and the relationships between the data, using the example shown in Figure 5 as an example.

[0130] In the example shown in Figure 5, the time axis includes seven time nodes t0, t1, t2, t3, t4, t5 and t6, as well as the source data d0, d1, d2, d3, d4, d5 and d6 contained in each time node, as well as the data calculation results v0, v1, v2, v3, v4, v5 and v6 contained in each time node, and the query data q0, q1, q2, q3, q4, q5 and q6 contained in each time node.

[0131] For source data, the time node of the source data indicates the time when the corresponding order data transaction occurred. For data calculation results, the time node of the data calculation result indicates the time when the data calculation result was output. For query data, the time node of the query data indicates the time when the query request was made.

[0132] In the example shown in FIG5 , the source data di may represent the cumulative order quantity generated at and before time node ti, where i is any value greater than or equal to zero and less than or equal to 6.

[0133] In the example shown in FIG5 , the data calculation result vi may represent the cumulative order quantity calculated at time node ti, where i is any value greater than or equal to zero and less than or equal to 6.

[0134] In the example shown in FIG5 , the query data qi may represent the cumulative order quantity obtained by querying at time node ti, where i is any value greater than or equal to zero and less than or equal to 6.

[0135] It will be appreciated that the examples shown in FIG5 with t0, t1, ..., t2, ..., d1, d2, ..., v1, v2, ..., q1, q2, ... are for illustration purposes only. In actual use, more time nodes may be included, and each time node may contain more data. Each set of data may also include a time axis. For example, the time interval of the source data time axis may be shorter, while the time interval of the query data time axis may be longer.

[0136] The following uses an example with reference to the data flow diagram shown in FIG5 to illustrate the process of determining the possible cause of an abnormality in query data.

[0137] For example, as of time node t4, the user's cumulative order volume for the day is 100. That is, the cumulative order volume for day d4 recorded at time node t4 in the source data is 100. If v4 = 100, the data calculation result obtained at time node t4 is correct; if q4 = 100, the query data obtained by the query request at time node t4 is also correct.

[0138] If q4≠100, it means that the query data obtained by the query request at time node t4 is incorrect.

[0139] Exemplarily, if q4=null and there is a query exception in the exception record before the t4 time node, the cause of the query exception is determined to be a request query exception; if there is no query exception in the exception record before the t4 time node, and there is an update exception, the cause of the query exception is determined to be an update exception of new data in the source database at the first time node, and the historical data in the source database before the first time node expires and is automatically cleared; if there is no query exception and no update exception in the exception record before the t4 time node, the cause of the query exception is determined to be that the data queried in the source database at the first time node expires and is cleared.

[0140] For example, q4<100, if there is an update anomaly in the abnormal record before the t4 time node, it is determined that the cause of the query anomaly is an update anomaly when the source database updates the database at the first time node or before the first time node; if there is no update anomaly in the abnormal record before the t4 time node, it is determined that the cause of the query anomaly is an error in the data calculation result.

[0141] For example, if q4>100, it is determined that the cause of the query anomaly is an error in the data calculation result.

[0142] In the case of incorrect data calculation results, the data calculation results v0 to v4 can be sorted according to the time sequence of t0 to t4 to locate the time node of the data calculation results.

[0143] By comparing the calculation result of the calculation result time node with the source data of the time node, the type of calculation result error can be determined. The error type can include missing calculation, repeated calculation, etc.

[0144] Optionally, the user terminal that sent the exception query request may display the timing diagram shown in FIG5 . After determining the cause of the exception, the cause of the exception may be sent to the user terminal that sent the exception query request, so that the cause of the exception and the time when the cause of the exception occurred are displayed in the timing diagram of the user terminal. For example, the cause of the exception may be displayed at the corresponding time position in the timing diagram.

[0145] As shown in FIG6 , the following describes the anomaly tracing method provided in an embodiment of the present application in conjunction with a data flow diagram.

[0146] After obtaining the source data, the feature calculation task is used to perform calculations to obtain data calculation results, which are then stored in a temporary storage location. In the example shown in the figure, the temporary storage location is Kafka.

[0147] After determining that the data calculation results are stored in a temporary storage location, the data can be accessed by the data bus. The data bus can obtain the latest data and exception records generated during the data calculation and data storage process and store them in the latest results database. Data services are provided based on this latest results database. These data services may include servers for data query and data modification. Data query data and exception records generated during the data service process can also be stored in the temporary storage location.

[0148] When a data stream needs to be constructed, various data can be collected from various temporary storage locations and the time-series-based data can be stored in a database. For example, the collected data can be stored in a source data database, an update exception record database, a calculation result database, a query data database, and a query exception record database.

[0149] In this embodiment, the calculation result database can serve as a source database for query data. When a query request needs to be responded to, query data can be obtained from the calculation result database. The obtained query data can be stored in a query data database to record the query situation.

[0150] Through data aggregation, the required data is obtained from various databases based on key information and window size. The aggregated data is then arranged in time series to obtain a data stream.

[0151] By comparing the data in the data stream, the cause of the anomaly can be determined, and finally the anomaly can be located by tracing back the anomaly cause.

[0152] Through the method in the embodiment of the present application, storage can be implemented for various types of data source data, data calculation results, query data, update anomalies, and query anomaly classifications. Then, data aggregation can be implemented according to the limitations of key information and window size, and a data stream can be constructed based on the aggregated data to serve as analysis data for query data anomalies. Compared with the prior art method of locating the cause of query anomalies by calculation by business personnel, a relatively accurate cause can be obtained in a shorter time. The analysis of the cause of query anomalies is efficient and reduces the impact on business. And by displaying the cause of the anomaly on the user terminal, the cause of the anomaly and the time of occurrence can be visualized, making the presentation of the cause of the anomaly more convincing.

[0153] Based on the same application concept, the embodiments of the present application also provide an exception tracing device corresponding to the exception tracing method. Since the principle of solving the problem by the device in the embodiments of the present application is similar to that of the aforementioned exception tracing method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiments of the aforementioned method, and the repeated parts will not be repeated.

[0154] Please refer to Figure 7, which is a functional module diagram of the anomaly tracing device provided in an embodiment of the present application. The various modules in the anomaly tracing device in this embodiment are used to execute the various steps in the above-mentioned method embodiment. The anomaly tracing device includes an acquisition module 510, a comparison module 520, and a determination module 530; the contents of each module are as follows: the acquisition module 510 is used to acquire a data stream, the data stream includes query data of each time node, anomaly records of each time node, and source data of each time node; the comparison module 520 is used to compare the query data at each time node with the source data at its corresponding time node to determine whether the query data at each time node is accurate; the determination module 530 is used to analyze the anomaly records in the data stream if the query data of the first time node is inaccurate, and determine the cause of the anomaly of the query data of the first time node.

[0155] The various modules included in the anomaly tracing device provided in this embodiment can be used to execute the various steps in the above-mentioned anomaly tracing method. Therefore, more details of the anomaly tracing device can be found in the steps implemented in the above-mentioned method, which will not be described in detail here.

[0156] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the anomaly tracing method described in the above method embodiment are executed.

[0157] The computer program product of the anomaly tracing method provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the anomaly tracing method described in the above method embodiments. For details, please refer to the above method embodiments and will not be repeated here.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0159] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0160] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0161] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0162] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An abnormality tracing method, characterized in that: include: Obtaining a data stream, wherein the data stream includes query data at each time node, exception records at each time node, and source data at each time node; Comparing the query data at each time node with the source data at its corresponding time node to determine whether the query data at each time node is accurate; If the query data at the first time node is inaccurate, the abnormal records in the data stream are analyzed to determine the abnormal cause of the query data at the first time node.

2. The method according to claim 1, characterized in that The analyzing the abnormal records in the data stream to determine the abnormal cause of the query data at the first time node includes: According to the abnormal records at each time node, determining whether there is an abnormal record before or before the first time node; If so, the abnormal cause of the query data at the first time node is determined based on the abnormal record at or before the first time node.

3. The method according to claim 2, characterized in that The query data includes cumulative quantity; the anomaly in the anomaly record includes update anomaly; The determining, based on the abnormal records at or before the first time node, the abnormal cause of the query data at the first time node includes: If the cumulative number of the first time node in the query data is less than the cumulative number recorded in the source data at the first time node, and there is an update exception record in the exception record at the first time node or before the first time node, then it is determined that the abnormal cause of the query data at the first time node is a data update exception.

4. The method according to claim 3, characterized in that The determining, based on the abnormal records at or before the first time node, a cause of abnormality in the query data at the first time node, further includes: If the query data of the first time node is empty, and there is an update exception in the exception record of the first time node or before the first time node, it is determined that the cause of the exception at the first time node is a data update exception.

5. The method according to claim 2, characterized in that The determining, based on the abnormal records at or before the first time node, the abnormal cause of the query data at the first time node includes: If the query data at the first time node is empty, and there is no abnormal record at or before the first time node, it is determined that the abnormal cause of the query data at the first time node is data clearing.

6. The method according to claim 2, characterized in that The determining, based on the abnormal records at or before the first time node, the abnormal cause of the query data at the first time node includes: If there is no abnormal record in the data stream at the first time node and before the first time node, and the query data of the first time node is not empty, it is determined that the abnormal cause of the query data of the first time node is a data calculation error.

7. The method according to claim 6, characterized in that The method further comprises: The occurrence time of the data calculation error is determined according to the data flow of the time node before the first time node in the data flow.

8. The method according to claim 7, characterized in that The determining, based on the data stream of the time node before the first time node in the data stream, the time when the data calculation error occurs, includes: Comparing the query data of the time node before the first time node with the source data of the corresponding time node to determine whether the query data of the time node before the first time node is accurate; If the query data at the time nodes before the first time node are all accurate, determining that the time when the data calculation error occurred is between the first time node and the time node before the first time node; If, before the first time node, only the query data of the second time node is inaccurate, it is determined that the time when the data calculation error occurs is between the second time node and a time node before the second time node.

9. The method according to claim 7, characterized in that The data stream further includes data calculation results at each time node, wherein the data calculation results include the cumulative number of businesses at each time node determined based on the source data; The determining, based on the data stream of the time node before the first time node in the data stream, the time when the data calculation error occurs, includes: Comparing the query data of the time node before the first time node with the data calculation results of the corresponding time node to determine whether the data calculation results of the time node before the first time node are accurate; If, before the first time node, only the data calculation result of the third time node is inaccurate, it is determined that the occurrence time of the data calculation error is between the third time node and a time node before the third time node.

10. The method according to any one of claims 1 to 9, characterized in that: The obtaining of the data stream includes: Based on key information carried in the abnormal query request, a data flow associated with the key information is determined, wherein the key information includes one or more of a user identity identifier and an IP address.

11. The method according to claim 10, characterized in that The data flow is determined by: For source data corresponding to the key information, at each time node formed at a preset time interval, calculating a data calculation result at each time node based on the source data; Query the abnormal records of the key information at each time point; Query the query data of the key information at each time node; The key information, the source data, the data calculation results, the abnormal records and the query data are stored in chronological order to form a data stream corresponding to the key information.

12. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method according to any one of claims 1 to 11.

14. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images