Abnormality tracing method, electronic equipment and computer readable storage medium

Abstract

The invention provides an exception tracing method, electronic equipment and a computer readable storage medium, and the method can comprise the steps: obtaining a data stream which comprises query data based on a time record, an exception record based on the time record, and source data based on the time record; judging whether the query data at each time node is matched with the source data on the corresponding time node or not; and if the query data of the first time node is not matched with the source data on the first time node, determining an exception reason according to the exception record distribution in the data stream.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an exception tracing method, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the increase of network users, the amount of business generated by network users is also increasing. In this kind of scenario, when the user queries the business data, there may be a difference between the query result and the user's expectation. In this case, the user needs to manually calculate offline to verify the reason for the difference. However, this implementation method requires more time as the amount of business data increases. SUMMARY

[0003] The purpose of the present application is to provide an exception tracing method, an electronic device and a computer readable storage medium, which can more efficiently determine the cause of the exception.

[0004] In a first aspect, the present application provides an exception tracing method, comprising: obtaining a data stream, the data stream comprising query data recorded based on time, exception records recorded based on time and source data recorded based on time; determining whether the query data at each time node matches the source data at the corresponding time node; if the query data at a first time node does not match the source data at the first time node, determining the cause of the exception according to the distribution of the exception records in the data stream.

[0005] Through the above implementation method, the comparison with the source data can determine whether the query data is abnormal. In the case of exception, the determination of the cause of the exception can be realized by pre-recording the exception. This implementation method can not require the user to manually calculate the massive calculation data, thereby improving the efficiency of exception determination. Furthermore, the exception can be located more quickly based on the record of the exception.

[0006] In an optional implementation, the determining the cause of the exception according to the distribution of the exception records in the data stream comprises: determining whether there are exception records before and at the first time node in the data stream; if there are exception records at or before the first time node, determining the cause of the exception according to the exception records.

[0007] In an optional implementation, the query data comprises cumulative quantity; the exception records comprise update exception records; the determining the cause of the exception according to the exception records comprises: if the cumulative quantity at the first time node is less than the cumulative quantity of the source data at the first time node, and there are update exception records at or before the first time node in the data stream, determining that the cause of the exception is data update exception.

[0008] In an optional embodiment, the exception records include query exception records and update exception records; determining the cause of the exception based on the exception records includes: if the query data of the first time node is empty, and the data stream has a query exception record at the first time node or before it, then determining that the cause of the exception is a data update exception; if the query data of the first time node is empty, and the data stream has no query exception record at the first time node or before it, but has an update exception record, then determining that the cause of the exception is a data update exception and data clearing.

[0009] In the above implementation, the cause of the abnormality can be located more quickly based on the previously recorded abnormality records.

[0010] In an optional embodiment, determining the cause of the exception based on the distribution of abnormal records in the data stream includes: if there is no abnormal record in the data stream at or before the first time node, and the query data of the first time node is empty, determining that the cause of the exception is data clearing.

[0011] In an optional embodiment, determining the cause of the exception based on the distribution of abnormal records in the data stream includes: if there is no abnormal record in the data stream at or before the first time node, and the query data of the first time node is not empty, determining that the cause of the exception is a data calculation error.

[0012] In the above implementation, even if there is no exception record, the cause of the exception can be determined based on the specific value of the query data, which makes it convenient for users to locate and correct the exception.

[0013] In an optional implementation, the method further includes: determining the time when the data calculation error occurs based on the data stream of the time node before the first time node in the data stream.

[0014] In the above implementation, the time when the data calculation error occurs can also be determined through the data stream, which can help users locate the error more conveniently and modify the erroneous data.

[0015] In an optional implementation, the determining of the time of occurrence of the data calculation error according to the data stream of the time node before the first time node includes: determining whether the query data of the time node before the first time node in the data stream matches the source data of the corresponding time node; if the query data of the time node before the first time node in the data stream all match the source data of the corresponding time node, determining that the time of occurrence of the data calculation error is between the first time node and the time node before the first time node; and if the query data of the second time node before the first time node in the data stream does not match the source data of the second time node, determining that the time of occurrence of the data calculation error is between the second time node and the time node before the second time node.

[0016] In an optional implementation, the data stream further includes time-recorded data calculation results, and the data calculation results include the business cumulative quantity of different time points determined based on the source data; and the determining of the time of occurrence of the data calculation error according to the data stream of the time node before the first time node includes: identifying whether the data calculation results of the first time node and the time node before the first time node match the source data of the corresponding time node; and if the data calculation result of the third time node before the first time node in the data stream does not match the source data of the third time node, determining the time of occurrence of the data calculation error according to the third time node.

[0017] In an optional implementation, the obtaining of the data stream includes: determining the data stream associated with the key information based on the key information carried in the abnormal query request, wherein the key information includes one or more of a user identity and an IP address.

[0018] In the above implementation, the query of the abnormal reason of the data of the local user can be implemented based on the key information such as the user identity and the IP address.

[0019] In an optional implementation, the data stream is determined by: calculating the data calculation results of multiple time nodes based on the source data at each time node formed in a preset time interval for the source data corresponding to the key information; obtaining the abnormal record of the key information and the time information of the abnormal record; and storing the key information, the source data, the data calculation result and the abnormal record in chronological order to form the data stream corresponding to the key information.

[0020] In the above implementation, various types of data can be arranged in chronological order in advance, so that the abnormal reason can be quickly located when it is necessary to locate the abnormal reason.

[0021] In a second aspect, the present application provides an anomaly tracing device, comprising: an acquisition module configured to acquire a data stream, the data stream comprising query data recorded based on time, anomaly records recorded based on time, and source data recorded based on time; a judgment module configured to judge whether the query data at each time node matches the source data at the corresponding time node; and a determination module configured to, if the query data at a first time node does not match the source data at the first time node, determine an anomaly cause according to the distribution of the anomaly records in the data stream.

[0022] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory storing machine readable instructions executable by the processor, when the electronic device is running, the machine readable instructions being executed by the processor to perform the steps of the method according to any one of the preceding embodiments.

[0023] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to perform the steps of the method according to any one of the preceding embodiments.

[0024] In a fifth aspect, the present application provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the method according to any one of the preceding embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0026] Figure 1 A schematic diagram of the server interacting with the user terminal provided by the embodiments of the present application;

[0027] Figure 2 A block schematic diagram of the electronic device provided by the embodiments of the present application;

[0028] Figure 3 A flowchart of the anomaly tracing method provided by the embodiments of the present application;

[0029] Figure 4 A partial flowchart of the anomaly tracing method provided by the embodiments of the present application;

[0030] Figure 5A diagram of a visual data flow in an example provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of data flow provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the functional modules of the abnormality tracing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0034] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0035] In risk management, transactions, inquiries, and other operations can be conducted online. However, as data volumes increase, determining the cause of abnormal query results becomes increasingly difficult. The current approach involves offline measurement of business data and collecting and comparing logs generated during the data generation process to identify the cause of the anomaly. This process is time-consuming and labor-intensive, resulting in a prolonged impact on online business operations.

[0036] Based on the above research, the embodiments of the present application can provide an anomaly tracing method, electronic device and computer-readable storage medium, which can achieve relatively rapid determination of anomalies and their causes through comparison of data streams.

[0037] To facilitate understanding of this embodiment, the operating environment of an exception tracing method disclosed in the embodiment of this application is first introduced in detail.

[0038] like Figure 1 FIG2 is a schematic diagram illustrating the interaction between a server 110 and a user terminal 120 according to an embodiment of the present application. The server 110 communicates with one or more user terminals 120 via a network for data communication or interaction. The server 110 may be a network server 110, a database server 110, or the like. The user terminal 120 may be a personal computer (PC), a tablet computer, a smartphone, a personal digital assistant (PDA), or the like.

[0039] The user terminal 120 can provide a query interface through which an abnormality query request is sent to the server 110 to realize abnormality tracing. The server 110 can be configured to execute the steps in the abnormality tracing method to determine the abnormality cause after receiving the abnormality query request.

[0040] Optionally, the server 110 can also be configured to execute the steps in the abnormality tracing method to determine the abnormality cause according to a set time rule. For example, the set time rule can be every specified time length. The specified time length can be one week, one day, twelve hours, one hour, etc.

[0041] As shown in FIG. 2 is a block schematic diagram of an electronic device. The electronic device 200 can include a memory 211 and a processor 213. Those skilled in the art can understand that the structure shown in FIG. 2 is merely illustrative, and does not limit the structure of the electronic device 200. For example, the electronic device 200 can include more or fewer components than those shown in FIG. 2, or have a different configuration from that shown in FIG. 2. For example, the electronic device 200 can include a display unit, a positioning system, etc. Figure 2 Figure 2 As shown in FIG. 2 is a block schematic diagram of an electronic device. The electronic device 200 can include a memory 211 and a processor 213. Those skilled in the art can understand that the structure shown in FIG. 2 is merely illustrative, and does not limit the structure of the electronic device 200. For example, the electronic device 200 can include more or fewer components than those shown in FIG. 2, or have a different configuration from that shown in FIG. 2. For example, the electronic device 200 can include a display unit, a positioning system, etc. Figure 2 Figure 2 As shown in FIG. 2 is a block schematic diagram of an electronic device. The electronic device 200 can include a memory 211 and a processor 213. Those skilled in the art can understand that the structure shown in FIG. 2 is merely illustrative, and does not limit the structure of the electronic device 200. For example, the electronic device 200 can include more or fewer components than those shown in FIG. 2, or have a different configuration from that shown in FIG. 2. For example, the electronic device 200 can include a display unit, a positioning system, etc. Figure 1 Figure 2 As shown in FIG. 2 is a block schematic diagram of an electronic device. The electronic device 200 can include a memory 211 and a processor 213. Those skilled in the art can understand that the structure shown in FIG. 2 is merely illustrative, and does not limit the structure of the electronic device 200. For example, the electronic device 200 can include more or fewer components than those shown in FIG. 2, or have a different configuration from that shown in FIG. 2. For example, the electronic device 200 can include a display unit, a positioning system, etc. Figure 2

[0042] The memory 211 and the processor 213 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the elements can be electrically connected to each other through one or more communication buses or signal lines. The processor 213 is configured to execute the executable modules stored in the memory.

[0043] ​​​​The memory 211 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 211 is used to store programs, and the processor 213 executes the programs after receiving an execution instruction. The method executed by the electronic device 200 defined by the process disclosed in any embodiment of the present application may be applied to the processor 213 or implemented by the processor 213.

[0044] The processor 213 may be an integrated circuit chip with signal processing capabilities. The processor 213 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), a dedicated integrated circuit.

[0045] An application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0046] The electronic device 200 in this embodiment can be used to execute each step in each method provided in the embodiments of the present application. The following describes in detail the implementation process of the abnormality tracing method through several embodiments.

[0047] See also Figure 3 , is a flow chart of the abnormality tracing method provided by the embodiment of the present application. The abnormality tracing method provided by the embodiment of the present application can be applied to an electronic device, and the steps in the abnormality tracing method are executed by the electronic device. Figure 3 The specific process shown is described in detail.

[0048] Step 310: Acquire data stream.

[0049] The data stream includes query data based on time records, exception records based on time records, and source data based on time records.

[0050] Optionally, the data stream may include data from multiple time nodes, and each time node may store corresponding query data and source data.

[0051] Optionally, each query data item may be stored in association with the query time of the query data to form query data based on time records. The source data may be stored according to a plurality of preset time stages to form source data based on time records.

[0052] The exception record is associated with the time node at which it occurs, forming an exception record based on the time record.

[0053] In this embodiment, the source data may represent original business data. For example, the method of the present application may be used in a trading website to determine the cause of abnormal query data. The original business data may represent order data generated by the trading website.

[0054] In this embodiment, the query data may be data generated by a query request initiated based on a need to view source data. For example, the query data may be the cumulative number of orders, cumulative order amount, cumulative sales volume of goods, cumulative sales volume of specified goods, etc.

[0055] In this embodiment, the exception record may be an exception that occurs during the process of maintaining source data.

[0056] For example, an exception record may be an exception generated when storing source data, such as a source data storage failure, a source data storage location access failure, a source data storage request timeout, etc. An exception record may be an exception generated during the computation of the source data, such as a source data access failure required for the computation, etc. An exception record may also be an exception generated when storing a result computed from the source data, such as a computation result storage failure, a computation result storage location access failure, a computation result storage request timeout, etc.

[0057] Optionally, the condition for triggering anomaly tracing may be an actively initiated anomaly query request, and anomaly tracing is initiated based on the anomaly query request. Exemplarily, the anomaly query request may be submitted through an interactive interface of an electronic device that executes the method, or may be submitted by a user terminal that is in communication with the electronic device.

[0058] Optionally, the abnormal query request may carry key information. The key information may be information such as a user identity, IP address, etc. The user identity may be an identifier such as a user account, user mobile phone number, user ID number, etc.

[0059] Based on the key information carried in the abnormal query request, the data stream associated with the key information is determined. Subsequent steps 320 and 330 can determine whether the query data of the key information is abnormal based on the data stream associated with the key information.

[0060] Optionally, the triggering condition for exception tracing can be when a query anomaly is detected. Optionally, multiple tracing time points can be pre-set, and exception tracing can be initiated at each tracing time point. The tracing time points can be set as needed, for example, three, five, or other tracing time points can be set per day.

[0061] Step 320: Determine whether the query data at each time node matches the source data at its corresponding time node.

[0062] If the query data at the first time node does not match the source data at the first time node, it indicates that there is an anomaly in the query data at the current first time node, and step 330 may be executed to determine the cause of the anomaly.

[0063] If the query data at each time node matches the source data at that time node, it means that the query data at each time node is normal.

[0064] Based on the difference in the query data, the determination method of step 320 may also be different.

[0065] If the query data is numerical data, it can be determined whether the query data of the time node is equal to the corresponding value of the source data at the time node. If they are equal, it can be determined that the query data of the time node matches the source data at the time node.

[0066] For example, the query data may include the cumulative number of orders generated from the start time to a specified time point. The cumulative number of orders obtained from the query data at each time point can be checked to see if it is the same as the cumulative number of orders in the source data at that time point. If they are the same, it indicates that the query data matches the source data. The start time can be set as needed, for example, 00:00 every day; another example, 08:00 every day; another example, the first day of each month, etc.

[0067] Exemplarily, the query data can include the accumulated consumption amount generated from the start time to the specified time node, and whether the accumulated consumption amount obtained through the query data of each time node is the same as the accumulated consumption amount of the source data of the time node can be determined. If they are the same, it can be indicated that the query data matches the source data.

[0068] If the query data is the original business data, whether the number of original business data contained in the query data of the time node is the same as the number of original business data in the source data of the time node can be determined. If the number is the same, it can be determined that the query data of the time node matches the source data of the time node.

[0069] In step 330, the abnormal reason is determined according to the distribution of the abnormal records in the data stream.

[0070] If there is an abnormal record, it can be determined whether the abnormal record recorded based on the abnormal record causes the query data to be abnormal.

[0071] If there is no abnormal record, it can be determined from the source database of the query data whether there is an error to determine the abnormal reason. Exemplarily, the abnormal reason of the query data can be determined by identifying the source database. For example, it can be determined whether there is data in the source database of the query data, or whether the data in the source database of the query data is incorrect, etc.

[0072] Optionally, after the abnormal reason is determined, the abnormal reason can be sent to the user terminal sending the abnormal query request to display the abnormal reason in the display interface of the user terminal.

[0073] In the above manner, whether an abnormality occurs can be determined by comparing the source data and the abnormal record. Compared with the manual calculation method to locate the abnormal reason, the time required to find the abnormal reason can be reduced. Through the rapid determination of the abnormal reason, the query abnormality can be solved faster and the possible query abnormality can be prevented, and the service delay caused by the query abnormality in the application scenario of the abnormality tracing method can be reduced.

[0074] In an embodiment, the step 330 described above can include determining whether there is an abnormal record before the first time node in the data stream. If there is an abnormal record before the first time node, the abnormal reason is determined according to the abnormal record.

[0075] In this embodiment, it can be determined whether the exception represented by the exception record will cause an exception of the query data. For example, if the exception record indicates that part of the data storage fails, and the part of the data will be output as the query data to respond to the query request, it can be determined that the exception record will cause an exception of the query data. For example, if the exception record indicates that the source data storage fails, and the source data will not be output as the query data to respond to the query request, it can be determined that the exception record will not cause an exception of the query data.

[0076] If the exception represented by the exception record will cause an exception of the query data, it can be determined that the exception represented by the exception record is the cause of the exception of the query data.

[0077] Optionally, the query data can include a cumulative quantity, and the exception record includes an update exception record.

[0078] Taking the application scenario of the exception tracing method as a transaction system, the cumulative quantity can be a cumulative number of transaction orders generated before a time node at which the query data is triggered.

[0079] For example, the data update exception can represent an exception caused when the latest data is stored in the source database of the query data.

[0080] The above determining the exception cause according to the exception record can include: if the cumulative quantity of the first time node is less than the cumulative quantity of the source data at the first time node, and there is an update exception record before or at the first time node, determining that the exception cause is a data update exception.

[0081] If the cumulative quantity of the first time node is less than the cumulative quantity of the source data at the first time node, it can be indicated that the query data obtained at the first time node does not completely contain the transaction orders accumulated before the first time node, which can indicate that there is a transaction order that has not been counted before the query data is obtained, or there is an error in counting the transaction orders. On this basis, if there is an update exception record before or at the first time node, it can be indicated that the data in the source database for querying does not completely contain the transaction orders accumulated before the first time node, which leads to that the query data obtained at the first time node does not completely contain the transaction orders accumulated before the first time node.

[0082] Optionally, the exception record can include a query exception record and an update exception record.

[0083] The above determining the exception cause according to the exception record can include: if the query data of the first time node is empty, and there is a query exception record before or at the first time node, determining that the exception cause is a data update exception.

[0084] The determining of the abnormal reason according to the abnormal record can include: if the query data of the first time node is empty, and there is no query abnormal record before the first time node, and there is an update abnormal record, then determining that the abnormal reason is data update abnormality and data emptying.

[0085] In the embodiment, if the query data of the first time node is empty, it means that the data cannot be queried at the first time node, and the reason can be that there is no data in the source database. The reason that there is no data in the source database can be that the source database is updated incorrectly, so that there is no data, or the source database is not updated incorrectly, but the data is expired due to long storage time, so that the data is emptied.

[0086] Further, if the query data before the first time node is normal, and the query data of the first time node is abnormal, it can mean that the latest data is not updated to the source database in time, and the historical data is emptied, so that the data cannot be queried at the first time node.

[0087] In another embodiment, the step 330 can include: if there is no abnormal record in the data stream before the first time node, and the query data of the first time node is empty, determining that the abnormal reason is data emptying.

[0088] Further, if the query data before the first time node is normal, it can mean that the data of the source database is normal before the first time node, and the data cannot be queried at the first time node, which can mean that the historical data is emptied, so that the data cannot be queried at the first time node.

[0089] In another embodiment, the step 330 can include: if there is no abnormal record in the data stream before the first time node, and the query data of the first time node is not empty, determining that the abnormal reason is data calculation error.

[0090] If there is no abnormal record in the data stream before the first time node, it means that there is no abnormality in the data management before the first time node, but the query data of the first time node is abnormal, which can mean that there is an error in the calculation of the data in the source database.

[0091] In order to better correct the abnormality of the query data and reduce the probability of subsequent query abnormality, the abnormality tracing method of the embodiment of the application can further include: determining the occurrence time of the data calculation error according to the data stream of the time node before the first time node in the data stream.

[0092] Optionally, the time of occurrence of the data calculation error can be determined by determining whether the query data of the time node before the first time node in the data stream matches the source data of the corresponding time node.

[0093] The manner of determining whether the query data of the time node before the first time node in the data stream matches the source data of the corresponding time node is the same as the manner of determining whether the query data of each time node in the step 320 described above matches the source data of the corresponding time node, and thus will not be described herein again.

[0094] Exemplarily, the query data before the first time node can be compared with the source data to determine the time node at which the anomaly occurs, and then the time of occurrence of the data calculation error can be determined by querying the time node at which the anomaly occurs.

[0095] If the query data of the time node before the first time node in the data stream all match the source data of the corresponding time node, it is determined that the time of occurrence of the data calculation error is between the first time node and the time node before the first time node.

[0096] If the query data of the second time node before the first time node in the data stream does not match the source data of the second time node, it is determined that the time of occurrence of the data calculation error is between the second time node and the time node before the second time node.

[0097] In more scenarios, there can be multiple query data before the first time node that cannot match the source data. In this case, the time of occurrence of the data calculation error can be determined based on multiple time nodes.

[0098] Exemplarily, if the query data of multiple time nodes before the first time node is abnormal continuously, it can be determined that the earliest time node of the multiple continuous time nodes is the time of occurrence of the data calculation error, or the time of occurrence of the data calculation error is between the earliest time node of the multiple continuous time nodes and the time node before the earliest time node.

[0099] Exemplarily, if the query data of multiple time nodes before the first time node is abnormal not completely continuously, it can be indicated that there is a data calculation error before, but the error business data calculation is corrected. In this case, the earliest time node of the continuous time nodes formed with the first time node can be determined as the time of occurrence of the data calculation error, or the time of occurrence of the data calculation error is between the earliest time node of the multiple continuous time nodes and the time node before the earliest time node; if the query data of the time node adjacent to the first time node is not abnormal, it can be determined that the time of occurrence of the data calculation error is between the first time node and the time node before the first time node.

[0100] In this embodiment, the data stream may further include a data calculation result of the time record, and the data calculation result includes the cumulative number of services at different time points determined based on the source data.

[0101] Optionally, the occurrence time of the data calculation error may be determined by identifying whether the data calculation results of the first time node and the time nodes before the first time node match the source data of the corresponding time nodes.

[0102] If, in the data stream, the data calculation result of a third time node before the first time node does not match the source data of the third time node, the time when the data calculation error occurs is determined based on the third time node.

[0103] If the cumulative number of businesses obtained by the data calculation result of the third time node before the first time node is not equal to the cumulative number of businesses contained in the source data of the third time node, it may indicate that the data calculation result of the third time node before the first time node does not match the source data of the third time node.

[0104] In this embodiment, before executing steps 310 to 330, data streams may be prepared in advance. Figure 4 As shown, the data flow is determined by the following steps 410 to 430.

[0105] Step 410 , for source data corresponding to the key information, at each time node formed at a preset time interval, calculate data calculation results of multiple time nodes based on the source data.

[0106] For example, the data calculation results may include the cumulative order amount, cumulative order quantity, cumulative order amount, cumulative product sales, cumulative sales of a specified product, etc. The data type may also vary depending on the data calculation result. For example, the data type may include integer, floating point, string, etc.

[0107] Optionally, after obtaining the data calculation results, they can be stored in the source database. The aforementioned query data can also be obtained by querying the source database.

[0108] Optionally, the data calculation result can be achieved through online calculation. The source data can be calculated every first set time period to obtain the data calculation result. For example, the first set time period can be 1 second, 5 seconds, 10 seconds, etc.

[0109] Optionally, the data calculation result can be achieved through offline calculation, and the source data can be calculated every second set time period to obtain the data calculation result. For example, the second set time period can be 1 hour, 5 hours, 1 day, etc.

[0110] Step 420: Obtain abnormal records of key information and time information of the abnormal records.

[0111] For example, the exception record may be an exception that occurs during the process of calculating the data calculation result, or an exception that occurs when the data calculation result is stored.

[0112] The exception record can be used to indicate data calculation anomalies, data storage anomalies, and data query anomalies. Data calculation anomalies may include missed orders, duplicate orders, and order calculation errors. Data storage anomalies may include storage failures, storage location access errors, and storage request response timeouts. Data query anomalies may include empty query results, query request failures, and query link failures.

[0113] In step 430 , the key information, source data, data calculation results, and exception records are stored in chronological order to form a data stream corresponding to the key information.

[0114] Exemplarily, the above-mentioned source data, data calculation results, and exception records may be arranged in time sequence to obtain a data stream.

[0115] Optionally, the data stream corresponding to each key information is stored separately to construct a data stream for each key information.

[0116] In order to prevent the amount of data from becoming larger and larger, a window size can be set for the data stream. For example, the window size of the data stream can be limited by time.

[0117] Optionally, the window size can be set as needed. The window size can be determined based on the time period typically targeted by query requests. For example, if a user's query requirements are typically for data for the same day, the window size can be equal to or greater than a day, such as 24 hours, 25 hours, or 42 hours. For example, if a user's query requirements are typically for data within a week, the window size can be equal to or greater than a week, such as one week, eight days, or ten days.

[0118] In order to enrich the content of the data stream and facilitate faster and more convenient location of anomalies, the data stream may also include query data.

[0119] Through this data stream, you can quickly locate the data calculation results, source data, query data, and exception records at each time based on time. By locating this data, you can determine the cause of the exception and locate the time when the exception occurred.

[0120] like Figure 5 As shown,Figure 5 A visual data flow diagram in one example is shown. In Figure 5 In the example shown, it contains a time axis, source data at each time node on the time axis, data calculation result, query data, etc.

[0121] In Figure 5 In the example shown, the time axis contains seven time nodes t0, t1, t2, t3, t4, t5 and t6, and the source data d0, d1, d2, d3, d4, d5 and d6 contained in each time node, and the data calculation result v0, v1, v2, v3, v4, v5 and v6 contained in each time node, and the query data q0, q1, q2, q3, q4, q5 and q6 contained in each time node.

[0122] Among them, in terms of source data, the time node where the source data is located represents the transaction time corresponding to the order data. In terms of data calculation result, the time node where the data calculation result is located represents the output time of the data calculation result. In terms of query data, the time node where the query data is located represents the time of the query request.

[0123] In one example, the source data di can represent the cumulative order amount generated at the time node ti and before the time node ti. Wherein i is any value greater than or equal to zero and less than or equal to 6.

[0124] In one example, the data calculation result vi can represent the cumulative order amount calculated at the time node ti. Wherein i is any value greater than or equal to zero and less than or equal to 6.

[0125] In one example, the query data qi can represent the cumulative order amount queried at the time node ti. Wherein i is any value greater than or equal to zero and less than or equal to 6.

[0126] It can be understood that, Figure 5 t0, t1…, t2…, d1, d2…, v1, v2…, q1, q2… in the example shown are for illustration. In actual use, more time nodes can be included, and each time node can include more data. Each group of data can also include a time axis, for example, the time interval of the time axis of the source data can be shorter, and the time interval of the time axis of the query data can be longer.

[0127] The following will be combined Figure 5 The data flow diagram shown, to illustrate the determination process of the abnormal reason with an example.

[0128] For example, as of time node t3, the user's cumulative order volume for the day is 100. If v4 = 100, it means that the data calculation result obtained at time node t3 is correct; if q4 = 100, it means that the query data obtained by the query request at time node t3 is also correct.

[0129] If q4≠100, it means that the query data obtained by the query request at time node t3 is incorrect.

[0130] For example, if q4=null and there is a query exception record before the t3 time node, the cause of the query exception is determined to be a request query exception; if there is no query exception record before the t3 time node, and there is an update exception record, the cause of the query exception is determined to be an update exception of the new feature result, and the old feature result is automatically cleared due to expiration; if there is no query exception record and no update exception record before the t3 time node, the cause of the query exception is determined to be that the feature expired and was cleared.

[0131] For example, if q4<100, if there is an update exception record before the t3 time node, the cause of the query exception is determined to be an abnormal feature result update; if there is no update exception record before the t3 time node, the cause of the query exception is determined to be an error in the data calculation result.

[0132] For example, if q4>100, it is determined that the cause of the query anomaly is an error in the data calculation result.

[0133] In the case of incorrect data calculation results, the data calculation results v1 to v4 can be sorted according to the time sequence of t0 to t3 to locate the time node of the data calculation result.

[0134] By comparing the calculation result of the calculation result time node with the source data of the time node, the type of calculation result error can be determined. The error type can include missing calculation, repeated calculation, etc.

[0135] Optionally, the user terminal that sends the abnormal query request can display Figure 5 As shown in the timing diagram, after determining the abnormal cause, the abnormal cause can be sent to the user terminal that sends the abnormal query request, so that the abnormal cause and the abnormal cause occurrence time are displayed in the timing diagram of the user terminal.

[0136] like Figure 6 As shown, the following describes the abnormality tracing method provided in the embodiment of the present application in combination with the data flow diagram.

[0137] After obtaining the source data, the feature calculation task is used to perform calculations to obtain data calculation results, which are then stored in a temporary storage location. In the example shown in the figure, the temporary storage location is Kafka.

[0138] After determining that the data calculation result is stored in the temporary storage position, the data bus can call the data, obtain the latest data and the exception record generated in the data calculation and data storage process, and store the obtained data in the latest result database. Data services are provided based on the latest result database, which can include data query, data change, and other servers. The data query data and exception records generated during the data service process can also be stored in the temporary storage position.

[0139] When it is necessary to build a data stream, various data can be collected from various temporary storage positions, and the data stored based on the time sequence can be stored in a database. For example, the collected data can be stored in a source data database, an update exception record database, a calculation result database, a query data database, and a query exception record database.

[0140] In this embodiment, the calculation result database can serve as a source database for query data. When a query request is needed, query data can be obtained from the calculation result database. The obtained query data can be stored in the query data database to record the query situation.

[0141] Through data aggregation, the required data can be obtained from various databases based on key information and window size. Then, the aggregated data is arranged in time sequence to obtain a data stream.

[0142] By comparing the data in the data stream, the determination of the cause of the exception can be realized, and finally the cause of the exception can be traced to realize the positioning of the exception.

[0143] Through the method in the embodiments of the present application, various types of data source data, data calculation results, query data, update exceptions, and query exceptions can be classified and stored. Then, data aggregation can be realized according to the limitation of key information and window size, and a data stream can be constructed based on the aggregated data to serve as analysis data for query data exceptions. Compared with the way of measuring and calculating by business personnel in the prior art to locate the cause of the query exception, the cause of the query exception can be obtained relatively accurately in a shorter time. The cause analysis efficiency of the query exception is high, and the impact on the business is reduced. In addition, by displaying the cause of the exception on the user terminal, the cause of the exception and the time of occurrence can be visualized, and the presentation of the cause of the exception is more convincing.

[0144] Based on the same application concept, the embodiments of the present application also provide an exception tracing device corresponding to the exception tracing method. Since the principle of solving problems in the device in the embodiments of the present application is similar to that of the foregoing exception tracing method embodiments, the implementation of the device in the embodiments can be referred to the description in the foregoing method embodiments, and the repeated parts will not be described herein.

[0145] Please refer to Figure 7 FIG. 5 is a schematic diagram of functional modules of an abnormality tracing device provided in an embodiment of the present application. Each module in the abnormality tracing device in the embodiment is configured to execute each step in the method embodiments described above. The abnormality tracing device includes an acquisition module 510, a judgment module 520, and a determination module 530. The content of each module is as follows: the acquisition module 510 is configured to acquire a data stream, the data stream including query data recorded based on time, abnormality records recorded based on time, and source data recorded based on time; the judgment module 520 is configured to judge whether the query data at each time node matches the source data at the corresponding time node; and the determination module 530 is configured to, if the query data at a first time node does not match the source data at the first time node, determine an abnormality cause according to the distribution of the abnormality records in the data stream.

[0146] Each module included in the abnormality tracing device provided in the embodiment can be configured to execute each step in the abnormality tracing method described above, and therefore, more content of the abnormality tracing device can be referred to the steps implemented by the method described above, which will not be described herein again.

[0147] In addition, an embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is run by a processor to execute the steps of the abnormality tracing method described in the method embodiments.

[0148] The computer program product of the abnormality tracing method provided in the embodiment of the present application includes a computer readable storage medium storing program codes. The instructions included in the program codes can be used to execute the steps of the abnormality tracing method described in the method embodiments. For details, refer to the method embodiments described above, which will not be described herein again.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other means. The apparatus embodiments described above are only illustrative, for example, the flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0150] In addition, each functional module in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0151] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0152] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0153] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An abnormality tracing method, characterized in that: include: Acquire a data stream, wherein the data stream includes query data based on time records, exception records based on time records, and source data based on time records; Determine whether the query data at each time node matches the source data at the corresponding time node; If the query data at the first time node does not match the source data at the first time node, the cause of the anomaly is determined based on the distribution of the anomaly records in the data stream.

2. The method according to claim 1, characterized in that The determining the cause of the abnormality according to the distribution of abnormal records in the data stream includes: Determine whether there are abnormal records at or before the first time node in the data stream; If there is an abnormal record at or before the first time node, the cause of the abnormality is determined based on the abnormal record.

3. The method according to claim 2, characterized in that The query data includes cumulative quantity; the exception record includes update exception record; Determining the cause of the abnormality according to the abnormality record includes: If the cumulative number of the first time node is less than the cumulative number of source data at the first time node, and the data flow has an update exception record at or before the first time node, the cause of the exception is determined to be a data update exception.

4. The method according to claim 2, characterized in that The abnormal records include query abnormal records and update abnormal records; Determining the cause of the abnormality according to the abnormality record includes: If the query data at the first time node is empty, and the data stream has a query exception record at or before the first time node, determining that the cause of the exception is a data update exception; If the query data at the first time node is empty, and the data stream has no query exception record at or before the first time node, but has an update exception record, then the cause of the exception is determined to be data update exception and data clearing.

5. The method according to claim 2, characterized in that The determining the cause of the abnormality according to the distribution of abnormal records in the data stream includes: If there is no abnormal record in the data stream at or before the first time node, and the query data at the first time node is empty, it is determined that the cause of the abnormality is data clearing.

6. The method according to claim 2, characterized in that Determining the cause of the abnormality according to the distribution of abnormal records in the data stream includes: If there is no abnormal record in the data stream at or before the first time node, and the query data at the first time node is not empty, it is determined that the cause of the abnormality is a data calculation error.

7. The method according to claim 6, characterized in that The method further comprises: The occurrence time of the data calculation error is determined according to the data flow of the time node before the first time node in the data flow.

8. The method according to claim 7, characterized in that The determining, based on the data stream of the time node before the first time node in the data stream, the time when the data calculation error occurs, includes: Determining whether query data at a time node before the first time node in the data stream matches source data at its corresponding time node; If the query data of the time nodes before the first time node in the data stream all match the source data of the corresponding time nodes, it is determined that the time when the data calculation error occurred is between the first time node and the time node before the first time node; If in the data stream, the query data of a second time node before the first time node does not match the source data of the second time node, it is determined that the time when the data calculation error occurred is between the second time node and a time node before the second time node.

9. The method according to claim 7, characterized in that The data stream also includes a data calculation result of the time record, and the data calculation result includes the cumulative number of businesses at different time points determined based on the source data; The determining, based on the data stream of the time node before the first time node in the data stream, the time when the data calculation error occurs, includes: Identifying whether data calculation results of the first time node and time nodes before the first time node match source data of the corresponding time nodes; If, in the data stream, the data calculation result of a third time node before the first time node does not match the source data of the third time node, the time when the data calculation error occurs is determined according to the third time node.

10. The method according to any one of claims 1 to 9, characterized in that: The obtaining of the data stream includes: Based on key information carried in the abnormal query request, a data flow associated with the key information is determined, wherein the key information includes one or more of a user identity identifier and an IP address.

11. The method according to claim 10, characterized in that The data flow is determined by: For source data corresponding to the key information, at each time node formed at a preset time interval, calculating data calculation results of multiple time nodes based on the source data; Obtaining abnormal records of the key information and time information of the abnormal records; The key information, the source data, the data calculation results and the exception records are stored in chronological order to form a data stream corresponding to the key information.

12. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method according to any one of claims 1 to 11.

14. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images