Verification method and device of service system

Abstract

The embodiment of the invention provides a verification method and device of a service system, and relates to the technical field of data processing. The method comprises the following steps: acquiring process state data of a target business system; acquiring a data flow of at least one service node of the target service system according to the process state data; verifying the data stream of the at least one service node to obtain to-be-verified data in the data stream of the at least one service node, and adding the to-be-verified data to a to-be-verified data queue; on the basis of the process state data, performing multilateral verification on each piece of to-be-verified data in the to-be-verified data queue to filter noise data in the to-be-verified data queue; and determining whether abnormal data exists in the target business system based on the to-be-verified data queue after the noise data is filtered out. The embodiment of the invention is used for reducing or avoiding noise data in the verification result.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a verification method and device for a business system. Background Art

[0002] Ensuring data consistency on each business node of a business system is a basic requirement of the business system. However, as the throughput of the business system increases, ensuring data consistency on each business node faces great challenges.

[0003] In order to ensure the consistency of data on each business node of the business system, the common practice in related technologies is to extract data on the specified business node and verify the data on the business node according to the business status to determine whether there is abnormal data with inconsistent data in the business system. However, due to reasons such as delayed data arrival and jitter of the verification algorithm, there may be noise data in the abnormal data obtained by verifying the data on the business node, which in turn causes the verification result of the data on the business node to be unable to accurately reflect whether there is abnormal data in the business system. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a verification method and device for a business system, which is used to reduce or avoid noise data in the verification results, so that the verification results can more accurately reflect whether there is abnormal data in the business system.

[0005] In order to achieve the above purpose, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a verification method for a business system, comprising:

[0007] Obtain process state data of the target business system;

[0008] Acquire a data flow of at least one service node of the target service system according to the process state data;

[0009] Verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and adding the data to be verified to a queue of data to be verified;

[0010] Based on the process state data, each of the to-be-verified data in the to-be-verified data queue is subjected to multilateral verification respectively, so as to filter out noise data in the to-be-verified data queue;

[0011] Based on the to-be-verified data queue after filtering out noise data, determine whether there is abnormal data in the target business system.

[0012] As an optional implementation of the embodiment of the present application, the acquiring the data flow of at least one service node of the target service system according to the process state data includes:

[0013] Preprocessing the process state data to obtain a data stream of the at least one service node;

[0014] The preprocessing includes at least one of data filtering and data aggregation.

[0015] As an optional implementation manner of the embodiment of the present application, the data stream of the at least one service node is a data stream of a target service node, and the verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes:

[0016] Determine whether there is data in the data stream of the target service node that does not meet the preset conditions;

[0017] If the first data in the data stream of the target service node does not meet the preset condition, the first data is determined to be the data to be verified in the data stream of the target service node.

[0018] As an optional implementation manner of the embodiment of the present application, the data stream of the at least one service node is a data stream of a target service node, and the verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes:

[0019] Acquire data of an upstream service node of the target service node and / or data of a downstream service node of the target service node;

[0020] Determining whether there is data in the data flow of the service node that is inconsistent with data of the upstream service node and / or data of the downstream service node;

[0021] If the second data in the data stream of the target service node is inconsistent with the data of the upstream service node and / or the data of the downstream service node, the second data is determined to be the data to be verified in the data stream of the target service node.

[0022] As an optional implementation manner of the embodiment of the present application, the data stream of the at least one service node includes: a data stream of a first service node and a data stream of a second service node, and the verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes:

[0023] Associating data in the data flow of the first service node with data in the data flow of the second service node;

[0024] If the third data in the data stream of the first service node cannot be successfully associated with the data in the data stream of the second service node, the third data is determined to be the data to be verified in the data stream of the first service node and the data stream of the second service node.

[0025] As an optional implementation manner of the embodiment of the present application, associating the data in the data flow of the first service node with the data in the data flow of the second service node includes:

[0026] Obtaining a first timestamp of the data to be associated in the data stream of the first service node;

[0027] Determine a second timestamp according to the first timestamp and a preset offset duration;

[0028] Determine a check window according to the second timestamp, an upper window bound, and a lower window bound;

[0029] The data to be associated is associated in the check window of the data flow of the second service node.

[0030] As an optional implementation of the embodiment of the present application, after acquiring the data of the target business system, the method further includes: writing the process state data into a preset storage engine;

[0031] Before performing multilateral verification on each of the data to be verified in the data to be verified queue based on the process state data, the method further includes: reading the process state data from the preset storage engine.

[0032] As an optional implementation of the embodiment of the present application, before performing multilateral verification on each piece of data to be verified in the queue of data to be verified based on the process state data, the method further includes:

[0033] Obtain the query rate per second (QPS) of the queue of data to be verified;

[0034] Determine whether the QPS of the to-be-verified data queue is greater than a preset threshold;

[0035] If so, the QPS in the to-be-verified data queue is limited to the preset threshold.

[0036] As an optional implementation of the embodiment of the present application, before performing multilateral verification on each piece of data to be verified in the queue of data to be verified based on the process state data, the method further includes:

[0037] The data to be verified in the queue of data to be verified is filtered based on preset rules.

[0038] As an optional implementation of the embodiment of the present application, the method further includes:

[0039] In a case where it is determined that first abnormal data exists in the target business system, obtaining a type of the first abnormal data and / or a cause of the first abnormal data according to a multilateral verification result corresponding to the first abnormal data;

[0040] The type of the first abnormal data and / or the cause of the first abnormal data are output.

[0041] As an optional implementation of the embodiment of the present application, the method further includes:

[0042] When it is determined that the target business system has second abnormal data, a compensation strategy corresponding to the second abnormal data is called to compensate for the second abnormal data.

[0043] In a second aspect, an embodiment of the present application provides a verification device for a business system, including:

[0044] An acquisition unit, used to acquire process state data of a target business system;

[0045] A preprocessing unit, configured to obtain a data stream of at least one service node of the target service system according to the process state data;

[0046] A first verification unit, configured to verify the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and add the data to be verified to a queue of data to be verified;

[0047] A second verification unit, configured to perform multilateral verification on each of the data to be verified in the data queue to be verified based on the process state data, so as to filter out noise data in the data queue to be verified;

[0048] A processing unit is used to determine whether there is abnormal data in the target business system based on the queue of data to be verified after filtering out noise data.

[0049] As an optional implementation of the embodiment of the present application, the preprocessing unit is specifically used to preprocess the process state data to obtain the data flow of the at least one service node;

[0050] The preprocessing includes at least one of data filtering and data aggregation.

[0051] As an optional implementation of an embodiment of the present application, the data flow of at least one business node is the data flow of a target business node; the first verification unit is specifically used to determine whether there is data in the data flow of the target business node that does not meet the preset conditions; if the first data in the data flow of the target business node does not meet the preset conditions, it is determined that the first data is the data to be verified in the data flow of the target business node.

[0052] As an optional implementation of an embodiment of the present application, the data flow of at least one business node is the data flow of a target business node, and the first verification unit is specifically used to obtain data of an upstream business node of the target business node and / or data of a downstream business node of the target business node; determine whether there is data in the data flow of the business node that is inconsistent with the data of the upstream business node and / or the data of the downstream business node; if the second data in the data flow of the target business node is inconsistent with the data of the upstream business node and / or the data of the downstream business node, determine that the second data is the data to be verified in the data flow of the target business node.

[0053] As an optional implementation of an embodiment of the present application, the data stream of the at least one business node includes: a data stream of a first business node and a data stream of a second business node; the first verification unit is specifically used to associate data in the data stream of the first business node with data in the data stream of the second business node; if third data in the data stream of the first business node fails to be successfully associated with data in the data stream of the second business node, then the third data is determined to be data to be verified in the data stream of the first business node and the data stream of the second business node.

[0054] As an optional implementation of the embodiment of the present application, the first verification unit is specifically used to obtain a first timestamp of the data to be associated in the data stream of the first business node; determine a second timestamp based on the first timestamp and a preset offset duration; determine a verification window based on the second timestamp, an upper bound of the window, and a lower bound of the window; and associate the data to be associated in the verification window of the data stream of the second business node.

[0055] As an optional implementation of the embodiment of the present application, the acquisition unit is further used to write the process state data into a preset storage engine after acquiring the data of the target business system;

[0056] The second verification unit is further configured to read the process state data from the preset storage engine before performing multilateral verification on each of the data to be verified in the data to be verified queue based on the process state data.

[0057] As an optional implementation of the embodiment of the present application, the second verification unit is also used to obtain the QPS of the data queue to be verified before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data; determine whether the QPS of the data queue to be verified is greater than a preset threshold; if so, limit the QPS in the data queue to be verified to the preset threshold.

[0058] As an optional implementation of the embodiment of the present application, the second verification unit is also used to filter the data to be verified in the data queue to be verified based on preset rules before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data.

[0059] As an optional implementation of an embodiment of the present application, the processing unit is also used to, when it is determined that there is first abnormal data in the target business system, obtain the type of the first abnormal data and / or the cause of the first abnormal data according to the multilateral verification result corresponding to the first abnormal data, and output the type of the first abnormal data and / or the cause of the first abnormal data.

[0060] As an optional implementation of the embodiment of the present application, the processing unit is further used to call a compensation strategy corresponding to the second abnormal data to compensate for the second abnormal data when it is determined that the second abnormal data exists in the target business system.

[0061] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the verification method of the business system described in any of the above embodiments when executing the computer program.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement the verification method of the business system described in any of the above-mentioned embodiments.

[0063] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer, the computer implements the verification method of the business system described in any of the above-mentioned embodiments.

[0064] The verification method of the business system provided in the embodiment of the present application first obtains the process state data of the target business system, then obtains the data stream of at least one business node of the target business system according to the process state data, and verifies the data stream of the at least one business node to obtain the data to be verified in the data stream of the at least one business node, and adds the data to be verified to the queue of data to be verified, and then performs multilateral verification on each data to be verified in the queue of data to be verified based on the process state data to filter out the noise data in the queue of data to be verified, and determines whether there is abnormal data in the target business system based on the queue of data to be verified after filtering out the noise data. Since the embodiment of the present application verifies the data stream of at least one business node to obtain the data to be verified in the data stream of at least one business node, and adds the data to be verified in the at least one business node to the queue of data to be verified, it will further perform multilateral verification on each data to be verified in the queue of data to be verified based on the process state data of the target business system to filter out the noise data in the queue of data to be verified, so that the verification method of the business system provided in the embodiment of the present application can reduce or avoid the noise data in the queue of data to be verified, so that the verification result can more accurately reflect whether there is abnormal data in the business system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0067] Figure 1 A software architecture diagram of a verification system for a business system provided in an embodiment of the present application;

[0068] Figure 2 A flowchart of a verification method for a business system provided in an embodiment of the present application;

[0069] Figure 3 One of the schematic diagrams for verifying the data provided in the embodiment of the present application;

[0070] Figure 4 The second schematic diagram of verifying the data provided in the embodiment of the present application;

[0071] Figure 5A structural diagram of a verification device for a business system provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0074] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0075] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0076] The following first describes the verification system on which the verification method of the business system provided in the embodiment of the present application relies.

[0077] Reference Figure 1 As shown, the verification system relied on by the verification method of the business system provided in the embodiment of the present application includes: a front-end interaction page 101, a basic library 102, a front-end response service 103, a real-time task management service 104, a hosting platform 105, a queue of data to be verified 106, an exception handling service 107, a secondary verification service 108, an offline task management service 109, an external dependency 110 and other services and functional modules.

[0078] Among them, the front-end interactive page 101 is configured to interact with the user and receive the verification task creation, business node indicator configuration, exception handling and other instructions input by the user. The basic library 102 is configured to save and manage the basic capabilities such as the relational database service (RDS), open application programming interface (API), constants of the business system, so as to provide other functional modules in the verification system with the basic capabilities. The front-end response service 103 is configured to respond to the instructions received by the front-end interactive page 101, for example: to respond to the verification task creation instruction to perform verification task creation, task access, task scheduling, etc. The real-time task management service 104 is configured to respond to the call of the front-end response service 103 to perform real-time task management operations such as starting the real-time verification task, editing the real-time task, and deleting the real-time task. The hosting platform 105 is configured to realize the functions of managing the Flink structured query language (SQL), verifying the data based on Flink SQL, generating a visualization page of the verification results, and writing the verified data to be verified into the data queue 106 to be verified. The queue of data to be verified 106 is configured to receive the data to be verified written by the hosting platform 105, and input the data to be verified to the exception handling service 107. The exception handling service 107 is configured to process the data to be verified in the queue of data to be verified 106, including: limiting the flow of data to be verified, filtering the data to be verified, compensating for abnormal data, and other functions. The secondary verification service 108 is configured to perform secondary verification on the data to be verified in response to the scheduling of the exception handling service 107. The offline task management service 109 is configured to perform real-time task management operations such as starting offline verification tasks, editing offline tasks, and deleting offline tasks in response to the call of the front-end response service 103. The external dependency 110 is configured to save and manage the external dependencies of the verification system, including Remote Dictionary Server (Redis), MySQL (MyStructured Query Language), etc.

[0079] The present application embodiment provides a verification method for a business system, referring to Figure 2 As shown, the verification method of the business system includes the following steps S21 to S25:

[0080] S21. Obtain process state data of the target business system.

[0081] In the embodiment of the present application, process state data refers to the path data generated in the process of implementing the business service. Process state data is relative to final state data (result data), and the final state data includes the data results generated after executing the business operation.

[0082] S22: Acquire a data flow of at least one service node of the target service system according to the process state data.

[0083] In some embodiments, the above step S22 (obtaining a data flow of at least one service node of the target service system according to the process state data) includes:

[0084] The process state data is preprocessed to obtain the data stream of the at least one service node, wherein at least one of data filtering and data aggregation is performed.

[0085] In some embodiments, the at least one service node may be configured by a user.

[0086] S23: Verify the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and add the data to be verified to a queue of data to be verified.

[0087] That is, one or more data streams of business nodes are selected from the data streams of multiple business nodes of the target business system, and the selected data streams are verified to obtain the data to be verified in the one or more business nodes, and the data to be verified is added to the data queue to be verified.

[0088] In some embodiments, the data to be verified in the service node may include: data that does not meet preset conditions, data that is inconsistent with upstream data and / or downstream data, and data that is inconsistent with data of other service nodes.

[0089] S24. Perform multilateral verification on each of the data to be verified in the data queue to be verified based on the process state data, so as to filter out noise data in the data queue to be verified.

[0090] The data to be verified added to the queue of data to be verified is likely not abnormal data caused by data inconsistency in the target business system, but caused by delayed data arrival, data loss, and unstable verification algorithm. Such data to be verified is not really abnormal data and does not need to be compensated. Therefore, such data to be verified needs to be filtered out of the queue of data to be verified as noise data, and multilateral verification can filter out such data to be verified very well. For example: due to delayed data arrival, a certain data cannot be detected in the data flow of a certain node within a specified time period, so it is added to the queue of data to be verified as data to be verified. When the data to be verified is multilaterally verified based on process state data, it is determined that the data of other business nodes in the business link to which the business node belongs are consistent, then it can be determined that the data to be verified is noise data, and then the data to be verified is filtered out of the queue of data to be verified.

[0091] In some embodiments, after the above step S21 (obtaining the data of the target business system), the process state data can be written into a preset storage engine, and before the above step S24, the process state data is first read from the preset storage engine, and then each data to be verified in the data queue to be verified is multilaterally verified based on the process state data.

[0092] S25. Determine whether there is abnormal data in the target business system based on the to-be-verified data queue after filtering out noise data.

[0093] That is, if the data queue to be verified after filtering out the noise data still contains data to be verified, it is determined that there is abnormal data in the target business system; and if the data queue to be verified after filtering out the noise data does not contain data to be verified, it is determined that there is no abnormal data in the target business system.

[0094] The verification method of the business system provided in the embodiment of the present application first obtains the process state data of the target business system, then obtains the data stream of at least one business node of the target business system according to the process state data, and verifies the data stream of the at least one business node to obtain the data to be verified in the data stream of the at least one business node, and adds the data to be verified to the queue of data to be verified, and then performs multilateral verification on each data to be verified in the queue of data to be verified based on the process state data to filter out the noise data in the queue of data to be verified, and determines whether there is abnormal data in the target business system based on the queue of data to be verified after filtering out the noise data. Since the embodiment of the present application verifies the data stream of at least one business node to obtain the data to be verified in the data stream of at least one business node, and adds the data to be verified in the at least one business node to the queue of data to be verified, it will further perform multilateral verification on each data to be verified in the queue of data to be verified based on the process state data of the target business system to filter out the noise data in the queue of data to be verified, so that the verification method of the business system provided in the embodiment of the present application can reduce or avoid the noise data in the queue of data to be verified, so that the verification result can more accurately reflect whether there is abnormal data in the business system.

[0095] In the verification method of the business system provided in the embodiment of the present application, the verification of the data flow of the at least one business node can be carried out by verifying the data flow of a single business node or by verifying the data flow of two business nodes. The following describes the implementation methods of verifying the data flow of a single business node to obtain the data to be verified in the data flow of the single business node and verifying the data flow of two business nodes to obtain the data to be verified in the data flow of two business nodes.

[0096] In some embodiments, the data stream of the at least one service node is the data stream of the target service node (the data stream of a single service node), and the data stream of the at least one service node is verified to obtain the data to be verified in the data stream of the at least one service node. The implementation method includes the following steps I and II:

[0097] Step I: determine whether there is data in the data stream of the target service node that does not meet the preset conditions.

[0098] For the data stream of a single service node, the verification method can be to verify whether the service status of the service node during the service processing is all in a successful state. Therefore, the embodiment of the present application can set the preset condition based on determining whether the service status of the service node during the service processing is all in a successful state, thereby determining whether there is data to be verified in the data stream of the target service node. For example: setting the error code to 0 as the preset condition, thereby determining whether there is data with an error code (err_code) not being 0 in the data stream of the service node, thereby determining whether there is data to be verified in the data stream of the target service node.

[0099] Step II: If the first data in the data stream of the target service node does not meet the preset condition, determine that the first data is the data to be verified in the data stream of the target service node.

[0100] As described in the above embodiment, if a certain error code in the data stream of the target service node is not 0, the error code is determined to be the data to be verified in the data stream of the target service node.

[0101] In some embodiments, the data stream of the at least one service node is the data stream of the target service node (the data stream of a single service node), and the data stream of the at least one service node is verified to obtain the data to be verified in the data stream of the at least one service node. The implementation method includes the following steps 1 to 3:

[0102] Step 1: Acquire data of an upstream service node of the target service node and / or data of a downstream service node of the target service node.

[0103] In some embodiments, data of an upstream business node of the target business node and / or data of a downstream business node of the target business node may be obtained by remote procedure call (PRC), access to a non-relational database (Redis), or the like.

[0104] Step 2: Determine whether the data flow of the service node contains data that is inconsistent with the data of the upstream service node and / or the data of the downstream service node.

[0105] Although it is impossible to verify the consistency of upstream and downstream data based on the data flow of a single business node alone, it is possible to obtain the data of its upstream business node and / or the data of the downstream business node through RPC, accessing Redis, etc., thereby verifying whether the upstream and downstream data of the data flow of a single business node are consistent.

[0106] Step 3: If the second data in the data stream of the target service node is inconsistent with the data of the upstream service node and / or the data of the downstream service node, determine that the second data is the data to be verified in the data stream of the target service node.

[0107] In some embodiments, the data stream of the at least one service node includes: a data stream of a first service node and a data stream of a second service node (data streams of two service nodes); the verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes the following steps a and b:

[0108] Step a: Associating data in the data flow of the first service node with data in the data flow of the second service node.

[0109] In some embodiments, the data stream of the first business node and the data stream of the second business node can be cached in corresponding memories respectively. When new data arrives at the first business node, an attempt is made to associate the data with the memory that caches the data stream of the second business node. When new data arrives at the second business node, an attempt is made to associate the data with the memory that caches the data stream of the first business node.

[0110] Since caching all the data streams of the first service node and the second service node in the memory will occupy a large memory space, in some embodiments, only data within a certain period of time may be cached in the memory.

[0111] In the related art, when associating data in the data streams of two service nodes, data association is generally performed based on a fixed window. Figure 3 As shown, the data flow receiving period of the two service nodes is divided into time windows (31, 32, 33...), and the data of the first service node and the data of the second service node in the same time window are associated. This method of data association based on a fixed window is likely to divide the two data that should be associated into different time windows, thereby increasing the noise data in the queue of data to be verified.

[0112] Taking the above problem as an example, the implementation method of associating the data in the data flow of the first service node and the data in the data flow of the second service node in the embodiment of the present application includes the following steps a1 to a4:

[0113] Step a1: Obtain a first timestamp of the data to be associated in the data stream of the first service node.

[0114] The data to be associated may be any data in the data flow of the first service node.

[0115] Step a2: determine a second timestamp according to the first timestamp and a preset offset duration.

[0116] In some embodiments, determining the second timestamp according to the first timestamp and a preset delay duration includes: summing the first timestamp and the preset offset duration to obtain the second timestamp.

[0117] Step a3: determine a check window according to the second timestamp, an upper window bound, and a lower window bound.

[0118] In some embodiments, the verification window is determined based on the second timestamp, the upper bound of the window, and the lower bound of the window, including: summing the second timestamp and the lower bound of the window to obtain the end time of the verification window, and subtracting the second timestamp from the upper bound of the window to obtain the start time of the verification window.

[0119] Assume: the second timestamp is t, the upper bound of the window is T R , the lower bound of the window is T L , then the start and end times of the verification window are: tT L and t+T R .

[0120] Step a4: Associating the data to be associated in the check window of the data flow of the second service node.

[0121] Reference Figure 4 As shown, Figure 4 The above embodiment is illustrated by taking the data stream of the first business node as including: A, B, C, D, ... N, the data stream of the second business node as including: 1, 2, 3, 4, ... n, and data A should be associated with data 3. If the data stream of the first business node and the data stream of the second business node are matched in a traditional window matching manner, since data A is divided into window 41 and data 3 is divided into window 42, data A and data 3 are not in the same window, so data A and data 3 will be written into the queue of data to be verified as data to be verified. If according to the above embodiment, the first timestamp t1 of data A is first obtained, and then according to the first timestamp t1 and the preset offset duration t offset Determine the second timestamp t2, and then according to the second timestamp t2, the upper limit of the window T R And the window lower bound T L Determine the verification window 43, and finally associate the data A in the verification window 43 of the data flow of the second business node. Then, the data A and the data 3 can be associated in the verification window 43, thereby avoiding writing the data A and the data 3 as data to be verified into the data queue to be verified.

[0122] Step b: If the third data in the data stream of the first service node cannot be successfully associated with the data in the data stream of the second service node, determine that the third data is the data to be verified in the data stream of the first service node and the data stream of the second service node.

[0123] In some embodiments, the verification method of the business system provided in the embodiments of the present application also includes: before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data, obtaining the query rate per second (Queries-per-second, QPS) of the data queue to be verified; determining whether the QPS of the data queue to be verified is greater than a preset threshold; if so, limiting the QPS in the data queue to be verified to the preset threshold.

[0124] The QPS of the data queue to be verified can represent the amount of traffic processed by the data queue to be verified within a specified time. A larger QPS means that the data queue to be verified processes more traffic within a specified time, and a smaller QPS means that the data queue to be verified processes less traffic within a specified time.

[0125] Since the large magnitude of the data to be verified may affect the normal progress of other tasks, the embodiment limits the QPS of the data queue to be verified to the preset threshold when the QPS of the data queue to be verified is greater than the preset threshold, thereby avoiding the normal progress of other tasks affected by the business system verification.

[0126] In some embodiments, the verification method of the business system provided by the embodiment of the present application also includes: before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data, filtering the data to be verified in the data queue to be verified based on preset rules.

[0127] Exemplarily, the preset rules may include preset configured abnormal data that does not need to be compensated, preset configured abnormal data that does not need to be stored, etc.

[0128] In some embodiments, the verification method of the business system provided in the embodiments of the present application also includes: when it is determined that there is first abnormal data in the target business system, obtaining the type of the first abnormal data and / or the cause of the first abnormal data according to the multilateral verification result corresponding to the first abnormal data; and outputting the type of the first abnormal data and / or the cause of the first abnormal data.

[0129] That is, the verification method of the business system provided in the embodiment of the present application can also automatically attribute abnormal data, shorten the time length of abnormal data processing, and reduce the impact of abnormal data.

[0130] In some embodiments, the verification method of the business system provided in the embodiments of the present application further includes: when it is determined that second abnormal data exists in the target business system, calling a compensation strategy corresponding to the second abnormal data to compensate for the second abnormal data.

[0131] That is, the verification method of the business system provided in the embodiment of the present application can also automatically compensate for abnormal data to further reduce the impact of the abnormal data.

[0132] Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application also provides a verification device for a business system. This embodiment corresponds to the above method embodiment. For ease of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the verification device for the business system in this embodiment can correspond to all the contents of the above method embodiment.

[0133] The present application provides a verification device for a business system. Figure 5 It is a schematic diagram of the structure of the verification device of the business system, such as Figure 5 As shown, the verification device 500 of the business system includes:

[0134] An acquisition unit 51 is used to acquire process state data of a target business system;

[0135] A preprocessing unit 52, configured to obtain a data stream of at least one service node of the target service system according to the process state data;

[0136] A first verification unit 53, configured to verify the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and add the data to be verified to a queue of data to be verified;

[0137] A second verification unit 54 is used to perform multilateral verification on each of the to-be-verified data in the to-be-verified data queue based on the process state data, so as to filter out noise data in the to-be-verified data queue;

[0138] The processing unit 55 is used to determine whether there is abnormal data in the target business system based on the to-be-verified data queue after filtering out noise data.

[0139] As an optional implementation of the embodiment of the present application, the preprocessing unit 52 is specifically used to preprocess the process state data to obtain the data flow of the at least one service node;

[0140] The preprocessing includes at least one of data filtering and data aggregation.

[0141] As an optional implementation of the embodiment of the present application, the data flow of at least one business node is the data flow of the target business node; the first verification unit 53 is specifically used to determine whether there is data in the data flow of the target business node that does not meet the preset conditions; if the first data in the data flow of the target business node does not meet the preset conditions, it is determined that the first data is the data to be verified in the data flow of the target business node.

[0142] As an optional implementation of an embodiment of the present application, the data flow of at least one business node is the data flow of a target business node, and the first verification unit 53 is specifically used to obtain data of an upstream business node of the target business node and / or data of a downstream business node of the target business node; determine whether there is data in the data flow of the business node that is inconsistent with the data of the upstream business node and / or the data of the downstream business node; if the second data in the data flow of the target business node is inconsistent with the data of the upstream business node and / or the data of the downstream business node, determine that the second data is the data to be verified in the data flow of the target business node.

[0143] As an optional implementation of an embodiment of the present application, the data flow of the at least one business node includes: the data flow of a first business node and the data flow of a second business node; the first verification unit 53 is specifically used to associate the data in the data flow of the first business node with the data in the data flow of the second business node; if the third data in the data flow of the first business node fails to be successfully associated with the data in the data flow of the second business node, then the third data is determined to be the data to be verified in the data flow of the first business node and the data flow of the second business node.

[0144] As an optional implementation of the embodiment of the present application, the first verification unit 53 is specifically used to obtain a first timestamp of the data to be associated in the data stream of the first business node; determine a second timestamp based on the first timestamp and a preset offset duration; determine a verification window based on the second timestamp, an upper bound of the window, and a lower bound of the window; and associate the data to be associated in the verification window of the data stream of the second business node.

[0145] As an optional implementation of the embodiment of the present application, the acquisition unit 51 is further configured to write the process state data into a preset storage engine after acquiring the data of the target business system;

[0146] The second verification unit is further configured to read the process state data from the preset storage engine before performing multilateral verification on each of the data to be verified in the data to be verified queue based on the process state data.

[0147] As an optional implementation of the embodiment of the present application, the second verification unit 54 is also used to obtain the QPS of the data queue to be verified before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data; determine whether the QPS of the data queue to be verified is greater than a preset threshold; if so, limit the QPS in the data queue to be verified to the preset threshold.

[0148] As an optional implementation of the embodiment of the present application, the second verification unit 54 is also used to filter the data to be verified in the data queue to be verified based on preset rules before performing multilateral verification on each data to be verified in the data queue to be verified based on the process state data.

[0149] As an optional implementation of an embodiment of the present application, the processing unit 55 is also used to, when it is determined that there is first abnormal data in the target business system, obtain the type of the first abnormal data and / or the cause of the first abnormal data according to the multilateral verification result corresponding to the first abnormal data, and output the type of the first abnormal data and / or the cause of the first abnormal data.

[0150] As an optional implementation of the embodiment of the present application, the processing unit 55 is further used to call a compensation strategy corresponding to the second abnormal data to compensate for the second abnormal data when it is determined that the second abnormal data exists in the target business system.

[0151] The verification device of the business system provided in this embodiment can execute the verification method of the business system provided in the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0152] Based on the same inventive concept, an embodiment of the present application also provides an electronic device. Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device provided in this embodiment includes: a memory 61 and a processor 62, wherein the memory 61 is used to store a computer program, and the processor 62 is used to execute the verification method of the business system provided in the above embodiment when executing the computer program.

[0153] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device implements the verification method of the business system provided in the above embodiment.

[0154] Based on the same inventive concept, an embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computing device implements the verification method of the business system provided in the above embodiment.

[0155] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0156] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware model components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0157] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0158] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A verification method for a business system, characterized in that: include: Obtain process state data of the target business system; Acquire a data flow of at least one service node of the target service system according to the process state data; Verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and adding the data to be verified to a queue of data to be verified; Based on the process state data, each of the to-be-verified data in the to-be-verified data queue is subjected to multilateral verification respectively, so as to filter out noise data in the to-be-verified data queue; Based on the to-be-verified data queue after filtering out noise data, determine whether there is abnormal data in the target business system.

2. The method according to claim 1, characterized in that The step of acquiring the data flow of at least one service node of the target service system according to the process state data comprises: Preprocessing the process state data to obtain a data stream of the at least one service node; The preprocessing includes at least one of data filtering and data aggregation.

3. The method according to claim 1, characterized in that The data stream of the at least one service node is a data stream of a target service node, and the verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes: Determine whether there is data in the data stream of the target service node that does not meet the preset conditions; If the first data in the data stream of the target service node does not meet the preset condition, the first data is determined to be the data to be verified in the data stream of the target service node.

4. The method according to claim 1, characterized in that: The data stream of the at least one service node is a data stream of a target service node; and verifying the data stream of the at least one service node to obtain data to be verified in the data stream of the at least one service node includes: Acquire data of an upstream service node of the target service node and / or data of a downstream service node of the target service node; Determining whether there is data in the data flow of the service node that is inconsistent with data of the upstream service node and / or data of the downstream service node; If the second data in the data stream of the target service node is inconsistent with the data of the upstream service node and / or the data of the downstream service node, the second data is determined to be the data to be verified in the data stream of the target service node.

5. The method according to claim 1, characterized in that The data stream of the at least one service node includes: a data stream of a first service node and a data stream of a second service node; and verifying the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node includes: Associating data in the data flow of the first service node with data in the data flow of the second service node; If the third data in the data stream of the first service node cannot be successfully associated with the data in the data stream of the second service node, the third data is determined to be the data to be verified in the data stream of the first service node and the data stream of the second service node.

6. The method according to claim 5, characterized in that The associating the data in the data flow of the first service node with the data in the data flow of the second service node includes: Obtaining a first timestamp of the data to be associated in the data stream of the first service node; Determine a second timestamp according to the first timestamp and a preset offset duration; Determine a check window according to the second timestamp, an upper window bound, and a lower window bound; The data to be associated is associated in the check window of the data flow of the second service node.

7. The method according to claim 1, characterized in that After acquiring the data of the target business system, the method further includes: writing the process state data into a preset storage engine; Before performing multilateral verification on each of the data to be verified in the data to be verified queue based on the process state data, the method further includes: reading the process state data from the preset storage engine.

8. The method according to any one of claims 1 to 7, characterized in that: Before performing multilateral verification on each of the data to be verified in the queue of data to be verified based on the process state data, the method further includes: Obtaining the query rate per second of the to-be-verified data queue; Determine whether the query rate per second of the to-be-verified data queue is greater than a preset threshold; If so, the query rate per second in the to-be-verified data queue is limited to the preset threshold.

9. The method according to any one of claims 1 to 7, characterized in that: Before performing multilateral verification on each of the data to be verified in the queue of data to be verified based on the process state data, the method further includes: The data to be verified in the queue of data to be verified is filtered based on preset rules.

10. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In a case where it is determined that first abnormal data exists in the target business system, obtaining a type of the first abnormal data according to a multilateral verification result corresponding to the first abnormal data; The type of the first abnormal data is output.

11. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: When it is determined that the target business system has second abnormal data, a compensation strategy corresponding to the second abnormal data is called to compensate for the second abnormal data.

12. A verification device for a business system, characterized in that: include: An acquisition unit, used to acquire process state data of a target business system; A preprocessing unit, configured to obtain a data stream of at least one service node of the target service system according to the process state data; A first verification unit, configured to verify the data stream of the at least one service node to obtain the data to be verified in the data stream of the at least one service node, and add the data to be verified to a queue of data to be verified; A second verification unit, configured to perform multilateral verification on each of the data to be verified in the data queue to be verified based on the process state data, so as to filter out noise data in the data queue to be verified; A processing unit is used to determine whether there is abnormal data in the target business system based on the queue of data to be verified after filtering out noise data.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the verification method of the business system according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the verification method for the business system according to any one of claims 1 to 11.

Images