Data deviation degree detection method and device, storage medium and electronic equipment

Abstract

The application discloses a data deviation detection method and device, a storage medium and an electronic equipment. It relates to the field of big data. The method comprises the following steps: receiving an initial query statement for querying a target data source, wherein the predetermined query statement is used to obtain a query value of a predetermined index in the target data source at a query time; splitting the initial query statement to obtain a predetermined query statement and an instruction statement; modifying the predetermined query statement based on the instruction statement to obtain a target query statement, wherein the target query statement is used to obtain a historical value of the predetermined index at a first historical time before the query time; obtaining a data deviation of the predetermined index based on the predetermined query statement and the target query statement; and sending the data deviation to a predetermined detection server. Through the application, the problem that the predetermined detection server lacks the detection capability of the data deviation in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of big data, and more specifically, to a data deviation detection method, apparatus, storage medium, and electronic device. Background Technology

[0002] Currently, when using Grafana (an open-source data analytics and visualization suite) to provide log data visualization and alerting capabilities for data systems, its functionality is limited, only allowing threshold detection and alerting for single metric values. With increasing alerting demands, applications need to display and alert on metric deviations. Since Grafana's native data source does not support metric deviation calculation, one related technology requires developing new software with the necessary functionality to replace Grafana, which is not only costly but also complex to implement. Another related technology involves introducing third-party data source plugins to add deviation detection capabilities; however, existing third-party data source plugins only provide display functionality and cannot support Grafana's alerting capabilities, failing to integrate with Grafana without compromising existing functionality, resulting in unsatisfactory data detection results.

[0003] There is currently no effective solution to the problem that the predetermined detection server in the related technologies lacks the ability to detect data deviation. Summary of the Invention

[0004] The main objective of this application is to provide a data deviation detection method, apparatus, storage medium, and electronic device to address the problem in related technologies where predetermined detection servers lack the capability to detect data deviation.

[0005] To achieve the above objectives, according to one aspect of this application, a data deviation detection method is provided. The method includes: receiving an initial query statement for querying a target data source, wherein the initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source, the predetermined query statement being a predetermined query format supporting interaction with the target data source, the predetermined query statement being used to obtain a query value corresponding to a predetermined indicator in the target data source at a query time, the predetermined detection server querying the target data source through the intermediate node; splitting the initial query statement to obtain the predetermined query statement and the instruction statement; modifying the predetermined query statement based on the instruction statement to obtain a target query statement, wherein the target query statement is used to obtain a historical value corresponding to a first historical time before the query time for the predetermined indicator, the target query statement employing the predetermined query format; obtaining the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement; and sending the data deviation to the predetermined detection server.

[0006] To achieve the above objectives, according to another aspect of this application, a data deviation detection device is provided. The device includes: a receiving module, configured to receive an initial query statement for querying a target data source, wherein the initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source, the predetermined query statement being a predetermined query format supporting interaction with the target data source, the predetermined query statement being used to obtain a query value corresponding to a predetermined indicator in the target data source at a query time, the predetermined detection server querying the target data source through the intermediate node; a splitting module, configured to split the initial query statement to obtain the predetermined query statement and the instruction statement; a correction module, configured to correct the predetermined query statement based on the instruction statement to obtain a target query statement, wherein the target query statement is used to obtain a historical value corresponding to a first historical time before the query time for the predetermined indicator, the target query statement employing the predetermined query format; a first acquisition module, configured to acquire the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement; and a first sending module, configured to send the data deviation to the predetermined detection server.

[0007] To achieve the above objectives, according to another aspect of this application, a non-volatile storage medium is provided, the non-volatile storage medium storing a plurality of instructions adapted for loading by a processor and executing any one of the data deviation detection methods described herein.

[0008] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, comprising: one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any one of the data deviation detection methods described above.

[0009] This application employs the following steps: receiving an initial query statement for querying a target data source, wherein the initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value of a predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through the intermediate node. The initial query statement is split to obtain the predetermined query statement and the instruction statement. Based on the instruction statement, the predetermined query statement is modified to obtain a target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at a first historical time before the query time, and the target query statement adopts the predetermined query format. Based on the predetermined query statement and the target query statement, the data deviation of the predetermined indicator is obtained. The data deviation is sent to the predetermined detection server. This achieves the goal of adding data deviation detection capability to the predetermined detection server and solves the technical problem in related technologies where the predetermined detection server lacks the ability to detect data deviation. This effectively reduces the limitations of using the pre-defined detection server. Attached Figure Description

[0010] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0011] Figure 1 This is a flowchart of a data deviation detection method provided according to an embodiment of this application;

[0012] Figure 2 This is a schematic block diagram of a data deviation detection method provided according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of the data deviation detection method provided in the embodiments of this application; and

[0014] Figure 4 This is a schematic diagram of a data deviation detection device provided according to an embodiment of this application. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0019] Grafana is an open-source data analysis and visualization suite, most commonly used for visual analysis of application system metrics or time-series data such as logs, for example, for visualizing log data and issuing alerts in data systems.

[0020] It should be noted that all information (including but not limited to the reservation server, target data source, etc.) and data (including but not limited to the data queried in the target data source) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, there is an interface between the reservation server and the target data source. Before obtaining the data monitoring results, it is necessary to send a request to the aforementioned target data source through the interface, and obtain the relevant information after receiving the consent information from the aforementioned target data source.

[0021] Figure 1 This is a flowchart of a data deviation detection method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps, with the following execution entity being an intermediate node:

[0022] Step S102: Receive an initial query statement for querying the target data source. The initial query statement includes a predetermined query statement for the predetermined detection server to query the target data source, and an instruction statement for the intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value of the predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through the intermediate node.

[0023] It is understood that the original pre-defined detection server directly queries the target data source. To enrich the data detection capabilities of the pre-defined detection server, this embodiment adds an intermediate node, enabling the pre-defined detection server to query the target data source through the intermediate node. The intermediate node receives the initial query statement from the pre-defined detection server, which includes a pre-defined query statement from the pre-defined detection server and an instruction statement instructing the intermediate node to query the target data source. Of the pre-defined query statement and the instruction statement included in the initial query statement, the pre-defined query statement can directly query the target data source. It uses an interactive pre-defined query format and can be parsed and read, but can only obtain the query values ​​of pre-defined indicators in the target data source. The instruction statement included in the initial query statement is used by the pre-defined detection server to instruct the intermediate node to query, i.e., to inform the intermediate node of the relevant information to be queried. It should be noted that the instruction statement cannot directly interact with the target data source; therefore, the initial query statement containing the instruction statement also cannot directly interact with the target data source.

[0024] Optionally, the aforementioned pre-defined detection server can be Grafana.

[0025] Optionally, there are several ways to generate the initial query statement. For example, configuration settings can be made in the pre-defined detection server. First, the target data source is configured in the pre-defined detection server so that it can be queried. The access method is through an intermediate node, and the node address of the intermediate node is set in the pre-defined detection data source. Then, the initial query statement, query range, data aggregation metrics, aggregation dimensions, etc., are configured in the pre-defined detection server. When configuring the initial query statement, the query time, the address of the target data source, the calculation rules for determining the deviation, and whether to calculate the absolute value need to be determined according to specific requirements. Query time periods for each moment within a certain period can also be added for batch queries. Configuration is based on specific needs and is not specifically limited. When generating the initial query statement, the pre-defined detection server will generate the pre-defined query statement according to the above configuration information and the original pre-defined query format. It will also generate an instruction statement to the intermediate node according to the above-configured query time, target data source address, calculation rules, and whether to calculate the absolute value. Combining the pre-defined query statement and the instruction statement generates the initial query statement sent to the intermediate node.

[0026] To make it easier to understand, let's take a specific example: the initial query statement is "@log_name.keyword: The query query is: `mlcc.log AND pubMetric.serviceType.keyword:T AND pubMetric.logType:E##1h##122.46.137.167##value##noabs`, where the predefined query for the target data source is `@log_name.keyword:`. The command `mlcc.log AND pubMetric.serviceType.keyword:T AND pubMetric.logType:E` is executed. The instruction statement is `##1h##122.46.137.167##value##noabs`, where `1h` (hour) is the query time period, and the query is performed in batches for each query time. `122.46.137.167` is the address of the target data source, `value` is the calculation rule for calculating the difference, and `noabs` indicates that the absolute value is not calculated.

[0027] Step S104: Split the initial query statement to obtain the predefined query statement and the instruction statement;

[0028] It is understandable that, since the initial query statement includes a predefined query statement and an instruction statement, and the instruction statement is not in the predefined query format, the intermediate node needs to separate the predefined query statement and the instruction statement from the initial query statement.

[0029] Step S106: Based on the instruction statement, the predetermined query statement is modified to obtain the target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at the first historical time before the query time, and the target query statement adopts the predetermined query format.

[0030] It is understandable that the instruction statement can instruct the intermediate nodes to modify the predetermined query statement so that the generated target query statement can retrieve the historical value corresponding to the first historical moment.

[0031] In one optional embodiment, modifying a predetermined query statement based on an instruction statement to obtain a target query statement includes: if the instruction statement includes a query time, modifying the predetermined query statement based on a predetermined time period and the query time to obtain the target query statement.

[0032] It is understandable that, when the above instruction statement includes a query time, in order to retrieve the historical value of the first historical moment before the query time within a predetermined time period, the predetermined query statement needs to be modified to obtain the target query statement. The target query statement also follows a predetermined query format and can directly interact with the target data source. Through the above processing, intermediate nodes can generate a target query statement that can retrieve historical values ​​based on the information provided by the instruction statement.

[0033] Optionally, the instruction statement can contain multiple query times to enable batch queries within a certain period, which can yield multiple historical values.

[0034] Step S108: Based on the predetermined query statement and the target query statement, obtain the data deviation of the predetermined indicator;

[0035] It is understandable that intermediate nodes can obtain the data deviation of predetermined indicators in the target data source based on the predetermined query statement and the target query statement.

[0036] In one optional embodiment, obtaining the data deviation of a predetermined indicator based on a predetermined query statement and a target query statement includes: using the predetermined query statement to obtain a query value; using the target query statement to obtain a historical value; and, if the instruction statement includes a calculation rule, using the calculation rule to process the query value and the historical value to obtain the data deviation.

[0037] It is understandable that intermediate nodes use a predefined query statement to obtain the query value and a target query statement to obtain the historical value. When the instruction statement includes a calculation rule, the intermediate node uses the calculation rule to process the obtained query value and historical value to obtain the data deviation.

[0038] Optionally, the above calculation rules can be various, for example: calculating the difference between the query value and the historical value, or the percentage difference. For cases where the calculation rule is to calculate the difference, the data deviation can be determined by... The mathematical expression is obtained, where, For query value, These are historical values. For calculations where the rule is to calculate the percentage difference, the data deviation is determined by... The mathematical expression is obtained.

[0039] In one optional embodiment, a calculation rule is used to process the query value and historical value to obtain the data deviation, including: when the instruction statement includes an absolute value calculation instruction, the calculation rule is used to process the query value and historical value to obtain an initial result; the initial result is then subjected to absolute value calculation to obtain the data deviation.

[0040] It's understandable that when the instruction statement includes an absolute value calculation command, the intermediate node uses calculation rules to process the query value and historical values ​​to obtain an initial result. Since the magnitude of the deviation between the query value and historical values ​​is variable, when only the magnitude of the deviation is considered and not the direction, an absolute value calculation command is included in the instruction statement. The intermediate node then performs absolute value calculation on the initial result to obtain the data deviation.

[0041] Step S110: Send the data deviation to the designated detection server.

[0042] It is understandable that the intermediate nodes do not have the visualization capabilities of the pre-defined detection server in order to enrich the functional settings of the pre-defined detection server, so it is necessary to send the data deviation back to the pre-defined detection server.

[0043] In one optional embodiment, sending the data deviation to a predetermined detection server includes: determining a predetermined return format of the predetermined detection server; encapsulating the data deviation using the predetermined return format to obtain a detection feedback statement; and sending the detection feedback statement to the predetermined detection server.

[0044] It is understandable that the pre-defined return format supports interaction with the pre-defined detection server. Intermediate nodes use this pre-defined return format to encapsulate the data deviation and generate detection feedback statements. The intermediate nodes then send these feedback statements to the pre-defined detection server.

[0045] In one optional embodiment, after sending the detection feedback statement to the predetermined detection server, the predetermined detection server performs the following processing: based on the detection feedback statement, determines the data deviation of the predetermined indicator; determines N second historical moments before the query time, and the historical deviation corresponding to each of the N second historical moments, where N is a positive integer greater than or equal to 1, and the query time and the N second historical moments are consecutive in time order; based on the historical deviation corresponding to each of the N second historical moments and the data deviation, obtains the deviation evaluation value corresponding to the query time; based on the deviation evaluation value corresponding to the query time, generates an alarm prompt.

[0046] It should be noted that the execution entity for the following processing is the pre-defined detection server. This means that the pre-defined detection server obtains the data deviation of a pre-defined indicator based on the received detection feedback statement. The pre-defined detection server records the historical deviation corresponding to N second historical moments, obtained using the same method as generating the data deviation. Since the N second historical moments are sequentially consecutive with the query time, they can be considered as the deviation over a period prior to the query time. Based on the historical deviations corresponding to the N second historical moments and the data deviation, a deviation evaluation value corresponding to the query time can be calculated. The pre-defined detection server can generate an alarm based on the deviation evaluation value corresponding to the query time, enabling relevant personnel to promptly detect anomalies in the target data source.

[0047] Optionally, the deviation evaluation value corresponding to the query time can be generated in various ways. For example, the average of the historical deviations and data deviations corresponding to N second historical times can be used as the deviation evaluation value. Other statistical methods, such as mode, median, variance, etc., can also be used to calculate the average, depending on specific needs.

[0048] In one optional embodiment, an alarm prompt is generated based on the deviation evaluation value corresponding to the query time, including: obtaining the deviation evaluation values ​​corresponding to M third historical times, wherein the M third historical times are before the query time, the query time and the M third historical times are consecutive in time order, and M is a positive integer greater than or equal to 1; and generating an alarm prompt when any one of the deviation evaluation values ​​corresponding to the M third historical times and the deviation evaluation value corresponding to the query time is greater than a predetermined alarm threshold.

[0049] It should be noted that the execution entity for the following processing is the predetermined detection server. This server records deviation evaluation values ​​for M third historical moments, which are historical moments preceding and consecutive to the query moment. Since a deviation evaluation value exceeding the threshold at a single moment would directly generate an alarm, leading to frequent and continuous alarms and impacting normal detection processing, it is necessary to evaluate the deviation value in conjunction with the deviation evaluation values ​​for the M third historical moments. An alarm is generated when each of the deviation evaluation values ​​for the M third historical moments and the deviation evaluation value for the query moment exceeds a predetermined alarm threshold.

[0050] In other words, if the M third historical moments are assumed to be within 5 minutes before the query time, it means that the deviation evaluation value has been greater than the predetermined alarm threshold for these 5 minutes. In this case, it can be determined that an abnormal situation has occurred, and an alarm prompt will be generated.

[0051] The data deviation detection method provided in this application aims to add data deviation detection capability to a predetermined detection server, solving the problem in related technologies where the predetermined detection server lacks the ability to detect data deviation. This, in turn, reduces the limitations of using the predetermined detection server.

[0052] Based on the above embodiments, this application also provides a specific implementation method. Figure 2 This is a schematic block diagram of a data deviation detection method provided in the embodiments of this application. Figure 2 Grafana, as a pre-defined detection server, cannot obtain data deviation when directly querying the target data source. The target data source can be ElasticSearchCluster (ES), a distributed real-time log analytics cluster. It allows users to store, search, and analyze large amounts of data in a distributed environment. Each node contains a complete copy of the index and can communicate and synchronize through a coordinator node.

[0053] To enable Grafana to detect data deviation in Elasticsearch, an intermediate node is set up so that Grafana does not directly query Elasticsearch, but processes the data through the intermediate node and obtains the data deviation returned by the intermediate node, and then provides visualization and alert services for the data deviation.

[0054] The Grafana page configures the data source and dashboard settings. The data source is configured as the node address of the Grafana intermediate node, and the dashboard is configured with the initial query statement for querying Elasticsearch and the preset alarm parameters in Grafana.

[0055] After receiving a query request from Grafana, the intermediate node parses it to obtain the initial query statement. It's important to note that this initial query statement cannot directly query Elasticsearch; it's a non-predefined query format. The initial query statement includes two parts: a predefined query statement in a predefined format and an instruction statement. The predefined query statement can directly interact with Elasticsearch. Based on the initial query statement, the intermediate node generates the target query statement, processes the results returned by Elasticsearch, and obtains the data deviation. Finally, the intermediate node generates a detection feedback statement based on the calculated data deviation in a predefined return format and sends it to Grafana.

[0056] Grafana parses the detection feedback statements returned by intermediate nodes and displays the data deviation on the dashboard. At the same time, it monitors and alerts the data deviation in real time according to the pre-defined alert configuration information in Grafana.

[0057] Figure 3 This is a schematic diagram of the data deviation detection method provided in the embodiments of this application, showing Grafana's ability to display data deviation. When the data deviation exceeds 1, that is, higher than the part shown by the dotted line, and the duration is greater than the preset 5 minutes, Grafana will issue a prompt.

[0058] According to the above implementation method, the embodiments of this application can achieve the following effects: based on the existing monitoring and alarm functions of Grafana, optimization is performed without the need to introduce third-party data source plugins, and the data deviation display and alarm capabilities of Grafana are realized at low cost, meeting the increasing alarm needs of connected applications.

[0059] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0060] This application also provides a data deviation detection device. It should be noted that the data deviation detection device of this application can be used to execute the data deviation detection method provided in this application. The data deviation detection device provided in this application is described below.

[0061] Figure 4 This is a schematic diagram of a data deviation detection device according to an embodiment of this application. Figure 4 As shown, the device includes: a first receiving module 402, a first splitting module 404, a first correction module 406, a first acquisition module 408, and a first sending module 410, which are described in detail below:

[0062] The first receiving module 402 is used to receive an initial query statement for querying the target data source. The initial query statement includes a predetermined query statement for the predetermined detection server to query the target data source, and an instruction statement for the intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value corresponding to the predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through the intermediate node.

[0063] The first splitting module 404 is used to split the initial query statement to obtain the predetermined query statement and the instruction statement;

[0064] The first correction module 406 is used to correct the predetermined query statement based on the instruction statement to obtain the target query statement. The target query statement is used to obtain the historical value of the predetermined indicator at the first historical time before the query time. The target query statement adopts a predetermined query format.

[0065] The first acquisition module 408 is used to acquire the data deviation of a predetermined indicator based on the predetermined query statement and the target query statement;

[0066] The first sending module 410 is used to send the data deviation to the predetermined detection server.

[0067] The data deviation detection device provided in this application embodiment includes a first receiving module 402 for receiving an initial query statement for querying a target data source. The initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value of a predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through an intermediate node. A first splitting module 404 is used to split the initial query statement to obtain the predetermined query statement and the instruction statement. A first correction module 406 is used to correct the predetermined query statement based on the instruction statement to obtain a target query statement. The target query statement is used to obtain the historical value of the predetermined indicator at a first historical time before the query time, and the target query statement adopts a predetermined query format. A first acquisition module 408 is used to acquire the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement. A first sending module 410 is used to send the data deviation to the predetermined detection server. This achieves the goal of adding data deviation detection capability to the pre-defined detection server, solving the problem of the lack of data deviation detection capability in the pre-defined detection server in related technologies, thereby reducing the limitations of the pre-defined detection server in use.

[0068] In an optional embodiment, the first correction module 406 includes a second correction module, configured to correct a predetermined query statement based on a predetermined time period and the query time when the instruction statement includes a query time, to obtain a target query statement.

[0069] In an optional embodiment, the first acquisition module 408 includes: a second acquisition module, used to obtain a query value using a predetermined query statement; a third acquisition module, used to obtain a historical value using a target query statement; and a first calculation module, used to process the query value and the historical value using calculation rules when the instruction statement includes calculation rules, to obtain the data deviation.

[0070] In an optional embodiment, the first calculation module includes: a second calculation module, used to process the query value and historical value using calculation rules to obtain an initial result when the instruction statement includes an absolute value calculation instruction; and a third calculation module, used to perform absolute value calculation on the initial result to obtain the data deviation.

[0071] In an optional embodiment, the first sending module 410 includes: a first determining module, configured to determine a predetermined return format of a predetermined detection server; a first encapsulation module, configured to encapsulate the data deviation using the predetermined return format to obtain a detection feedback statement; and a second sending module, configured to send the detection feedback statement to the predetermined detection server.

[0072] In an optional embodiment, the device further includes: a second determining module, configured to determine the data deviation of a predetermined indicator based on a detection feedback statement; a third determining module, configured to determine N second historical times prior to the query time, and the historical deviation corresponding to each of the N second historical times, wherein N is a positive integer greater than or equal to 1, and the query time and the N second historical times are consecutive in time order; a fourth obtaining module, configured to obtain a deviation evaluation value corresponding to the query time based on the historical deviation corresponding to each of the N second historical times and the data deviation; and a first generating module, configured to generate an alarm prompt based on the deviation evaluation value corresponding to the query time.

[0073] In an optional embodiment, the first generation module includes: a fifth acquisition module, configured to acquire deviation evaluation values ​​corresponding to M third historical moments, wherein the M third historical moments are prior to the query moment, the query moment and the M third historical moments are consecutive in time, and M is a positive integer greater than or equal to 1; and a second generation module, configured to generate an alarm prompt when any one of the deviation evaluation values ​​corresponding to the M third historical moments and the deviation evaluation value corresponding to the query moment is greater than a predetermined alarm threshold.

[0074] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0075] It should be noted that the receiving module 402, splitting module 404, correction module 406, first acquisition module 408, and first sending module 410 correspond to steps S102 to S110 in the embodiments. The instances and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules, as part of the device, can run in a computer terminal.

[0076] It should be noted that the optional or preferred implementation methods of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.

[0077] The aforementioned data deviation detection device includes a processor and a memory. All of the aforementioned units are stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to achieve the corresponding functions.

[0078] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and data deviation detection can be achieved by adjusting kernel parameters.

[0079] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0080] This invention provides a non-volatile storage medium storing a program that, when executed by a processor, implements a data deviation detection method.

[0081] This invention provides a processor for running a program, wherein the program executes a data deviation detection method during runtime.

[0082] This invention provides an electronic device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: receiving an initial query statement for querying a target data source, wherein the initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source and is used to obtain the query value of a predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through an intermediate node. The initial query statement is then split to obtain a predetermined query statement and an instruction statement. Based on the instruction statement, the predetermined query statement is modified to obtain a target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at a first historical time before the query time, and the target query statement adopts a predetermined query format. Based on the predetermined query statement and the target query statement, the data deviation of the predetermined indicator is obtained. The data deviation is then sent to the predetermined detection server. The device described herein can be a server, PC, PAD, mobile phone, etc.

[0083] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: receiving an initial query statement for querying a target data source, wherein the initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source, the predetermined query statement being a predetermined query format that supports interaction with the target data source, the predetermined query statement being used to obtain the query value of a predetermined indicator in the target data source at the query time, the predetermined detection server querying the target data source through an intermediate node; splitting the initial query statement to obtain a predetermined query statement and an instruction statement; modifying the predetermined query statement based on the instruction statement to obtain a target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at the first historical time before the query time, the target query statement adopting a predetermined query format; obtaining the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement; and sending the data deviation to the predetermined detection server.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0089] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0090] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0091] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting data deviation, characterized in that, include: The system receives an initial query statement to query a target data source. The initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value of a predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through the intermediate node. The initial query statement is split to obtain the predetermined query statement and the instruction statement; Based on the instruction statement, the predetermined query statement is modified to obtain the target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at the first historical time before the query time, and the target query statement adopts the predetermined query format; Based on the predetermined query statement and the target query statement, the data deviation of the predetermined indicator is obtained; The data deviation is sent to the predetermined detection server; The step of modifying the predetermined query statement based on the instruction statement to obtain the target query statement includes: when the instruction statement includes a query time, modifying the predetermined query statement based on a predetermined time period and the query time to obtain the target query statement.

2. The method according to claim 1, characterized in that, The step of obtaining the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement includes: The query value is obtained by using the predetermined query statement; The historical value is obtained by using the target query statement; If the instruction statement includes a calculation rule, the calculation rule is used to process the query value and the historical value to obtain the data deviation.

3. The method according to claim 2, characterized in that, The process of using the calculation rules to process the query value and the historical value to obtain the data deviation includes: If the instruction statement includes an absolute value calculation command, the calculation rules are used to process the query value and the historical value to obtain an initial result; The absolute value of the initial result is calculated to obtain the data deviation.

4. The method according to any one of claims 1 to 3, characterized in that, Sending the data deviation to the predetermined detection server includes: Determine the predetermined return format of the predetermined detection server; The data deviation is encapsulated using the predetermined return format to obtain a detection feedback statement; The detection feedback statement is sent to the predetermined detection server.

5. The method according to claim 4, characterized in that, After the detection feedback statement is sent to the predetermined detection server, the predetermined detection server performs the following processing: Based on the detection feedback statement, the data deviation of the predetermined indicator is determined; Determine the N second historical moments preceding the query time, and the historical deviation corresponding to each of the N second historical moments, where N is a positive integer greater than or equal to 1, and the query time and the N second historical moments are consecutive in time. Based on the historical deviations corresponding to the N second historical moments and the data deviations, the deviation evaluation value corresponding to the query moment is obtained. An alarm is generated based on the deviation evaluation value corresponding to the query time.

6. The method according to claim 5, characterized in that, The generation of an alarm notification based on the deviation evaluation value corresponding to the query time includes: Obtain the deviation evaluation values ​​corresponding to M third historical moments, wherein the M third historical moments are those prior to the query time, the query time and the M third historical moments are consecutive in time, and M is a positive integer greater than or equal to 1. If any one of the deviation evaluation values ​​corresponding to the M third historical times and the deviation evaluation value corresponding to the query time is greater than the predetermined alarm threshold, the alarm prompt is generated.

7. A data deviation detection device, characterized in that, include: A receiving module is configured to receive an initial query statement for querying a target data source. The initial query statement includes a predetermined query statement for a predetermined detection server to query the target data source, and an instruction statement instructing an intermediate node to query the target data source. The predetermined query statement is a predetermined query format that supports interaction with the target data source. The predetermined query statement is used to obtain the query value of a predetermined indicator in the target data source at the query time. The predetermined detection server queries the target data source through the intermediate node. The splitting module is used to split the initial query statement to obtain the predetermined query statement and the instruction statement; The correction module is used to correct the predetermined query statement based on the instruction statement to obtain the target query statement, wherein the target query statement is used to obtain the historical value of the predetermined indicator at the first historical time before the query time, and the target query statement adopts the predetermined query format. The first acquisition module is used to acquire the data deviation of the predetermined indicator based on the predetermined query statement and the target query statement; The first sending module is used to send the data deviation to the predetermined detection server; The correction module is further configured to, when the instruction statement includes a query time, correct the predetermined query statement based on a predetermined time period and the query time to obtain the target query statement.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores multiple instructions, which are adapted to be loaded by a processor and executed by the data deviation detection method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: One or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the data deviation detection method according to any one of claims 1 to 6.

Images