Data exception processing method, storage medium and electronic device

Abstract

The application discloses a data exception processing method, a storage medium and an electronic device. The method comprises the following steps: acquiring a data missing rate of each data table in a group of data tables and a data fluctuation parameter of each data table, the group of data tables comprising business data generated by a target application during runtime, and the data fluctuation parameter of each data table being used for representing a fluctuation degree of the business data in each data table over time; determining a data quality parameter of each data table according to the data missing rate of each data table and the data fluctuation parameter of each data table; determining a weight value of each data table according to operation information of a multi-dimensional operation performed on each data table, the weight value of each data table being used for representing an importance degree of each data table; and determining that the business data in the group of data tables is abnormal in a case where the data quality parameter of each data table and the weight value of each data table satisfy a preset abnormal condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and in particular, to a data exception processing method, a storage medium and an electronic device. BACKGROUND

[0002] Currently, business data generated in the process of application running can be stored in a data table of a database. If the business data in the data table is abnormal, the accuracy of data operations performed based on the business data will be affected. In order to discover the abnormality of the business data in time, the integrity of the data table can be detected.

[0003] In the related art, the integrity of each data table and the integrity of the entire database can be measured from the dimension of data missing rate. For each data table, the data missing rate of the data table can be determined by the missing business data in the data table within a period of time and all business data in the data table within the period of time, and then the integrity measurement of the data table is obtained.

[0004] However, since the business data has a certain randomness, even if the missing rate of the business data is low, the existing business data can still have abnormalities, which leads to the inability to accurately identify the data exception of the data table. As can be seen, the way of identifying the data exception in the data table by the data missing rate in the related art has the problem of low accuracy of identifying the data exception of the data table. SUMMARY

[0005] Embodiments of the present application provide a data exception processing method, a storage medium and an electronic device to at least solve the problem of low accuracy of identifying the data exception of the data table in the related art by the way of identifying the data exception in the data table by the data missing rate.

[0006] According to an aspect of the embodiments of the present application, a method for processing data anomaly is provided. The method comprises: obtaining a data missing rate of each data table in a group of data tables, and obtaining a data fluctuation parameter of each data table in the group of data tables, wherein the group of data tables comprises business data generated by a target application during runtime, and the data fluctuation parameter of each data table is used to represent a degree of fluctuation of the business data in the data table over time; determining a data quality parameter of each data table in the group of data tables according to the data missing rate of each data table in the group of data tables and the data fluctuation parameter of each data table in the group of data tables; determining a weight value of each data table in the group of data tables according to operation information of a multi-dimensional operation performed on each data table in the group of data tables, wherein the weight value of each data table in the group of data tables is used to represent an importance of the data table; and determining that the business data in the group of data tables is abnormal in a case where the data quality parameter of each data table in the group of data tables and the weight value of each data table in the group of data tables satisfy a preset abnormal condition.

[0007] According to another aspect of the embodiments of the present application, a device for processing data anomaly is also provided. The device comprises: a first obtaining unit configured to obtain a data missing rate of each data table in a group of data tables, and obtain a data fluctuation parameter of each data table in the group of data tables, wherein the group of data tables comprises business data generated by a target application during runtime, and the data fluctuation parameter of each data table is used to represent a degree of fluctuation of the business data in the data table over time; a first determining unit configured to determine a data quality parameter of each data table in the group of data tables according to the data missing rate of each data table in the group of data tables and the data fluctuation parameter of each data table in the group of data tables; a second determining unit configured to determine a weight value of each data table in the group of data tables according to operation information of a multi-dimensional operation performed on each data table in the group of data tables, wherein the weight value of each data table in the group of data tables is used to represent an importance of the data table; and a third determining unit configured to determine that the business data in the group of data tables is abnormal in a case where the data quality parameter of each data table in the group of data tables and the weight value of each data table in the group of data tables satisfy a preset abnormal condition.

[0008] As an optional solution, the first obtaining unit comprises an execution module configured to perform the following steps for each data table in the group of data tables, wherein, when performing the following steps, the each data table is a current data table: determining a data item number of service data generated in each preset period in a group of preset periods in the current data table, to obtain a data item number sequence, wherein the data item number sequence comprises a current data item number and a group of historical data item numbers, the current data item number is a data item number of service data generated in a last preset period in the group of preset periods in the current data table, and the group of historical data item numbers are data item numbers in the data item number sequence except the current data item number; determining a data fluctuation parameter of the current data table according to fluctuation of the current data item number relative to the group of historical data item numbers.

[0009] As an optional solution, the execution module comprises: a first determining submodule configured to determine an average item number of the group of historical data item numbers in a case that the data item number sequence is a stationary sequence; a second determining submodule configured to determine the data fluctuation parameter of the current data table as a first parameter value in a case that an item number difference between the current data item number and the average item number is greater than a preset difference threshold, wherein the first parameter value is used to represent that service data generated in the last preset period in the current data table is fluctuated abnormally.

[0010] As an optional solution, the execution module further comprises: a first detecting submodule configured to detect whether the current data item number is located in a first item number reference range in a case that the item number difference between the current data item number and the average item number is less than or equal to the preset difference threshold, wherein the first item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the group of historical data item numbers; a third determining submodule configured to determine the data fluctuation parameter of the current data table as a second parameter value in a case that the current data item number is located in the first item number reference range, wherein the second parameter value is different from the first parameter value; and a fourth determining submodule configured to determine the data fluctuation parameter of the current data table as the first parameter value in a case that the current data item number is located out of the first item number reference range.

[0011] As an optional solution, the execution module comprises: a fifth determination submodule, configured to, in a case where the data item number sequence is a non-stationary sequence and a periodic sequence, and the set of preset periods comprises a plurality of preset period sets and a current preset period set, determine, from the set of historical data items, a target data item number corresponding to the current data item number in each preset period set of the plurality of preset period sets, to obtain a set of target data items, wherein the number of preset periods in each preset period set of the plurality of preset period sets and the current preset period set is the same, the current data item number is a data item number of service data generated in a current preset period in the current preset period set, and a preset period corresponding to each target data item has the same position in the corresponding preset period set as the current preset period in the current preset period set; a second detection submodule, configured to detect whether the current data item number is located in a second item number reference range, wherein the second item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the set of target data items; a sixth determination submodule, configured to, in a case where the current data item number is located outside the second item number reference range, determine that a data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to indicate that service data generated in the last preset period in the current data table fluctuates abnormally; and a seventh determination submodule, configured to, in a case where the current data item number is located in the second item number reference range, determine that the data fluctuation parameter of the current data table is a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to indicate that service data generated in the last preset period in the current data table fluctuates normally.

[0012] As an optional solution, the execution module comprises: a third detection submodule, configured to, in a case where the data item number sequence is a non-stationary sequence and a non-periodic sequence, detect whether the current data item number is located in a third item number reference range, wherein the third item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the set of historical data items; an eighth determination submodule, configured to, in a case where the current data item number is located outside the third item number reference range, determine that a data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to indicate that service data generated in the last preset period in the current data table fluctuates abnormally; and a ninth determination submodule, configured to, in a case where the current data item number is located in the third item number reference range, determine that the data fluctuation parameter of the current data table is a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to indicate that service data generated in the last preset period in the current data table fluctuates normally.

[0013] As an optional solution, the apparatus further comprises: an extracting unit, configured to perform a fluctuation coefficient extracting operation on the data item number sequence to obtain a set of fluctuation coefficients; a fourth determining unit, configured to determine that the data item number sequence is a stationary type sequence in a case where a ratio of a standard deviation of the data item number sequence to a standard deviation of the set of fluctuation coefficients is less than or equal to a preset ratio; and a fifth determining unit, configured to determine that the data item number sequence is a non-stationary type sequence in a case where the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients is less than the preset ratio.

[0014] As an optional solution, the apparatus further comprises: a second obtaining unit, configured to obtain, in the data item number sequence, a data item number corresponding to each preset period in a first preset period set to obtain a first reference item number sequence, and obtain a data item number corresponding to each preset period in a second preset period set to obtain a second reference item number sequence, wherein the first preset period set and the second preset period set are adjacent preset period sets in the set of preset periods; a sixth determining unit, configured to determine a difference value of data item numbers at each same position of the first reference item number sequence and the second reference item number sequence to obtain a target item number difference sequence; a seventh determining unit, configured to determine that the data item number sequence is a periodic type sequence in a case where a standard deviation of the target item number difference sequence is less than or equal to a preset standard deviation threshold; and an eighth determining unit, configured to determine that the data item number sequence is a non-periodic type sequence in a case where the standard deviation of the target item number difference sequence is greater than the preset standard deviation threshold.

[0015] As an optional solution, the second determining unit comprises: a first determining module, configured to determine a weight value of each dimension operation in the multi-dimensional operation according to an operation number of the multi-dimensional operation performed on each data table in the set of data tables; a second determining module, configured to determine a reference evaluation value of each data table according to the operation number of the multi-dimensional operation performed on the each data table and the weight value of the each dimension operation; and a third determining module, configured to determine a weight value of each data table according to the reference evaluation value of the each data table.

[0016] As an optional solution, the first determining module comprises: a tenth determining sub-module, configured to determine a weight value of each dimension operation in the multi-dimensional operation performed on each data table in the set of data tables; and an eleventh determining sub-module, configured to determine a weight value of each dimension operation according to the weight value of the each dimension operation in the each data table.

[0017] As an optional solution, the tenth determining sub-module comprises: an executing sub-unit, configured to perform a standardization operation on the operation number of each data table in the group of data tables under each dimension operation in the multi-dimension operation, to obtain a first reference value of each data table under each dimension operation, wherein the first reference value of each data table under each dimension operation is a ratio of a first difference value and a second difference value, the first difference value is a difference between the operation number of each data table under each dimension operation and a first operation number, the second difference value is a difference between a second operation number and the first operation number, the first operation number is a minimum operation number in the operation number of the group of data tables under each dimension operation, and the second operation number is a maximum operation number in the operation number of the group of data tables under each dimension operation; and a determining sub-unit, configured to determine a ratio of the first reference value of each data table under each dimension operation and a second reference value as a weight value of each dimension operation under each data table, wherein the second reference value is a sum of the first reference value of each data table under each dimension operation.

[0018] As an optional solution, the third determining module comprises: a twelfth determining sub-module, configured to determine a ratio of the reference evaluation value of each data table and a reference weight value of the group of data tables as the weight value of each data table, wherein the reference weight value of the group of data tables is a sum of the reference evaluation value of each data table; or a first executing sub-module, configured to perform weighted summation on the reference evaluation value of each data table and a preset evaluation value of each data table, to obtain an updated reference evaluation value of each data table; and a thirteenth determining sub-module, configured to determine a ratio of the updated reference evaluation value of each data table and an updated reference weight value of the group of data tables as the weight value of each data table, wherein the updated reference weight value of the group of data tables is a sum of the updated reference evaluation value of each data table.

[0019] As an optional solution, the third determining unit comprises: a second executing sub-module, configured to perform weighted summation on the data quality parameter of each data table in the group of data tables according to the weight value of each data table, to obtain a data quality parameter of the group of data tables; and a fourteenth determining sub-module, configured to determine that the service data in the group of data tables is abnormal in a case where the data quality parameter of the group of data tables is greater than or equal to a preset parameter threshold.

[0020] According to another aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is configured to execute the data exception processing method when running.

[0021] According to a further aspect of the embodiments of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data exception processing method as above.

[0022] According to a further aspect of the embodiments of the present application, an electronic device is also provided, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to execute the data exception processing method as above by using the computer program.

[0023] In the embodiments of the present application, the data quality parameter of the data table is determined based on the data missing rate of the data table and the data fluctuation parameter of the data table. Since the missing rate of the data table can measure the integrity of the data table, and the data fluctuation parameter of the data table can measure the degree of fluctuation of the service data in the data table over time, the data quality of the data table can be measured from different dimensions, and the rationality of measuring the data quality in each data table is improved. Since the multi-dimensional operation performed on the data table can measure the importance of the data table from an objective dimension, the weight value of the data table is determined based on the operation information of the multi-dimensional operation performed on the data table. The determined weight value can be used to represent the importance of the data table in the database. The data quality of the entire database (or a group of data tables in the database) is measured based on the data quality parameter of the data table and the weight value of the data table, which can achieve the purpose of more accurately determining the data exception in the database, and achieve the technical effect of improving the accuracy of data exception identification, thereby solving the problem of low accuracy of data exception identification in the related art by using the data missing rate to identify the data exception in the data table. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate embodiments of the present application and a description thereof, and do not constitute an improper limitation to the present application. In the drawings:

[0025] Figure 1 is a schematic diagram of an application environment of an optional data exception processing method according to an embodiment of the present application;

[0026] Figure 2 is a flowchart of an optional data exception processing method according to an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of an optional core table according to an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of an optional partition table according to an embodiment of the present application;

[0029] Figure 5 is a schematic diagram of an optional abnormal partition table according to an embodiment of the present application;

[0030] Figure 6 is a schematic diagram of another optional abnormal partition table according to an embodiment of the present application;

[0031] Figure 7 is a schematic diagram of an optional data item number sequence type determination according to an embodiment of the present application;

[0032] Figure 8 is a schematic diagram of an optional wavelet transform according to an embodiment of the present application;

[0033] Figure 9 is a schematic diagram of a flow of an optional data item number sequence type determination according to an embodiment of the present application;

[0034] Figure 10 is a schematic diagram of a flow of another optional data abnormality processing method according to an embodiment of the present application;

[0035] Figure 11 is a structural block diagram of an optional data abnormality processing apparatus according to an embodiment of the present application;

[0036] Figure 12 is a structural diagram of an optional electronic device according to an embodiment of the present application;

[0037] Figure 13 is a structural block diagram of a computer system of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] According to one aspect of the embodiment of the present application, a method for processing data anomalies is provided. Optionally, as an optional implementation, the method for processing data anomalies can be applied to, but is not limited to, Figure 1 In the environment shown, the system may include but is not limited to: a terminal device 102 , which may include but is not limited to a memory 104 , a processor 106 , and a display 108 , a network 110 , and a server 112 .

[0041] For example, the process of the above data anomaly processing method may include the following steps:

[0042] In step S102, a target application, such as an instant messaging application, may be running on the terminal device 102. The terminal device 102 may obtain service data generated during the running of the target application. The obtained service data may be service data of all or part of the services allowed to be provided by the target application.

[0043] In steps S104 to S106 , the terminal device 102 may send the service data obtained through the network 110 to the server 112 .

[0044] In step S108, the server 112 saves the business data received from the terminal device 102 into the corresponding data table in the database 114 through the database 114 and the processing engine 116. The business data of different businesses can be saved in different data tables, and the data quality parameters of at least some of the data tables in the database and the weight value of each data table are determined. If the data quality parameters of each data table and the weight value of each data table meet the preset abnormal conditions, the server 112 can determine that the business data in the database 114 is abnormal.

[0045] Steps S110-S112, the server 112 sends the warning message of the service data anomaly to the terminal device 102 through the network 110, and the processor 106 in the terminal device 102 can save the received warning message, and can also display the warning message of the service data anomaly through the display 108, etc.

[0046] Here, the terminal device that acquires the service data and the terminal device that displays the warning message of the service data anomaly can be the same device, or can be different devices. In addition to the examples shown above, the steps described above can be completed independently by the terminal device 102, i.e., the terminal device 102 performs the steps of determining the service data anomaly, etc., thereby reducing the processing pressure of the server. Figure 1

[0047] In addition to the examples shown above, the steps described above can be completed independently by the terminal device 102, i.e., the terminal device 102 performs the steps of determining the service data anomaly, etc., thereby reducing the processing pressure of the server.

[0048] Optionally, as an optional implementation, taking the server 112 executing the data anomaly processing method in the embodiment as an example, Figure 2 is a flow diagram of an optional data anomaly processing method according to an embodiment of the application, as shown in Figure 2 the flow of the data anomaly processing method can include the following steps:

[0049] Step S202, acquiring the data missing rate of each data table in a group of data tables, and acquiring the data fluctuation parameter of each data table in the group of data tables, wherein the group of data tables includes service data generated by the target application when running, and the data fluctuation parameter of each data table is used to represent the degree of fluctuation of the service data in each data table over time.

[0050] ​The data anomaly processing method in the embodiment can be applied to a scenario of determining data anomalies of business data in a database through data quality parameters of data tables (referred to as library tables) in the database. The database can be a database corresponding to a target application and can be used to store business data generated by the target application during runtime. The target application can be an instant messaging application, a game application, a resource exchange application, etc., which is not limited in the embodiment.

[0051] The target application can provide various services, such as a registration service, a session service, a resource exchange service, etc. The database can include a plurality of data tables, and each of the plurality of data tables can be used to store business data of a service generated by the target application during runtime. The business data of a service can be stored in at least one data table.

[0052] In order to timely discover data anomalies of business data in the database, data anomalies of business data in the database can be detected based on data quality parameters of a group of data tables in the database, the group of data tables being all data tables in the database. In order to improve the efficiency of data anomaly detection, a group of data tables can be selected from the database according to a usage frequency of each candidate data table in the group of candidate data tables in the database and a preset reference value of each candidate data table in the database (which can be a reference value set based on business experience), and the group of data tables can be used as core tables of the database in this case. The group of candidate data tables can be data tables in the database that are allowed to be used to detect data anomalies of business data in the database.

[0053] In order to select the group of data tables, a first weight corresponding to each candidate data table can be determined according to the preset reference value of each candidate data table; a second weight corresponding to each candidate data table can be determined according to the usage frequency of each candidate data table in a target time period; a third weight corresponding to each candidate data table can be obtained by weighted summation of the first weight corresponding to each candidate data table and the second weight corresponding to each candidate data table; and a plurality of data tables can be selected from the candidate data tables of the database according to the third weight corresponding to each candidate data table, to obtain the group of data tables.

[0054] Optionally, determining the first weight corresponding to each candidate data table according to the preset reference value of each candidate data table can include determining the ratio of the preset reference value of each candidate data table to the sum of the preset reference values of all candidate data tables as the first weight corresponding to each candidate data table.

[0055] For example, the data tables can be scored according to business experience, and the score range can be [0, 100]. The first weight of each data table can be calculated according to formula (1), and the result range is [0, 1].

[0056]

[0057] wherein w1 i is the first weight of the i-th data table, s1 i is the score result of the i-th data table, and N (e.g., 10) is the total number of data tables.

[0058] Optionally, determining the second weight corresponding to each candidate data table according to the number of times each candidate data table is used in the target time period can include: determining, as the second weight corresponding to each candidate data table, the ratio of the number of times each candidate data table is used in the target time period to the sum of the number of times all candidate data tables are used in the target time period.

[0059] For example, the log of the database can be analyzed to count the number of times each data table is used in a month, and the second weight of each data table is calculated according to formula (2), and the result ranges from [0, 1]:

[0060]

[0061] wherein w2 i is the second weight of the i-th data table, s2 i is the number of times the i-th data table is used.

[0062] For N data tables, the scoring weight is W1 = [w11, w12, w13... w1 N ], the number of times the weight is called W2 = [w21, w22, w23... w2 N ], and then the final weight of each data table can be calculated by formula (3):

[0063] w3 i = (1 + a) * w1 i + a * w2 i (3)

[0064] wherein w3 i is the final weight (i.e., the third weight) of the i-th data table, and a = 0.8.

[0065] Optionally, selecting multiple data tables from the candidate data tables in the database according to the third weight corresponding to each candidate data table to obtain a set of data tables may include: sorting the set of candidate data tables in descending order according to the third weight corresponding to each candidate data table to obtain a sorted set of candidate data tables; and selecting a set of data tables from the set of candidate data tables that meet a data table selection condition according to the sorted set of candidate data tables. Here, the data table selection condition may be: selecting the first M data tables; and selecting data tables one by one according to the sorted set of candidate data tables until the sum of the third weight values ​​of the selected data tables reaches a preset weight threshold.

[0066] For example, N data tables can be sorted according to their final weights to obtain the sorting results of the N data tables, and the top M (for example, 6) data tables can be filtered out according to the sorting results of the N data tables, or data tables with a weight sum greater than 0.7 can be filtered out.

[0067] For example, Figure 3 As shown, the database contains 10 data tables, namely, T o1 ~T o10 ,By calculating the weight of each data table, we can screen out multiple core tables from the 10 data tables (i.e., determine the range of the core tables), i.e., T1 to T6.

[0068] The detection of data anomalies in the business data in the database can be performed according to a periodic detection time, or according to a pre-specified, non-periodic detection time, or can be performed at other forms of detection time. Optionally, a data table in a group of data tables can include a partition table, or can include a partition table, and the detection timing of anomaly detection can be determined based on the partition type of each data table in the group of data tables. For example, the data anomaly detection time corresponding to each data table can be determined based on the partition type of the data table, and when the data anomaly detection time corresponding to each data table arrives, a data anomaly detection is triggered.

[0069] It should be noted that a partition table is a table that divides a large data table into several small data tables. For example, Figure 4 As shown, data table T1 includes partition tables T1', T2', T3', and so on. Each small data table is used to record business data within a time period. For example, according to the partition type of the data table, the data table can be divided into hourly tables, daily tables, monthly tables, weekly tables, and non-partitioned tables. Taking the daily table as an example, each partition table in this data table is used to record business data within a day. Each data table has only one partition type, that is, a data table cannot be both a daily table and a weekly table at the same time.

[0070] It should be further noted that, since the detection time of data anomalies corresponding to different data tables can be different, if the detection time of data anomalies of a data table is reached, the data quality parameters of the updated data table updated from the last data anomaly detection to the current time can be recalculated, and the data quality parameters of the un-updated data table which has not been updated from the last data anomaly detection to the current time can be read to obtain the data quality parameters of the un-updated data table. Correspondingly, the time for determining the data quality parameters of each data table in the group of data tables can be the same or different.

[0071] Optionally, each data table in the database corresponds to a timing task, such as a timing update task, which can be uniquely identified by a task id. The evaluation process of the data quality parameters of the data table can be started only when it is monitored that the task of the last day is completed.

[0072] In order to effectively detect data anomalies of business data in the database, when determining the data quality parameters of each data table, the data missing rate of each data table and the data fluctuation parameter of each data table can be obtained respectively, where the data missing rate of each data table is used to represent the missing degree of the business data in each data table, and the data fluctuation parameter of each data table is used to represent the fluctuation degree of the business data in each data table over time.

[0073] In the embodiment, obtaining the data missing rate of each data table in the group of data tables can include: performing the following steps for each data table in the group of data tables to obtain the data missing rate of each data table, where, when performing the following steps, each data table is a current data table: determining a first detection period corresponding to the current data table; obtaining the data missing rate of the current data table in the first detection period to obtain the data missing rate of the current data table. The first detection period can be a pre-set detection period, for example, half a year, three months, one month, one week, etc. It can also be a detection period determined according to the partition type of each data table.

[0074] Optionally, different ways can be used to determine the data missing rate for non-partition tables and partition tables (for example, hour partition tables, day partition tables, month partition tables, etc.). Determining the first detection period corresponding to the current data table can include: when the current data table contains a partition table, determining the first detection period according to the partition period of the current data table, and the first detection period is a target multiple of the partition period. Correspondingly, the ratio of the number of abnormal partition tables in the first detection period to the total number of partition tables in the first detection period can be determined as the data missing rate of the current data table in the first detection period, thereby obtaining the data missing rate of the current data table.

[0075] Optionally, determining the first detection period according to the partition period of the current data table can comprise: in the case that the partition period of the current data table is greater than or equal to 1 day, determining the product of the first number and the partition period of the current data table as the first detection period; in the case that the partition period of the current data table is less than or equal to 1 day, determining the product of the first number and a reference period as the first detection period, wherein the reference period is 1 day.

[0076] For example, the detection period (also referred to as the statistical period) of the data missing rate of the data table can be determined, and the data of the last 7 (an example of the first number) periods can be counted. For the hour table, the data of 7 days*24 hours can be counted.

[0077] Optionally, the manner of determining the number of abnormal partition tables can be: determining the sum of the number of partition tables missing in the first detection period and the number of 0-value partition tables in the target monitoring as the number of abnormal partition tables in the first detection period.

[0078] For example, when calculating the data missing rate of the data table, the missing of partitions and the 0-value of partitions can be measured from two aspects. The statistical period can be calculated according to the business data of the last 7 periods to count the number of missing partitions and the number of 0-value partitions. The missing of partitions refers to the absence of a partition in the data table (for example, as shown in FIG. 2, the partition T2' is missing), and the 0-value of partitions refers to the presence of a partition in the data table but the business data in the partition is 0 (for example, as shown in FIG. 2, the business data in the partition T2' is 0). Figure 5 Figure 6 The manner of determining the data missing rate based on the number of missing partitions and the number of 0-value partitions can be as formula (4) as follows:

[0079]

[0080] wherein K i is the data missing rate of the i-th data table.

[0081] For the data table containing the hour table as the partition table, the data missing rate of the data table is: (7*24 periods of missing partitions+7*24 periods of 0-value partitions) / 7*24 periods of partition numbers;

[0082] For the data table containing the day table as the partition table, the data missing rate of the data table is: (7 periods of missing partitions+7 periods of 0-value partitions) / 7 periods of partition numbers;

[0083] For the data table containing the week table as the partition table, the data missing rate of the data table is: (7 periods of missing partitions+7 periods of 0-value partitions) / 7 periods of partition numbers;

[0084] ​For the data table containing the partition table of the month table, the data missing rate of the data table is: (7 period missing partition number+7 period 0 value partition number) / 7 period partition number.

[0085] Optionally, determining the first detection period corresponding to the current data table can comprise: when the current data table does not contain the partition table, determining the predetermined detection period as the first detection period, or determining the time period to which the service data in the current data table belongs as the first detection period. Correspondingly, when the first data table has a value in the first detection period, the data missing rate of the first data table in the first detection period is determined as the first missing rate (for example, 0); when the first data table has no value in the first detection period, the data missing rate of the first data table in the first detection period is determined as the second missing rate (for example, 1), so as to obtain the data missing rate of the current data table, where the first missing rate is less than the second missing rate.

[0086] For example, for a non-partition table, if there is service data (i.e., not empty) in the non-partition table, the data missing rate of the data table is determined as 0, otherwise, the data missing rate of the data table is determined as 1.

[0087] In the embodiment, obtaining the data fluctuation parameter of each data table in the group of data tables can be: performing the following steps on each data table in the group of data tables to obtain the data fluctuation parameter of each data table, wherein, when performing the following steps, each data table is a current data table: determining a second detection period corresponding to the current data table; determining the degree of fluctuation of the data amount of the service data in each sub-time period within the second detection period of the current data table as the data fluctuation parameter of the current data table. The second detection period and the first detection period can be the same detection period or different detection periods. The second detection period of each partition table is pre-set or determined according to the partition type of each data table. This is not limited in the embodiment.

[0088] Optionally, determining the degree of fluctuation of the data amount of the service data in each sub-time period within the second detection period of the current data table as the data fluctuation parameter of the current data table can comprise: determining the fluctuation degree of the data amount of the service data in the last sub-time period of the second detection period relative to the data amount of the service data in other sub-time periods except the last sub-time period in the second detection period as the data fluctuation parameter of the current data table.

[0089] Step S204, determining the data quality parameter of each data table in the group of data tables according to the data missing rate of each data table in the group of data tables and the data fluctuation parameter of each data table in the group of data tables.

[0090] In order to overcome the problem of imperfect dimension in measuring the integrity of a single data table only by the data missing rate, in the embodiment, the integrity (i.e., the data quality parameter) of a single data table can be measured from two aspects of the data missing rate and volatility (i.e., the data volatility parameter). For each data table, the data quality parameter of each data table can be determined according to the data missing rate of each data table and the data volatility parameter of each data table.

[0091] In determining the data quality parameter of each data table, the data quality parameter of each data table can be obtained by weighted summation of the data missing rate of each data table and the data volatility parameter of each data table, and the weight values of the data missing rate of each data table and the data volatility parameter of each data table can be the same or different, which is not limited in the embodiment.

[0092] For example, for the i-th data table, the data quality parameter Integrity i of the data table can be calculated as shown in formula (5):

[0093] Integrity i = (1 + β) * K i + β * L i (5)

[0094] wherein β = 0.8, K i is the data missing rate of the i-th data table, and L i is the data volatility parameter of the i-th data table.

[0095] In step S206, the weight value of each data table in the group of data tables is determined according to the operation information of the multi-dimension operation performed on each data table in the group of data tables, wherein the weight value of each data table in the group of data tables is used to represent the importance degree of each data table.

[0096] In the related art, the integrity of each data table is measured from a single dimension of the data missing rate, and the weight of each data table is the same, and the integrity measurement of each data table is added to obtain the integrity measurement of the entire library table. However, the above setting method of the weight of the data table does not consider the importance degree of the actual business meaning of the data table, resulting in unreasonable integrity measurement of the entire library table.

[0097] In the embodiment, the corresponding weight value of each data table is set to represent the importance degree of each data table, so as to improve the rationality of the integrity measurement of the entire library table. Here, as mentioned above, since the data volatility parameter is considered in measuring the integrity of the data table, the data quality parameter of the entire library table is measured in the embodiment.

[0098] In determining the weight value of each data table, the weight value of each data table can be determined according to operation information of the multi-dimensional operation performed on each data table, which can include but is not limited to at least one of operation information of a use operation and operation information of an update operation. Based on the operation information of the multi-dimensional operation performed on each data table, the importance of each data table can be represented, for example, the more the number of uses, the higher the importance, the more the number of updates, the higher the importance, and the like. Determining the weight value of each data table based on the operation information of the multi-dimensional operation can improve the rationality of the weight setting.

[0099] In step S208, in a case where the data quality parameter of each data table in the group of data tables and the weight value of each data table in the group of data tables satisfy a preset abnormal condition, it is determined that the business data in the group of data tables is abnormal.

[0100] After obtaining the data quality parameter of each data table and the weight value of each data table, whether the group of data tables satisfies a preset abnormal condition can be determined based on the data quality parameter of each data table and the weight value of each data table, so as to determine whether the business data in the group of data tables is abnormal. Here, the abnormality can be caused by a problem in the business data, or can be caused by a normal change in the business data.

[0101] The preset abnormal condition can be an abnormal condition for a single data table, for example, a data missing rate is greater than or equal to a preset missing rate threshold, a data fluctuation parameter satisfies a preset abnormal fluctuation condition (for example, the parameter value is 1), a data quality parameter of a data table is greater than or equal to a preset quality parameter threshold (in this case, the data quality parameter is negatively correlated with the data quality) or less than or equal to a preset quality parameter threshold (in this case, the data quality parameter is positively correlated with the data quality). Whether the business data of a certain data table or certain data tables in the group of data tables is abnormal can be determined based on whether a certain data table satisfies the preset abnormal condition.

[0102] The preset abnormal condition can be an abnormal condition for a group of data tables, for example, the data quality parameter of the group of data tables can be determined based on the data quality parameter of each data table and the weight value of each data table, and then whether the business data in the group of data tables is abnormal as a whole can be determined based on whether the data quality parameter of the group of data tables satisfies a preset abnormal condition.

[0103] Optionally, the target detection result of the set of data tables can be generated according to the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables. The target detection result can include at least one of the following: the data missing rate of each data table, the abnormal partition table in each data table, the data fluctuation parameter of each data table, the data quality parameter of each data table, the data quality parameter of the set of data tables, and identification information indicating whether the business data in the set of data tables is abnormal. The target detection result can be sent to a target device for display, or saved to a storage component. This is not limited in the embodiment.

[0104] According to the embodiments provided in the present application, the data missing rate of each data table in a set of data tables is obtained, and the data fluctuation parameter of each data table in the set of data tables is obtained, wherein the set of data tables includes business data generated by a target application during running, and the data fluctuation parameter of each data table is used to represent the degree of fluctuation of the business data in the data table over time.

[0105] According to the data missing rate of each data table in a set of data tables and the data fluctuation parameter of each data table in the set of data tables, the data quality parameter of each data table in the set of data tables is determined; according to the operation information of the multi-dimensional operation performed on each data table in the set of data tables, the weight value of each data table in the set of data tables is determined, wherein the weight value of each data table in the set of data tables is used to represent the importance of each data table; in the case that the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables satisfy a preset abnormal condition, it is determined that the business data in the set of data tables is abnormal, which solves the problem of low accuracy of data abnormality recognition in the related art by using the data missing rate to identify data abnormality in the data table, and improves the accuracy of data abnormality recognition.

[0106] As an optional solution, the data fluctuation parameter of each data table in a set of data tables includes:

[0107] S11, the following steps are performed for each data table in a set of data tables, wherein each data table is a current data table when the following steps are performed:

[0108] The number of data items of the business data generated in each preset period in a set of preset periods in the current data table is determined, and a data item number sequence is obtained;

[0109] According to the fluctuation of the current data item number relative to a set of historical data item numbers, the data fluctuation parameter of the current data table is determined.

[0110] Optionally, in the embodiment, when determining the data fluctuation parameter of each data table, each data table can be processed as a current data table respectively. For the current data table, the number of data items of the service data generated in each preset period in a group of preset periods can be determined, and a sequence of the number of data items is obtained, that is, a group of the number of data items.

[0111] The preset period can be a period corresponding to the partition type of the data table. For the data table containing a partition table, the partition period corresponding to the partition table can be determined as the preset period. For example, as shown in the following table, N1' and N3' are the number of data items of the service data in a certain preset period. Figure 6 For the data table not containing a partition table, the preset period can be determined based on a set period. For example, for a non-partition table, the preset period can be 1 day.

[0112] The sequence of the number of data items can include a current number of data items, that is, the number of data items of the service data generated in the last preset period in a group of preset periods in the sequence of the number of data items, that is, the number of data items of the service data generated in the nearest preset period. The number of data items other than the current number of data items in the sequence of the number of data items can be a group of historical numbers of data items.

[0113] Based on the fluctuation of the current number of data items relative to the group of historical numbers of data items, the data fluctuation parameter of the current data table can be determined. When determining the data fluctuation parameter of the current data table, the fluctuation range of the number of data items allowed by the current number of data items can be determined based on the group of historical numbers of data items, that is, if the current number of data items belongs to the determined fluctuation range of the number of data items, it can be determined that the current number of data items fluctuates normally, that is, the service data generated in the last preset period in the current data table fluctuates normally, otherwise, it is determined that the current number of data items fluctuates abnormally, that is, the service data generated in the last preset period in the current data table fluctuates abnormally.

[0114] Through the embodiments provided in the present application, the data fluctuation parameter of the data table is determined based on the fluctuation of the number of data items of the service data with time, which can improve the convenience of determining the data fluctuation parameter of the data table.

[0115] As an optional solution, the data fluctuation parameter of the current data table is determined according to the fluctuation of the current number of data items relative to the group of historical numbers of data items, comprising:

[0116] S21, in the case that the sequence of the number of data items is a stationary sequence, determining the average number of data items of the group of historical numbers of data items;

[0117] S22, in a case where the difference between the current data item number and the average item number is greater than the preset difference threshold, determining the data fluctuation parameter of the current data table as a first parameter value, wherein the first parameter value is used to represent a business data fluctuation abnormality in a last preset period in the current data table.

[0118] For a data table, after a corresponding data item number sequence is determined, a data item number fluctuation interval allowed for the current data item number can be determined based on a set of historical data item numbers. The interval center and the interval radius of the data item number fluctuation interval can be determined. If the current data item number does not belong to the data item number fluctuation interval, the data fluctuation parameter of the current data table is determined as a first parameter value, which is used to represent a business data fluctuation abnormality in a last preset period in the current data table; otherwise, the data fluctuation parameter of the current data table is determined as a second parameter value, which is used to represent a business data fluctuation normality in a last preset period in the current data table.

[0119] The interval center and the interval radius can be determined in one or more ways, which can include but are not limited to at least one of the following:

[0120] In a first way, the average item number of the set of historical data item numbers is determined as the interval center, and the product of the average item number and a first preset coefficient (for example, 30%, 20%, etc.) is determined as the interval radius.

[0121] In determining the interval center, the ratio of the set of historical data item numbers to the number of data item numbers contained in the set of historical data item numbers can be determined as the average item number of the set of historical data item numbers.

[0122] In a second way, the weighted average item number (which can be a weighted moving average item number, such as an exponential moving weighted average item number) of the set of historical data item numbers is determined as the interval center, and the product of the standard deviation (i.e., σ) of the set of historical data item numbers and a second preset coefficient (for example, 3) is determined as the interval radius.

[0123] In determining the interval center, in determining the weighted average item number, different weights can be set for the set of historical data item numbers, and the size of the weight of the data item number is positively correlated with the distance between the data item number and the current data item number.

[0124] Taking the number of EWMA (Exponentially Weighted Moving-Average) items as an example, the exponentially moving weighted average method refers to that the weighted coefficients of each value decrease exponentially with time, and the weighted coefficient of the value closer to the current time is larger. Considering that the recent observation value (i.e., the data item number close to the current data item number in the group of historical data item numbers) in the observation period (i.e., a group of preset periods) has a greater influence on the predicted value (the current data item number), the weighted moving average method can reflect the trend of recent changes.

[0125] The expression of EWMA is shown in formula (6):

[0126] v t =λ*v t-1 +(1-λ)*θ t (6)

[0127] wherein, θ t is the actual value at time t, the coefficient λ represents the rate of weighted decrease, the smaller the value is, the faster the decrease is, and v t is the value of EWMA at time t.

[0128] Here, each time can correspond to a preset period, and the actual value at each time is the corresponding data item number in a certain preset period, and θ t is the current data item number. Through formula (6), the exponentially moving weighted average number of a group of historical data item numbers can be determined.

[0129] When determining the interval radius, the standard deviation σ of a group of historical data item numbers can be determined first, and then 3σ is determined as the interval radius. σ is a quantitative form reflecting the dispersion degree of a group of data, and the formula for calculating σ is shown in formula (7):

[0130]

[0131] wherein, n is the number of data item numbers contained in a group of historical data item numbers, x i is the i-th data item number, is the average number of a group of historical data item numbers.

[0132] Here, in addition to 3σ, the product of σ and other coefficients can also be determined as the interval radius. Considering that the probability of the value distribution in (μ-3σ, μ+3σ) is 0.9973, it can be considered that the value of the data item number is almost entirely concentrated in the interval (μ-3σ, μ+3σ), and the possibility of exceeding this range is less than 0.3%, where μ is the interval center. Therefore, the interval radius can be set as 3σ, and it can be considered that the business data has an abnormal fluctuation when exceeding (μ-3σ, μ+3σ).

[0133] Based on the fluctuation degree of the business data, the sequence of the number of data items can be divided into a stationary sequence and a non-stationary sequence. The stationary sequence refers to a sequence in which the fluctuation amplitude of the data in the sequence meets a preset amplitude condition, and the non-stationary sequence refers to a sequence in which the fluctuation amplitude of the data in the sequence does not meet the preset amplitude condition. The sequence of the number of data items can be a stationary sequence or a non-stationary sequence. Whether the sequence of the number of data items is a stationary sequence can be preset based on the stored business data, or can be determined according to the fluctuation of the number of data items in the sequence of the number of data items, which is not limited in the embodiment.

[0134] If the sequence of the number of data items is a stationary sequence, the number of item fluctuations can be determined in the above manner one, and the interval radius can be a preset difference threshold value, for example, the product of the average number of items and 30%. If the number of item difference between the current number of data items and the average number of items is greater than the preset difference threshold value, it can be considered that the current number of data items is located outside the number of item fluctuation interval, and the data fluctuation parameter of the current data table is determined as the first parameter value.

[0135] If the number of item difference between the current number of data items and the average number of items is less than or equal to the preset difference threshold value, it can be considered that the current number of data items is located within the number of item fluctuation interval, the data fluctuation parameter of the current data table can be directly determined as the second parameter value, or the number of item fluctuation interval can be re-determined based on manner two, and the data fluctuation parameter of the current data table can be determined based on whether the current number of data items is located within the number of item fluctuation interval, or the number of item fluctuation interval can be re-determined in other manners, which is not limited in the embodiment.

[0136] Through the embodiments provided in the application, for the stationary sequence, the number of item fluctuation interval corresponding to the current number of data items is determined based on the average number of items, which can improve the convenience of determining the data fluctuation parameter of the data table.

[0137] As an optional solution, the data fluctuation parameter of the current data table is determined according to the fluctuation of the current number of data items relative to a group of historical numbers of data items, and further comprising:

[0138] S31, in the case where the number of item difference between the current number of data items and the average number of items is less than or equal to the preset difference threshold value, detecting whether the current number of data items is located within the first number of item reference range, wherein the first number of item reference range is a normal fluctuation range allowed for the current number of data items, which is determined according to the weighted average number of items of a group of historical numbers of data items;

[0139] S32, in the case where the current number of data items is located within the first number of item reference range, determining the data fluctuation parameter of the current data table as the second parameter value, wherein the second parameter value is different from the first parameter value;

[0140] S33, in a case where the current data item number is located outside the first item number reference range, determining that the data fluctuation parameter of the current data table is a first parameter value.

[0141] If the item number difference between the current data item number and the average item number is less than or equal to the preset difference threshold, the normal fluctuation range of the current data item number can be re-determined according to the weighted average item number of the set of historical data item numbers, to obtain the first item number reference range (i.e., the item number fluctuation interval). The manner can be manner two, or other manners except manner two, which is not limited in the embodiment.

[0142] If the current data item number is located within the first item number reference range, it is determined that the data fluctuation parameter of the current data table is a second parameter value, and if the current data item number is located outside the first item number reference range, it is determined that the data fluctuation parameter of the current data table is a first parameter value. The first parameter value and the second parameter value are similar to the foregoing embodiment, and will not be described here.

[0143] For example, as shown in FIG. 6, it is judged whether the data item number sequence (time series data) is a stationary sequence. If it is a stationary sequence, the mean value of the historical data is calculated, that is, the mean value of the set of historical data item numbers (i.e., the average item number). The difference between the current data item number and the mean value is compared with the mean value. If the difference between the current data item number and the mean value is greater than 30% of the mean value, an alarm is directly generated. If the difference between the current data item number and the mean value is not greater than 30% of the mean value, the ewma value and the 3σ value are calculated. If the current data item number is in the interval [ewma-3σ, ewma+3σ], no alarm is generated; otherwise, an alarm is generated. Figure 7 Here, if an alarm is generated, the data fluctuation parameter of the data table is 1 (i.e., the first parameter value), and if no alarm is generated, the data fluctuation parameter of the data table is 0 (i.e., the second parameter value).

[0144] Through the embodiments provided in the present application, the item number fluctuation interval corresponding to the current data item number is determined based on the average item number and the weighted average item number for the stationary sequence, respectively, which can improve the accuracy of determining the data fluctuation parameter of the data table.

[0145] As an optional solution, the data fluctuation parameter of the current data table is determined according to the fluctuation of the current data item number relative to the set of historical data item numbers, comprising:

[0146] S41, in a case where the data item number sequence is a non-stationary sequence and a periodic sequence, and a set of preset periods includes a plurality of preset period sets and a current preset period set, determining, from the set of historical data item numbers, a target data item number corresponding to the current data item number in each preset period set of the plurality of preset period sets, to obtain a set of target data item numbers;

[0147] ​

[0148] S42, detecting whether the current data item number is located in a second item number reference range, wherein the second item number reference range is a normal fluctuation range of the current data item number determined according to a weighted average item number of a group of target data item numbers;

[0149] S43, in a case where the current data item number is located outside the second item number reference range, determining that a data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to represent that the service data fluctuation in the last preset period in the current data table is abnormal;

[0150] S44, in a case where the current data item number is located in the second item number reference range, determining that the data fluctuation parameter of the current data table is a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to represent that the service data fluctuation in the last preset period in the current data table is normal.

[0151] If the data item number sequence is a non-stationary sequence, it can be further determined whether the data item number sequence is a periodic sequence or a non-periodic sequence. The periodic sequence refers to a sequence in which the data fluctuates periodically, and the non-periodic sequence refers to a sequence in which the data fluctuates irregularly. The data item number sequence can be a periodic sequence or a non-periodic sequence. Whether the data item number sequence is a periodic sequence or a non-periodic sequence can be pre-set based on the stored service data, or can be determined according to the fluctuation of the data item number of the data item number sequence, which is not limited in the embodiment.

[0152] If the data item number sequence is a non-stationary sequence and a periodic sequence, and a group of preset periods includes a plurality of preset period sets and a current preset period set, a target data item number corresponding to the current data item number in each preset period set of the plurality of preset period sets can be determined from a group of historical data item numbers, to obtain a group of target data item numbers. Here, the current data item number is the data item number of the service data generated in the current preset period (i.e., the last preset period) in the current preset period set, and the position of each target data item number in the corresponding preset period set is the same as the position of the current preset period in the current preset period set.

[0153] For example, one preset period is one day, one preset period set is one week, a group of preset periods includes the previous four weeks and the current week, and the current preset period is Wednesday. The data item number of each Wednesday in the previous four weeks can be determined to obtain a group of target data item numbers, which includes four data item numbers.

[0154] After obtaining the target data item number set, the first or second manner described above can be used to determine the item number fluctuation interval, and obtain the second item number reference range. If the current data item number is outside the second item number reference range, the data fluctuation parameter of the current data table is the first parameter value, and if the current data item number is within the second item number reference range, the data fluctuation parameter of the current data table is the second parameter value.

[0155] For example, as shown in FIG. 3, if the data item number sequence is a non-stationary sequence, it can be determined whether the sequence is a periodic sequence. If the sequence is a periodic sequence, the preset period corresponding to the current data item number can be determined as a day of the week, and the data item number of the day of the week is read from a set of historical data item numbers as a reference time sequence (i.e., a set of target data item numbers). At the same time, according to the new time sequence, the ewma value and the 3σ value are calculated, and if the current data item number is in the [ewma-3σ, ewma+3σ] interval, no alarm is generated; otherwise, an alarm is generated. Figure 7

[0156] Through the embodiments provided in the present application, for a periodic sequence, the item number fluctuation interval corresponding to the current data item number is determined by reading the data item number at the same position of other periods, which can improve the accuracy of determining the data fluctuation parameter of the data table.

[0157] As an optional solution, the data fluctuation parameter of the current data table is determined according to the fluctuation of the current data item number relative to a set of historical data item numbers, including:

[0158] S51, in the case that the data item number sequence is a non-stationary sequence and a non-periodic sequence, detecting whether the current data item number is within a third item number reference range, wherein the third item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of a set of historical data item numbers;

[0159] S52, in the case that the current data item number is outside the third item number reference range, determining that the data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to indicate that the business data fluctuation in the last preset period in the current data table is abnormal;

[0160] S53, in the case that the current data item number is within the third item number reference range, determining that the data fluctuation parameter of the current data table is a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to indicate that the business data fluctuation in the last preset period in the current data table is normal.

[0161] ​If the data item number sequence is neither a stationary sequence nor a periodic sequence, the first mode or the second mode described above can be used to determine the item number fluctuation interval, and a third item number reference range is obtained. If the current data item number is outside the third item number reference range, the data fluctuation parameter of the current data table is the first parameter value, and if the current data item number is within the third item number reference range, the data fluctuation parameter of the current data table is the second parameter value.

[0162] For example, as shown in Figure 7 If the data item number sequence is neither stationary nor periodic, the ewma value of a group of historical data item numbers and the value of 3σ can be directly calculated, and if the current data item number is in the interval [ewma-3σ, ewma+3σ], no alarm is generated; otherwise, an alarm is generated.

[0163] Through the embodiments provided in the present application, for a data item number sequence that is neither stationary nor periodic, the weighted average value of the data item number is used to determine the item number fluctuation interval corresponding to the current data item number, which can improve the accuracy of determining the data fluctuation parameter of the data table.

[0164] As an optional solution, the above method further includes:

[0165] S61, performing a fluctuation coefficient extraction operation on the data item number sequence to obtain a group of fluctuation coefficients;

[0166] S62, in a case where the ratio of the standard deviation of the data item number sequence to the standard deviation of the group of fluctuation coefficients is less than or equal to a preset ratio, determining that the data item number sequence is a stationary sequence;

[0167] S63, in a case where the ratio of the standard deviation of the data item number sequence to the standard deviation of the group of fluctuation coefficients is less than the preset ratio, determining that the data item number sequence is a non-stationary sequence.

[0168] In the present embodiment, whether the data item number sequence is a stationary sequence can be determined based on the fluctuation coefficient of the data item number sequence. The fluctuation coefficient extraction operation can be performed on the data item number sequence first to obtain a group of fluctuation coefficients, and the fluctuation coefficient is used to represent the fluctuation degree of the data item number, which can be high-frequency information extracted.

[0169] Taking wavelet transform as an example, wavelet transform is used to represent a function or a signal with a group of wavelet functions and basis functions. After wavelet transform, the data generates low-frequency information and high-frequency information, as shown in Figure 8 The low-frequency information (A in Figure 8 corresponds to the mean value, and the mean value is the local average value, which changes slowly, while the high-frequency information Figure 8D) in the DWT corresponds to the difference value, which is a local fluctuation value and changes rapidly.

[0170] After obtaining the set of fluctuation coefficients, the standard deviation of the data item number sequence and the standard deviation of the set of fluctuation coefficients can be calculated respectively, and the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients can be determined. If the determined ratio is less than or equal to a preset ratio, it is determined that the data item number sequence is a stationary sequence, otherwise, it is determined that the data item number sequence is a non-stationary sequence.

[0171] For example, as shown in FIG. 8, the process of determining whether the data item number sequence is stationary can include the following steps: Figure 9

[0172] Step S902, obtaining the data item number sequence;

[0173] Step S904, performing wavelet decomposition (for example, by wavelet transform) on the data item number sequence;

[0174] Step S906, extracting high-frequency coefficients of the data item number sequence;

[0175] Step S908, calculating the standard deviation of the high-frequency coefficients to obtain st1, which can measure the local fluctuation information of the data item number sequence;

[0176] Step S910, calculating the standard deviation of the data item number sequence (i.e., the original time series data) to obtain st2, which can measure the global fluctuation information of the data item number sequence;

[0177] Step S912, determining whether st2 / st1 is less than a set threshold value. If yes, it is determined that the data item number sequence is stationary, otherwise, it is determined that the data item number sequence is non-stationary.

[0178] Through the embodiments provided in the present application, by extracting the fluctuation coefficients of the data item number sequence and determining whether the data item number sequence is a stationary sequence according to the extracted set of fluctuation coefficients, the rationality of the stationary sequence determination can be improved.

[0179] As an optional solution, the above method further includes:

[0180] S71, in the data item number sequence, obtaining the data item number corresponding to each preset period in the first preset period set to obtain a first reference item number sequence, and obtaining the data item number corresponding to each preset period in the second preset period set to obtain a second reference item number sequence, wherein the first preset period set and the second preset period set are adjacent preset period sets in a set of preset periods;

[0181] ​S72, determine the difference of the data item numbers in each same position of the first reference item number sequence and the second reference item number sequence, to obtain a target item number difference sequence;

[0182] S73, in the case that the standard deviation of the target item number difference sequence is less than or equal to a preset standard deviation threshold, determine that the data item number sequence is a periodic sequence.

[0183] S74, in the case that the standard deviation of the target item number difference sequence is greater than the preset standard deviation threshold, determine that the data item number sequence is a non-periodic sequence.

[0184] In the determination of whether the data item number sequence is periodic, two adjacent preset period sets, i.e., a first preset period set and a second preset period set, can be determined first, and then in the data item number sequence, the data item numbers corresponding to each preset period in the first preset period set and the data item numbers corresponding to each preset period in the second preset period set are obtained respectively, to obtain the first reference item number sequence and the second reference item number sequence.

[0185] After obtaining the first reference item number sequence and the second reference item number sequence, the data item numbers in the corresponding positions of the two sequences can be subtracted, i.e., the difference of the data item numbers in each same position of the first reference item number sequence and the second reference item number sequence is determined, to obtain a target item number difference sequence, and the standard deviation of the target item number difference sequence is determined.

[0186] If the standard deviation of the target item number difference sequence is less than or equal to a preset standard deviation threshold (for example, 1), it can be determined that the data item number sequence is a periodic sequence; if the standard deviation of the target item number difference sequence is greater than the preset standard deviation threshold, it is determined that the data item number sequence is a non-periodic sequence.

[0187] For example, the data (i.e., the data item number) of the last 2 periods can be taken: for the hour table, the data of 48 hours in 2 days is taken, and for the day table, the month table, the non-partition table and the week table, the data of 14 periods is taken, and the data of 7 periods can be taken as 1 large period, and the data of 2 large periods is obtained in total.

[0188] The difference of the data in the corresponding positions of the 2 periods is obtained to obtain a group of difference values; the standard deviation of the group of difference values is obtained, and the obtained standard deviation and 1 are compared, if the standard deviation is less than 1, it is a periodic sequence, otherwise, it is a non-periodic sequence.

[0189] Through the embodiments provided in the present application, whether the data item number sequence is a periodic sequence is determined based on the data item numbers corresponding to the two adjacent preset period sets, which can improve the convenience of the determination of the periodic sequence.

[0190] As an optional solution, the weight value of each data table in the group of data tables is determined according to operation information of the multi-dimensional operation performed on each data table in the group of data tables, including:

[0191] S81, the weight value of each dimensional operation in the multi-dimensional operation is determined according to the operation number of the multi-dimensional operation performed on each data table in the group of data tables;

[0192] S82, the reference evaluation value of each data table is determined according to the operation number of the multi-dimensional operation performed on each data table and the weight value of each dimensional operation;

[0193] S83, the weight value of each data table is determined according to the reference evaluation value of each data table.

[0194] Since the operation number of each data table in each dimensional operation is different, the operation number of different data tables in the same dimensional operation will also be different, so the importance of different dimensional operations in determining the weight value of different data tables will also be different. In this embodiment, in order to determine the weight value of each data table, the weight value of each dimensional operation can be determined first.

[0195] The way to determine the weight value of each dimensional operation can be one or more, for example, the weight value of each dimensional operation can be pre-configured, or the weight value of each dimensional operation can be obtained by evaluating the operation number of the multi-dimensional operation performed on each data table. For example, for each dimensional operation, the weight value of the dimensional operation can be determined based on the change of the operation number of different data tables in the same dimensional operation. The minimum operation number and the maximum operation number of a group of data tables in a certain dimensional operation can be taken as a standard, and the weight value of the dimensional operation can be determined according to the change of the operation number of different data tables in the dimensional operation.

[0196] After obtaining the weight value of each dimensional operation, for each data table, the reference evaluation value of the data table can be determined according to the operation number of the multi-dimensional operation performed on the data table and the weight value of each dimensional operation. Here, the sum operation can be performed on the product of the operation number of each dimensional operation performed on the data table and the weight value of each dimensional operation, that is, the sum operation is performed on the operation number of the multi-dimensional operation performed on the data table based on the weight value of each dimensional operation, and the reference evaluation value of the data table is obtained.

[0197] For example, in order to determine the importance degree of 6 data tables [T1, T2, …, T6], the reference evaluation value of each data table can be determined based on the operation number of each data table in four dimensions (i.e., four dimensional operations) of the number of visits in the past 90 days, the number of people using the table in the past 90 days, the number of timing tasks using the table in the past 90 days, and the number of automatic updates in the past 90 days, wherein the operation number of the 6 data tables in the four dimensions is shown in Table 1:

[0198] Table 1

[0199]

[0200] Based on Table 1, the weight of each dimension can be determined, as shown in Table 2:

[0201] Table 2

[0202]

[0203] According to the number of operations of each data table in each dimension and the weight of each dimension, the reference evaluation value of each data table can be determined according to formula (8):

[0204]

[0205] wherein S i is the reference evaluation value of the i th data table, w4 j is the weight of the j th dimension, L is the total number of dimensions (for example, 4), X ij is the number of operations of the i th data table in the j th dimension. The determined reference evaluation value of each data table is shown in Table 3:

[0206] Table 3

[0207] Table 1 T1 T2 T3 T4 T5 T6 Reference evaluation value 83 61.7 32.85 30.7 59.3 22.8

[0208] After obtaining the reference evaluation value of each data table, the weight value of each data table can be determined according to the reference evaluation value of each data table. For example, the weight value of each data table can be determined according to the ratio of the reference evaluation value of each data table to the sum of the reference weight values of all data tables in the group of data tables. For another example, the weight value of each data table can be determined according to the reference evaluation value of each data table and the preset evaluation value or the preset weight value of each data table. The manner of determining the weight value of each data table is not limited in the embodiment.

[0209] Through the embodiments provided in the present application, by determining the weight value of each dimension operation first, and then determining the weight value of each data table based on the weight value of each dimension operation, the accuracy of determining the weight value of the data table can be improved.

[0210] As an optional solution, the weight value of each dimension operation in the multi-dimensional operation is determined according to the number of operations of the multi-dimensional operation performed on each data table in the group of data tables, comprising:

[0211] S91, determining the weight value of each dimension operation in the multi-dimensional operation under each data table according to the number of operations of the multi-dimensional operation performed on each data table in the group of data tables;

[0212] S92, determine the weight value of each dimension operation according to the weight value of each dimension operation under each data table.

[0213] In determining the weight value of each dimension operation, the weight value of the current dimension operation under each data table can be determined according to the operation number of the current dimension operation performed on each data table, in the unit of dimension. The weight value of the current dimension operation under each data table is positively correlated with the operation number of the current dimension operation performed on each data table, that is, the more the operation number of the current dimension operation performed on each data table, the greater the weight value of the current dimension operation under each data table.

[0214] According to the difference degree of the weight value of a dimension operation under different data tables, the influence of the dimension operation on the importance of determining the data table can be determined. Therefore, the weight value of each dimension operation can be determined according to the weight value of each dimension operation under each data table. For example, the operation reference value corresponding to each dimension operation can be determined according to the weight value of each dimension operation under each data table, and the operation reference value is used to represent the difference degree of the weight value of each dimension operation under each data table.

[0215] Optionally, the weight value of each dimension operation can be determined according to the information entropy of the weight value of each dimension operation under each data table. The information entropy is a measure of uncertainty. The greater the uncertainty, the greater the entropy, and the greater the amount of information contained. The smaller the uncertainty, the smaller the entropy, and the smaller the amount of information contained. Based on the information entropy, the objective weight value of each dimension operation can be determined according to the variability of the weight value of each dimension operation under different data tables.

[0216] For example, the ratio (i.e., the weight value) of each dimension under each data table is p ij , which represents the weight value of the jth dimension under the ith data table. In determining the weight value of each dimension, the information entropy of each dimension can be determined first, and the calculation method is shown in formula (9):

[0217]

[0218] Where E j is the information entropy of the jth dimension.

[0219] After determining the information entropy of each dimension, the information redundancy of each dimension can be determined, as shown in formula (10):

[0220] D j = 1-E j (10)

[0221] Where D j is the information redundancy of the jth dimension.

[0222] Correspondingly, the information redundancy of each dimension can be determined as the weight value of each dimension, and the information redundancy of multiple dimensions can be normalized according to formula (11) to obtain the weight value of each dimension.

[0223]

[0224] wherein, w4 j is the weight value of the jth dimension, and L is the total number of dimensions.

[0225] According to the embodiments provided in the present application, the weight value of each dimension operation under each data table is determined first, and the weight value of each dimension operation is determined based on the weight value of each dimension operation under each data table, which can provide the accuracy of the determination of the weight value of each dimension operation.

[0226] As an optional solution, the weight value of each dimension operation under each data table in the multi-dimensional operation is determined according to the operation number of the multi-dimensional operation performed on each data table in the group of data tables, comprising:

[0227] S101, performing a standardization operation on the operation number of each data table in the group of data tables under each dimension operation in the multi-dimensional operation to obtain a first reference value of each data table under each dimension operation;

[0228] S103, determining the ratio of the first reference value of each data table under each dimension operation to the second reference value as the weight value of each dimension operation under each data table, wherein the second reference value is the sum of the first reference value of each data table under each dimension operation.

[0229] In the present embodiment, when determining the weight value of each dimension operation under each data table, the standardization operation can be first performed on the operation number of each data table in the group of data tables under each dimension operation to obtain the first reference value of each data table under each dimension operation. When determining the first reference value of each data table under each dimension operation: the minimum operation number and the maximum operation number in the operation number of the group of data tables under each dimension operation can be determined first to obtain the first operation number and the second operation number; then the difference between the operation number of each data table under each dimension operation and the first operation number is determined to obtain the first difference value, and the difference between the second operation number and the first operation number is determined to obtain the second difference value, and the ratio of the first difference value to the second difference value is determined as the first reference value of each data table under each dimension operation.

[0230] Based on the first reference value of each data table under each dimension operation, the weight value of each dimension operation under each data table can be determined. For example, normalization operation can be performed on the first reference value of each data table under each dimension operation to obtain the weight value of each dimension operation under each data table. First, the sum of the first reference value of each data table under each dimension operation can be determined to obtain a second reference value, and then the ratio of the first reference value of each data table under each dimension operation to the second reference value is determined as the weight value of each dimension operation under each data table.

[0231] For example, the standardization operation can be performed on the operation number of the data table under each dimension according to formula (12):

[0232]

[0233] wherein Y ij is the first reference value of the i th data table under the j th dimension, max(X j ) is the maximum operation number of the data table under the j th dimension, and min(X j ) is the minimum operation number of the data table under the j th dimension.

[0234] After performing data standardization, the first reference value of each data table under each dimension obtained is shown in Table 4:

[0235] Table 4

[0236]

[0237] After performing the standardization operation, the ratio (i.e., the weight value) of each dimension under each data table can be calculated according to formula (13):

[0238]

[0239] The ratio of each dimension under each data table can be shown in Table 5:

[0240] Table 5

[0241]

[0242] According to the ratio of each dimension under each data table, the information entropy of each dimension can be determined according to formula (9), and the determined information entropy of each dimension can be shown in Table 6:

[0243] Table 6

[0244]

[0245] According to formula (10) and formula (11), the weight value of each dimension can be obtained by processing the information entropy of each dimension, which is shown in Table 2.

[0246] By the embodiments provided in the present application, by performing a standardization operation on the operation number of each data table under each dimension operation, and then determining the weight value of each dimension operation under each data table based on the result of the standardization operation, the accuracy and convenience of determining the weight value of each dimension operation under each data table can be improved.

[0247] As an optional solution, the weight value of each data table is determined according to the reference evaluation value of each data table, comprising:

[0248] S111, the ratio of the reference evaluation value of each data table to the reference weight value of a group of data tables is determined as the weight value of each data table, wherein the reference weight value of the group of data tables is the sum of the reference evaluation value of each data table; or

[0249] The reference evaluation value of each data table and the preset evaluation value of each data table are weighted and summed to obtain an updated reference evaluation value of each data table; and the ratio of the updated reference evaluation value of each data table to the updated reference weight value of a group of data tables is determined as the weight value of each data table, wherein the updated reference weight value of the group of data tables is the sum of the updated reference evaluation value of each data table.

[0250] In the present embodiment, the reference evaluation value of each data table can be directly used to determine the weight value of each data table. For example, the sum operation can be performed on the reference evaluation value of each data table to obtain the reference weight value of a group of data tables, and the ratio of the reference evaluation value of each data table to the reference weight value of the group of data tables is determined as the weight value of each data table.

[0251] For example, the reference evaluation value of the i th data table is S i The weight value of each data table can be calculated by formula (14):

[0252]

[0253] Alternatively, only the operation number of the multi-dimensional operation performed on each data table is used to determine the weight value of each data table, which may result in inaccurate determination of the weight value of the data table due to the neglect of the importance of the business data in the data table. To this end, a preset evaluation value can be set for each data table, and the reference evaluation value of each data table and the preset evaluation value of each data table are weighted and summed according to formula (15) to update the reference evaluation value of each data table, i.e., the updated reference evaluation value of each data table:

[0254] S i = (1-γ) * S i + γ * Ri (15)

[0255] wherein, γ is a weighting coefficient, R i is a preset evaluation value of the i th data table, and a value range of the preset evaluation value can be [0, 100].

[0256] After obtaining the reference evaluation value of each updated data table, the weight value of each data table can be determined in the same or similar manner as in the foregoing embodiments, which has been described and will not be repeated here.

[0257] According to different application scenarios, the weight value of each data table is determined in different manners in the embodiments provided in the present application, which can improve the flexibility of determining the weight value of the data table and also improve the accuracy of determining the weight value of the data table.

[0258] As an optional solution, in a case where the data quality parameter of each data table in a group of data tables and the weight value of each data table in the group of data tables satisfy a preset abnormal condition, it is determined that the business data in the group of data tables is abnormal, comprising:

[0259] S121, performing weighted summation on the data quality parameter of each data table according to the weight value of each data table in the group of data tables, to obtain a data quality parameter of the group of data tables;

[0260] S122, in a case where the data quality parameter of the group of data tables is greater than or equal to a preset parameter threshold, determining that the business data in the group of data tables is abnormal.

[0261] In the present embodiment, the data quality parameter of the group of data tables can be determined based on the data quality parameter of each data table and the weight value of each data table. Based on the weight value of each data table, the data quality parameter of each data table can be weighted and summed. When the weighted summation is performed, the weight value of each data table is used for the summation operation of the quality parameter of each data table.

[0262] For example, the integrity measure (i.e., the data quality parameter) of the entire library table can be calculated by formula (16):

[0263]

[0264] wherein, Core is the integrity measure of the entire library table, M is the total number of core tables, Integrity i is the integrity measure (i.e., the data quality parameter) of the i th core table, q i is the weight value of the i th data table.

[0265] After obtaining the data quality parameters for a set of data tables, it can be determined whether the data quality parameters for the set of data tables are greater than or equal to a preset parameter threshold. If so, it can be determined that the business data in the set of data tables is abnormal. Otherwise, it can be determined that the business data in the set of data tables is normal. Here, since the data quality parameters are determined based on the data missing rate and the data fluctuation parameter, the data quality parameters are negatively correlated with the data quality. Therefore, when the data quality parameters are greater than or equal to the preset parameter threshold, it can be determined that the business data in the set of data tables is abnormal.

[0266] Through the embodiments provided in the present application, whether business data in a group of data tables are abnormal is determined based on the data quality parameters of the group of data tables, which can improve the convenience of determining business data anomalies.

[0267] The following describes the method for processing data anomalies in the embodiment of the present application with reference to an optional example. In this optional example, a set of data tables is a set of core tables in a database, the data waveform parameter is represented by volatility, and the data quality parameter is represented by integrity.

[0268] like Figure 10 As shown, the process of the method for handling data anomalies in this optional example may include the following steps:

[0269] Step S1002: Determine the scope of the core table based on the number of times the data table is used, business experience, etc., and obtain core table information.

[0270] Step S1004 : Count the number of partitions of a single core table, and count the number of partitions of the core table and the amount of business data in each partition.

[0271] Step S1006 , calculating the data missing rate of a single core table, may be performed from two aspects: partition missing and partition zero value.

[0272] Step S1008 , calculating the volatility of a single core table, can first detect whether the data item number sequence of each core table is stationary, periodic or irregular, and then select different methods to perform volatility detection according to different types.

[0273] Step S1010: Determine the integrity metric of a single core table based on the data missing rate and volatility of the single core table. For example, the data missing rate and volatility of the single core table may be weightedly fitted to obtain the integrity metric of the single core table.

[0274] In step S1010, the integrity measure of all core tables is determined according to the integrity measure of a single core table. The weight of a single core table can be determined based on information entropy, and the weight of a single core table can be adjusted by a preset evaluation value of a single data table. The integrity measure of a single core table is multiplied by the weight of a single core table to obtain the integrity measure of all core tables.

[0275] Through the optional example, the integrity of the core table is calculated from the data missing rate and volatility, the dimension of the core table integrity measurement is richer, and the accuracy of the core table integrity measurement can be improved. The weight of the core table is determined based on information entropy, and the weight of the core table can be adjusted based on the preset evaluation value of the core table, which can improve the rationality of the determination of the weight of the core table. In addition, by detecting the data volatility of the data item sequence, the timeliness of abnormal fluctuation discovery can be improved.

[0276] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action order described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0277] According to another aspect of the embodiments of the present application, a data anomaly processing apparatus for implementing the data anomaly processing method is also provided. Figure 11 is a structural block diagram of an optional data anomaly processing apparatus according to an embodiment of the present application, as shown in Figure 11 The apparatus can include:

[0278] The first acquisition unit 1102 is configured to acquire a data missing rate of each data table in a group of data tables, and acquire a data fluctuation parameter of each data table in the group of data tables, wherein the group of data tables includes business data generated by a target application during running, and the data fluctuation parameter of each data table is used to represent a degree of fluctuation of business data in each data table over time.

[0279] The first determination unit 1104 is configured to determine a data quality parameter of each data table in the group of data tables according to the data missing rate of each data table in the group of data tables and the data fluctuation parameter of each data table in the group of data tables.

[0280] The second determination unit 1106 is configured to determine a weight value of each data table in the set of data tables according to operation information of the multi-dimensional operation performed on each data table in the set of data tables, where the weight value of each data table in the set of data tables is used to represent the importance of each data table.

[0281] The third determination unit 1108 is configured to determine that the business data in the set of data tables is abnormal in a case where the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables satisfy a preset abnormal condition.

[0282] It should be noted that the first acquisition unit 1102 in this embodiment can be configured to perform step S202, the first determination unit 1104 in this embodiment can be configured to perform step S204, the second determination unit 1106 in this embodiment can be configured to perform step S206, and the third determination unit 1108 in this embodiment can be configured to perform step S208.

[0283] According to the embodiments provided in the present application, the data missing rate of each data table in a set of data tables is obtained, and the data fluctuation parameter of each data table in the set of data tables is obtained, where the set of data tables includes business data generated by a target application during runtime, and the data fluctuation parameter of each data table is used to represent the fluctuation degree of the business data in each data table over time; the data quality parameter of each data table in the set of data tables is determined according to the data missing rate of each data table in the set of data tables and the data fluctuation parameter of each data table in the set of data tables; the weight value of each data table in the set of data tables is determined according to operation information of a multi-dimensional operation performed on each data table in the set of data tables, where the weight value of each data table in the set of data tables is used to represent the importance of each data table; and in a case where the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables satisfy a preset abnormal condition, it is determined that the business data in the set of data tables is abnormal, thereby solving the problem of low coverage of data table abnormal early warning in the related art by using the data missing rate to measure the integrity of the data table, and improving the coverage of data table abnormal early warning.

[0284] As an optional solution, the first acquisition unit 1102 includes:

[0285] The execution module is configured to perform the following steps for each data table in the set of data tables, where each data table is a current data table when the following steps are performed:

[0286] determine a data item number of the service data generated in each preset period in the current data table, to obtain a data item number sequence, wherein the data item number sequence comprises a current data item number and a set of historical data item numbers, the current data item number is a data item number of the service data generated in a last preset period in the current data table, and the set of historical data item numbers are data item numbers in the data item number sequence except the current data item number;

[0287] determine the data fluctuation parameter of the current data table according to fluctuation of the current data item number relative to the set of historical data item numbers.

[0288] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0289] As an optional solution, the execution module comprises:

[0290] The first determination submodule is configured to determine an average item number of the set of historical data item numbers in a case where the data item number sequence is a stationary sequence.

[0291] The second determination submodule is configured to determine the data fluctuation parameter of the current data table as a first parameter value in a case where the item number difference between the current data item number and the average item number is greater than a preset difference threshold, wherein the first parameter value is used to represent that the service data generated in the last preset period in the current data table is fluctuated abnormally.

[0292] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0293] As an optional solution, the execution module further comprises:

[0294] The first detection submodule is configured to detect whether the current data item number is located in a first item number reference range in a case where the item number difference between the current data item number and the average item number is less than or equal to the preset difference threshold, wherein the first item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the set of historical data item numbers.

[0295] The third determination submodule is configured to determine the data fluctuation parameter of the current data table as a second parameter value in a case where the current data item number is located in the first item number reference range, wherein the second parameter value is different from the first parameter value.

[0296] The fourth determination submodule is configured to determine the data fluctuation parameter of the current data table as the first parameter value in a case where the current data item number is located outside the first item number reference range.

[0297] Optional examples of the present embodiment can refer to the examples shown in the above processing method of data anomaly, which will not be repeated here in the present embodiment.

[0298] As an optional solution, the execution module comprises:

[0299] The fifth determination sub-module is configured to, in a case where the data item sequence is a non-stationary sequence and a periodic sequence, and a set of preset periods comprises a plurality of preset period sets and a current preset period set, determine, from a set of historical data items, a target data item corresponding to the current data item in each preset period set of the plurality of preset period sets, to obtain a set of target data items, wherein the number of preset periods in each preset period set of the plurality of preset period sets and the current preset period set is the same, the current data item is a data item number of service data generated in a current preset period in the current preset period set, and a preset period corresponding to each target data item has the same position in the corresponding preset period set as the current preset period in the current preset period set;

[0300] The second detection sub-module is configured to detect whether the current data item number is located in a second item number reference range, wherein the second item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the set of target data items;

[0301] The sixth determination sub-module is configured to, in a case where the current data item number is located outside the second item number reference range, determine that the data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to represent that the service data generated in the last preset period in the current data table fluctuates abnormally;

[0302] The seventh determination sub-module is configured to, in a case where the current data item number is located in the second item number reference range, determine that the data fluctuation parameter of the current data table is a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to represent that the service data generated in the last preset period in the current data table fluctuates normally.

[0303] Optional examples of the present embodiment can refer to the examples shown in the above processing method of data anomaly, which will not be repeated here in the present embodiment.

[0304] As an optional solution, the execution module comprises:

[0305] The third detection sub-module is configured to, in a case where the data item sequence is a non-stationary sequence and a non-periodic sequence, detect whether the current data item number is located in a third item number reference range, wherein the third item number reference range is a normal fluctuation range allowed for the current data item number and is determined according to a weighted average item number of the set of historical data items;

[0306] The eighth determining sub-module is configured to determine the data fluctuation parameter of the current data table as a first parameter value in a case where the current data item number is located outside the third item number reference range, wherein the first parameter value is used to represent that the service data fluctuation in the last preset period in the current data table is abnormal.

[0307] The ninth determining sub-module is configured to determine the data fluctuation parameter of the current data table as a second parameter value in a case where the current data item number is located within the third item number reference range, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to represent that the service data fluctuation in the last preset period in the current data table is normal.

[0308] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0309] As an optional solution, the above-mentioned device further includes:

[0310] The extraction unit is configured to perform a fluctuation coefficient extraction operation on the data item number sequence to obtain a set of fluctuation coefficients.

[0311] The fourth determining unit is configured to determine that the data item number sequence is a stationary sequence in a case where the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients is less than or equal to a preset ratio.

[0312] The fifth determining unit is configured to determine that the data item number sequence is a non-stationary sequence in a case where the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients is less than the preset ratio.

[0313] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0314] As an optional solution, the above-mentioned device further includes:

[0315] The second acquisition unit is configured to acquire, in the data item number sequence, the data item number corresponding to each preset period in the first preset period set to obtain a first reference item number sequence, and acquire the data item number corresponding to each preset period in the second preset period set to obtain a second reference item number sequence, wherein the first preset period set and the second preset period set are adjacent preset period sets in a set of preset periods.

[0316] The sixth determining unit is configured to determine the difference between the data item numbers at each same position in the first reference item number sequence and the second reference item number sequence to obtain a target item number difference sequence.

[0317] The seventh determination unit is configured to determine that the data item number sequence is a periodic sequence when the standard deviation of the target item number difference value sequence is less than or equal to a preset standard deviation threshold.

[0318] The eighth determination unit is configured to determine that the data item number sequence is a non-periodic sequence when the standard deviation of the target item number difference value sequence is greater than the preset standard deviation threshold.

[0319] Optional examples of the present embodiment can refer to the examples shown in the above data anomaly processing method, which will not be repeated here in the present embodiment.

[0320] As an optional solution, the second determination unit 1106 includes:

[0321] The first determination module is configured to determine a weight value of each dimension operation in the multi-dimensional operation according to the operation number of the multi-dimensional operation performed on each data table in the group of data tables.

[0322] The second determination module is configured to determine a reference evaluation value of each data table according to the operation number of the multi-dimensional operation performed on each data table and the weight value of each dimension operation.

[0323] The third determination module is configured to determine a weight value of each data table according to the reference evaluation value of each data table.

[0324] Optional examples of the present embodiment can refer to the examples shown in the above data anomaly processing method, which will not be repeated here in the present embodiment.

[0325] As an optional solution, the first determination module includes:

[0326] The tenth determination submodule is configured to determine a weight value of each dimension operation in the multi-dimensional operation under each data table in the group of data tables according to the operation number of the multi-dimensional operation performed on each data table.

[0327] The eleventh determination submodule is configured to determine a weight value of each dimension operation according to the weight value of each dimension operation under each data table.

[0328] Optional examples of the present embodiment can refer to the examples shown in the above data anomaly processing method, which will not be repeated here in the present embodiment.

[0329] As an optional solution, the tenth determination submodule includes:

[0330] The execution subunit is configured to perform a standardization operation on the operation number of each data table in the next group of data tables in each dimension operation in the multi-dimensional operation, to obtain a first reference value of each data table in each dimension operation, wherein the first reference value of each data table in each dimension operation is a ratio of a first difference value and a second difference value, the first difference value is a difference between the operation number of each data table in each dimension operation and a first operation number, the second difference value is a difference between a second operation number and the first operation number, the first operation number is a minimum operation number in the operation number of the next group of data tables in each dimension operation, and the second operation number is a maximum operation number in the operation number of the next group of data tables in each dimension operation;

[0331] The determination subunit is configured to determine a ratio of the first reference value of each data table in each dimension operation and a second reference value as a weight value of each dimension operation at each data table, wherein the second reference value is a sum of the first reference values of each data table in each dimension operation.

[0332] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0333] As an optional solution, the third determination module comprises:

[0334] The twelfth determination sub-module is configured to determine a ratio of the reference evaluation value of each data table and a reference weight value of the group of data tables as the weight value of each data table, wherein the reference weight value of the group of data tables is a sum of the reference evaluation values of each data table; or

[0335] The first execution sub-module is configured to perform weighted summation on the reference evaluation value of each data table and the preset evaluation value of each data table, to obtain an updated reference evaluation value of each data table.

[0336] The thirteenth determination sub-module is configured to determine a ratio of the updated reference evaluation value of each data table and an updated reference weight value of the group of data tables as the weight value of each data table, wherein the updated reference weight value of the group of data tables is a sum of the updated reference evaluation values of each data table.

[0337] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data anomaly processing method, which will not be repeated here in the present embodiment.

[0338] As an optional solution, the third determination unit 1108 comprises:

[0339] The second execution sub-module is configured to perform weighted summation on the data quality parameters of each data table in the group of data tables according to the weight value of each data table in the group of data tables, to obtain a data quality parameter of the group of data tables.

[0340] The fourteenth determination sub-module is configured to determine that the business data in the set of data tables is abnormal when the data quality parameter of the set of data tables is greater than or equal to the preset parameter threshold.

[0341] Optional examples of the present embodiment can refer to the examples shown in the above-mentioned data exception processing method, and the present embodiment will not be repeated here.

[0342] According to another aspect of the embodiments of the present application, an electronic device for implementing the above-mentioned data exception processing method is also provided, which can be a terminal device or a server as shown. Figure 1 As shown in the above-mentioned electronic device, the electronic device includes a memory 1202 and a processor 1204, the memory 1202 stores a computer program, and the processor 1204 is configured to execute the steps in any of the above-mentioned method embodiments through the computer program. Figure 12

[0343] Optionally, in the present embodiment, the above-mentioned electronic device can be located in at least one of a plurality of network devices in a computer network.

[0344] Optionally, in the present embodiment, the above-mentioned processor can be configured to execute the following steps through the computer program:

[0345] S1, obtaining a data missing rate of each data table in a set of data tables, and obtaining a data fluctuation parameter of each data table in the set of data tables, wherein the set of data tables includes business data generated by a target application during runtime, and the data fluctuation parameter of each data table is used to represent a degree of fluctuation of the business data in each data table over time.

[0346] S2, determining a data quality parameter of each data table in the set of data tables according to the data missing rate of each data table in the set of data tables and the data fluctuation parameter of each data table in the set of data tables.

[0347] S3, determining a weight value of each data table in the set of data tables according to operation information of a multi-dimensional operation performed on each data table in the set of data tables, wherein the weight value of each data table in the set of data tables is used to represent an importance of each data table;

[0348] S4, determining that the business data in the set of data tables is abnormal when the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables satisfy a preset abnormal condition.

[0349] Optionally, those skilled in the art can understand that, Figure 12 ​The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, or a terminal device such as an MID or a PAD. Figure 12 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 12 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 12 Different configurations shown.

[0350] Among them, the memory 1202 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for processing data anomalies in the embodiments of the present application. The processor 1204 executes various functional applications and data processing by running the software programs and modules stored in the memory 1202, that is, realizes the above-mentioned method for processing data anomalies. The memory 1202 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1202 may further include a memory remotely located relative to the processor 1204, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1202 can be specifically, but not limited to, used for serialized files and compiled files and other information. As an example, such as Figure 12 As shown, the memory 1202 may include, but is not limited to, the first display unit 1102, the second display unit 1104, and the first switching unit 1106 in the data anomaly processing device. In addition, it may also include, but is not limited to, other module units in the data anomaly processing device, which will not be repeated in this example.

[0351] Optionally, the transmission device 1206 is configured to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one embodiment, the transmission device 1206 includes a network interface controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In one embodiment, the transmission device 1206 is a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0352] In addition, the electronic device further includes: a display 1208 for displaying the target game interface; and a connection bus 1210 for connecting various module components in the electronic device.

[0353] In other embodiments, the terminal device or the server described above can be a node in a distributed system, where the distributed system can be a blockchain system, which can be a distributed system formed by the plurality of nodes connected through network communication. Wherein, the nodes can form a peer-to-peer (P2P) network, and any form of computing device, such as a server, a terminal, and the like, can become a node in the blockchain system by joining the peer-to-peer network.

[0354] According to an aspect of the present application, a computer program product is provided, which includes computer programs / instructions containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the detachable medium 1311. When the computer program is executed by the central processing unit 1301, various functions provided by the embodiments of the present application are executed. The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0355] Figure 13 The computer system structure block diagram of the electronic device for implementing the embodiments of the present application is schematically shown. As shown in the figure, Figure 13 The computer system 1300 includes a central processing unit 1301 (CPU), which can perform various appropriate actions and processes according to the programs stored in the read-only memory 1302 (ROM) or the programs loaded from the storage part 1308 to the random access memory 1303 (RAM). In the random access memory 1303, various programs and data required for system operation are also stored. The central processing unit 1301, the read-only memory 1302, and the random access memory 1303 are connected to each other through the bus 1304. The input / output interface 1305 (I / O interface) is also connected to the bus 1304.

[0356] The following components are connected to the input / output interface 1305: an input part 1306 including a keyboard, a mouse, and the like; an output part 1307 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage part 1308 including a hard disk, and the like; and a communication part 1309 including a network interface card such as a local area network card, a modem, and the like. The communication part 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output interface 1305 as necessary. A removable media 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 1310 as necessary, so that a computer program read out therefrom is installed in the storage part 1308 as necessary.

[0357] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication part 1309, and / or installed from the removable media 1311. When the computer program is executed by the central processing unit 1301, various functions defined in the system of the present application are performed.

[0358] It should be noted that, Figure 13 The computer system 1300 of the electronic device shown is merely an example, and should not impose any limitation on the functions and usage range of embodiments of the present application.

[0359] According to an aspect of the present application, a computer readable storage medium is provided, from which a processor of a computer device reads computer instructions, and the processor executes the computer instructions to cause the computer device to perform the method provided in various optional implementations in the above embodiments.

[0360] Optionally, in the present embodiment, the above computer readable storage medium can be configured to store a computer program for performing the following steps:

[0361] S1, obtaining a data missing rate of each data table in a group of data tables, and obtaining a data fluctuation parameter of each data table in the group of data tables, wherein the group of data tables comprises business data generated by a target application during runtime, and the data fluctuation parameter of each data table is used to represent a degree of fluctuation of business data in each data table over time.

[0362] S2, determining, according to the data missing rate of each data table in the set of data tables and the data fluctuation parameter of each data table in the set of data tables, a data quality parameter of each data table in the set of data tables.

[0363] S3, determining, according to operation information of the multi-dimension operation performed on each data table in the set of data tables, a weight value of each data table in the set of data tables, wherein the weight value of each data table in the set of data tables is used to represent the importance of each data table.

[0364] S4, determining that the business data in the set of data tables is abnormal in a case where the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables satisfy a preset abnormal condition.

[0365] Optionally, in the embodiment, it can be understood by those skilled in the art that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device by a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0366] The integrated units in the above-mentioned embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above-mentioned computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in the storage medium, including a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application.

[0367] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0368] In several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other manners. Of course, the described apparatus embodiments are merely schematic, and the division of units is merely logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, units or modules, and can be in electrical, mechanical or other forms.

[0369] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0370] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or at least two units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0371] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for processing data anomalies, characterized in that: include: Obtaining a data missing rate for each data table in a set of data tables, and obtaining a data fluctuation parameter for each data table in the set of data tables, wherein the set of data tables includes business data generated by a target application during runtime, and the data fluctuation parameter for each data table is used to characterize the degree to which the business data in each data table fluctuates over time; determining a data quality parameter for each data table in the set of data tables according to a data missing rate of each data table in the set of data tables and a data fluctuation parameter of each data table in the set of data tables; determining a weight value of each data table in the set of data tables according to operation information of a multi-dimensional operation performed on each data table in the set of data tables, wherein the weight value of each data table in the set of data tables is used to represent the importance of each data table; When the data quality parameter of each data table in the group of data tables and the weight value of each data table in the group of data tables meet a preset abnormality condition, it is determined that the business data in the group of data tables is abnormal.

2. The method according to claim 1, characterized in that The obtaining of the data fluctuation parameter of each data table in the set of data tables includes: The following steps are performed for each data table in the set of data tables, wherein each data table is a current data table when performing the following steps: Determine the number of data items of business data generated in the current data table within each preset period in a set of preset periods to obtain a data item number sequence, wherein the data item number sequence includes a current data item number and a set of historical data item numbers, the current data item number is the number of data items of business data generated in the current data table within the last preset period in the set of preset periods, and the set of historical data item numbers is the number of data items in the data item number sequence excluding the current data item number; determine a data fluctuation parameter of the current data table based on the fluctuation of the current data item number relative to the set of historical data item numbers.

3. The method according to claim 2, characterized in that Determining the data fluctuation parameter of the current data table according to the fluctuation of the current number of data items relative to the set of historical data item numbers includes: In the case where the data item number sequence is a stationary sequence, determining the average number of items in the set of historical data items; When the difference in the number of items between the current number of data items and the average number of items is greater than a preset difference threshold, the data fluctuation parameter of the current data table is determined to be a first parameter value, wherein the first parameter value is used to represent the business data fluctuation anomaly generated in the last preset period in the current data table.

4. The method according to claim 3, characterized in that Determining the data fluctuation parameter of the current data table based on the fluctuation of the current number of data items relative to the set of historical data items further includes: If the difference between the current number of data items and the average number of items is less than or equal to a preset difference threshold, detecting whether the current number of data items is within a first number reference range, wherein the first number reference range is a normal fluctuation range allowed for the current number of data items, determined based on a weighted average number of items of the set of historical data items; When the current number of data items is within the first number reference range, determining the data fluctuation parameter of the current data table to be a second parameter value, wherein the second parameter value is different from the first parameter value; When the current number of data items is outside the first number reference range, the data fluctuation parameter of the current data table is determined to be the first parameter value.

5. The method according to claim 2, characterized in that Determining the data fluctuation parameter of the current data table according to the fluctuation of the current number of data items relative to the set of historical data item numbers includes: In a case where the data item number sequence is a non-stationary sequence and a periodic sequence, and the set of preset periods includes a plurality of preset period sets and a current preset period set, determining a target number of data items corresponding to the current number of data items for each preset period set in the plurality of preset period sets from the set of historical data item numbers to obtain a set of target numbers of data items, wherein the number of preset periods in each preset period set in the plurality of preset period sets and the current preset period set is the same, the current number of data items is the number of data items of business data generated in the current preset period in the current preset period set, and the position of the preset period corresponding to each target number of data items in the corresponding preset period set is the same as the position of the current preset period in the current preset period set; detecting whether the current number of data items is within a second number reference range, wherein the second number reference range is a normal fluctuation range allowed for the current number of data items and is determined based on a weighted average number of the set of target data items; When the current number of data items is outside the second number reference range, determining that the data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to indicate abnormal business data fluctuation generated in the last preset period in the current data table; When the current number of data items is within the reference range of the second number of items, the data fluctuation parameter of the current data table is determined to be a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to indicate that the business data fluctuation generated in the last preset period in the current data table is normal.

6. The method according to claim 2, characterized in that Determining the data fluctuation parameter of the current data table according to the fluctuation of the current number of data items relative to the set of historical data item numbers includes: In a case where the data item number sequence is a non-stationary sequence and a non-periodic sequence, detecting whether the current number of data items is within a third number reference range, wherein the third number reference range is a normal fluctuation range allowed for the current number of data items and is determined based on a weighted average number of items of the set of historical data items; When the current number of data items is outside the third number reference range, determining that the data fluctuation parameter of the current data table is a first parameter value, wherein the first parameter value is used to indicate that the business data fluctuation abnormality generated in the last preset period in the current data table; When the current number of data items is within the reference range of the third number of items, the data fluctuation parameter of the current data table is determined to be a second parameter value, wherein the second parameter value is different from the first parameter value, and the second parameter value is used to indicate that the business data fluctuation generated in the last preset period in the current data table is normal.

7. The method according to claim 3, 5 or 6, characterized in that The method further comprises: Performing a fluctuation coefficient extraction operation on the data item number sequence to obtain a set of fluctuation coefficients; When the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients is less than or equal to a preset ratio, determining that the data item number sequence is a stationary sequence; When the ratio of the standard deviation of the data item number sequence to the standard deviation of the set of fluctuation coefficients is less than the preset ratio, it is determined that the data item number sequence is a non-stationary sequence.

8. The method according to claim 5 or 6, characterized in that The method further comprises: In the data item number sequence, obtaining the number of data items corresponding to each preset period in a first preset period set to obtain a first reference item number sequence, and obtaining the number of data items corresponding to each preset period in a second preset period set to obtain a second reference item number sequence, wherein the first preset period set and the second preset period set are adjacent preset period sets in the set of preset periods; determining a difference in the number of data items at each identical position between the first reference item number sequence and the second reference item number sequence to obtain a target item number difference sequence; When the standard deviation of the target item number difference sequence is less than or equal to a preset standard deviation threshold, determining that the data item number sequence is a periodic sequence; When the standard deviation of the target item number difference sequence is greater than the preset standard deviation threshold, it is determined that the data item number sequence is a non-periodic sequence.

9. The method according to claim 1, characterized in that The determining, based on operation information of the multi-dimensional operation performed on each data table in the set of data tables, a weight value of each data table in the set of data tables includes: determining a weight value of each dimensional operation in the multidimensional operation according to the number of operations of the multidimensional operation performed on each data table in the set of data tables; Determining a reference evaluation value of each data table according to the number of multi-dimensional operations performed on each data table and a weight value of each dimensional operation; A weight value of each data table is determined according to the reference evaluation value of each data table.

10. The method according to claim 9, characterized in that The determining, according to the number of operations of the multi-dimensional operation performed on each data table in the set of data tables, a weight value of each dimensional operation in the multi-dimensional operation includes: Determining a weight value of each dimensional operation in the multidimensional operation for each data table according to the number of operations of the multidimensional operation performed on each data table in the set of data tables; The weight value of the operation in each dimension is determined according to the weight value of the operation in each data table.

11. The method according to claim 10, characterized in that The determining, based on the number of operations of the multi-dimensional operation performed on each data table in the set of data tables, a weight value of each dimensional operation in the multi-dimensional operation for each data table, includes: performing a normalization operation on the operands of each data table in the set of data tables under each dimensional operation in the multi-dimensional operation to obtain a first reference value of each data table under the each dimensional operation, wherein the first reference value of each data table under the each dimensional operation is a ratio of a first difference to a second difference, the first difference is a difference between the operands of each data table under the each dimensional operation and the first operand, the second difference is a difference between the second operand and the first operand, the first operand is a minimum operand among the operands of the set of data tables under the each dimensional operation, and the second operand is a maximum operand among the operands of the set of data tables under the each dimensional operation; The ratio of the first reference value to the second reference value of each data table under the per-dimensional operation is determined as the weight value of the per-dimensional operation under each data table, wherein the second reference value is the sum of the first reference values ​​of each data table under the per-dimensional operation.

12. The method according to claim 9, characterized in that Determining the weight value of each data table according to the reference evaluation value of each data table includes: Determine the ratio of the reference evaluation value of each data table to the reference weight value of the group of data tables as the weight value of each data table, wherein the reference weight value of the group of data tables is the sum of the reference evaluation values ​​of each data table; or Perform weighted summation on the reference evaluation value of each data table and the preset evaluation value of each data table to obtain an updated reference evaluation value of each data table; determine the weight value of each data table as the ratio of the updated reference evaluation value of each data table to the updated reference weight value of the group of data tables, wherein the updated reference weight value of the group of data tables is the sum of the updated reference evaluation values ​​of each data table.

13. The method according to claim 1, wherein The determining that the business data in the set of data tables is abnormal when the data quality parameter of each data table in the set of data tables and the weight value of each data table in the set of data tables meet a preset abnormality condition includes: performing weighted summation on the data quality parameters of each data table according to the weight value of each data table in the set of data tables to obtain the data quality parameters of the set of data tables; When the data quality parameter of the group of data tables is greater than or equal to a preset parameter threshold, it is determined that the business data in the group of data tables is abnormal.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein the method according to any one of claims 1 to 13 is executed when the program is executed.

15. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 13 through the computer program.

Images