Embedded data indicator monitoring and alarm method and system based on data warehouse

Abstract

The embodiment of the present invention discloses an embedded data indicator monitoring and alarm method and system based on a data warehouse. Specifically, the following methods are included: reading the alarm configuration information in the monitoring and alarm configuration information table; parsing the indicator rules in the alarm configuration information to obtain the indicator rule parameters; judging whether the indicator to be monitored already exists based on the is_exsit field; obtaining each basic indicator value from the data monitoring result table and calculating the daily year-on-year or monthly year-on-year; reading the alarm configuration information that meets the current monitoring frequency in the monitoring and alarm configuration information table; matching and comparing the alarm trigger threshold rules in the alarm configuration information with the obtained basic indicator values ​​one by one, judging whether the basic indicator values ​​meet the alarm trigger threshold range; outputting the alarm information table based on the generated alarm content data. This application solves certain limitations of existing data indicator monitoring and alarm technologies, including that data must be transmitted through a third-party platform, that the function configuration method is relatively fixed and has low flexibility, and other issues.

Description

Technical Field

[0001] The present invention relates to the technical field of data warehouses, and in particular to an embedded data indicator monitoring and alarming method and system based on a data warehouse. Background Art

[0002] In the era of big data, with the development of the concept of data-driven business, business operations are increasingly dependent on data indicators, and thus the demand for monitoring data indicators has emerged. On the one hand, in order to understand the business operation status in a timely manner, business personnel need to quickly perceive data fluctuations so as to handle business situations in a timely manner; on the other hand, in order to deal with various sudden abnormal conditions in business operations, technical personnel need to perceive data anomalies in a timely manner so as to respond quickly and repair them. The existing data indicator monitoring and alarm technical means are generally achieved by associating or exporting data to a third-party data reporting platform or monitoring platform, and applying the monitoring configuration function therein. There are certain limitations in the use of such technical means: data must be passed through a third-party platform, and the configuration function module is relatively fixed and less flexible.

[0003] To sum up, the existing data indicator monitoring and alarm technology has certain limitations, including that data must pass through a third-party platform, the function configuration method is relatively fixed and has low flexibility. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an embedded data indicator monitoring and alarm method and system based on a data warehouse, so as to solve certain limitations of existing data indicator monitoring and alarm technologies, including that data must pass through a third-party platform, the function configuration method is relatively fixed and has low flexibility, and other problems.

[0005] To achieve the above object, an embodiment of the present invention provides an embedded data indicator monitoring and alarming method based on a data warehouse, the method comprising the following steps:

[0006] Read the alarm configuration information in the monitoring alarm configuration information table;

[0007] Parse the indicator rules in the alarm configuration information and obtain the indicator rule parameters; determine whether the indicator to be monitored already exists based on the is_exsit field. If the indicator to be monitored already exists, query the basic indicator value based on the indicator rule parameters and spliced ​​SQL; if the indicator to be monitored does not exist, query the basic indicator value based on the SQL statement in the indicator rule;

[0008] Merge, organize and store the basic indicator values ​​queried in the data monitoring result table;

[0009] Get the values ​​of each basic indicator from the data monitoring result table and calculate the daily or monthly year-on-year comparison;

[0010] Read the alarm configuration information that meets the current monitoring frequency from the monitoring alarm configuration information table;

[0011] Based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, a one-to-one match is performed to determine whether the basic indicator value meets the alarm trigger threshold range. If it meets the alarm trigger threshold range, alarm content data is generated; if it does not meet the alarm trigger threshold range, no alarm content data is generated;

[0012] Based on the generated alarm content data, an alarm information table is output.

[0013] Furthermore, whether the indicator to be monitored already exists is judged based on the is_exsit field. When is_exsit=1, it indicates that the indicator to be monitored already exists; when is_exsit=0, it indicates that the indicator to be monitored does not exist.

[0014] Furthermore, if the indicator to be monitored already exists, the SQL query basic indicator value is formed based on the table name parameter, dimension parameter and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field and table_indicator_name field.

[0015] Further, the values ​​of each basic indicator are obtained from the data monitoring result table and the daily year-on-year or monthly year-on-year growth is calculated;

[0016] Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value;

[0017] Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

[0018] Furthermore, the alert_period field is used to filter the alarm configuration information that meets the current monitoring frequency;

[0019] For 30-minute frequency alarms, the filter condition is: alert_period = '30mi';

[0020] For hourly frequency alarms, the filter condition is: alert_period = '1h';

[0021] For daily frequency alarms, the filter condition is: alert_period = '1d';

[0022] For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

[0023] An embedded data indicator monitoring and alarm system based on a data warehouse, the system includes:

[0024] A first reading unit, used to read the alarm configuration information in the monitoring alarm configuration information table;

[0025] The first parsing unit is used to parse the indicator rules in the alarm configuration information and obtain the indicator rule parameters; based on the is_exsit field, determine whether the indicator to be monitored already exists. If the indicator to be monitored already exists, query the basic indicator value according to the indicator rule parameters and the spliced ​​SQL; if the indicator to be monitored does not exist, query the basic indicator value according to the SQL statement in the indicator rule;

[0026] A processing unit, used to merge, organize and store the basic indicator values ​​queried into a data monitoring result table;

[0027] A calculation unit, used to obtain the values ​​of each basic indicator from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year;

[0028] The second reading unit is used to read the alarm configuration information that meets the current monitoring frequency in the monitoring alarm configuration information table;

[0029] The second parsing unit is used to match and compare the alarm trigger threshold rule in the alarm configuration information with the obtained basic indicator value one by one, and determine whether the basic indicator value meets the alarm trigger threshold range. If it meets the alarm trigger threshold range, alarm content data is generated; if it does not meet the alarm trigger threshold range, alarm content data is not generated;

[0030] The output unit is used to output an alarm information table based on the generated alarm content data.

[0031] Furthermore, whether the indicator to be monitored already exists is judged based on the is_exsit field. When is_exsit=1, it indicates that the indicator to be monitored already exists; when is_exsit=0, it indicates that the indicator to be monitored does not exist.

[0032] Furthermore, if the indicator to be monitored already exists, the SQL query basic indicator value is formed based on the table name parameter, dimension parameter and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field and table_indicator_name field.

[0033] Furthermore, the calculation unit is used to obtain each basic indicator value from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year growth;

[0034] Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value;

[0035] Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

[0036] Furthermore, the alert_period field is used to filter the alarm configuration information that meets the current monitoring frequency;

[0037] For 30-minute frequency alarms, the filter condition is: alert_period = '30mi';

[0038] For hourly frequency alarms, the filter condition is: alert_period = '1h';

[0039] For daily frequency alarms, the filter condition is: alert_period = '1d';

[0040] For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

[0041] The embodiments of the present invention have the following advantages:

[0042] 1. The present invention reads the alarm configuration information in the monitoring alarm configuration information table; parses the indicator rules in the alarm configuration information to obtain the indicator rule parameters; determines whether the indicator to be monitored already exists based on the is_exsit field, and if the indicator to be monitored already exists, queries the basic indicator value according to the indicator rule parameters and the splicing SQL; if the indicator to be monitored does not exist, queries the basic indicator value according to the SQL statement in the indicator rule; merges, organizes and stores the queried basic indicator values ​​in the data monitoring result table; obtains each basic indicator value from the data monitoring result table and calculates the day-on-year or month-on-month comparison; reads the alarm configuration information that meets the current monitoring frequency in the monitoring alarm configuration information table; performs one-to-one matching and comparison based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, determines whether the basic indicator value meets the alarm trigger threshold range, generates alarm content data if it meets the alarm trigger threshold range, and does not generate alarm content data if it does not meet the alarm trigger threshold range; outputs the alarm information table based on the generated alarm content data. It solves the problems that the existing data indicator monitoring and alarm technology has certain limitations, data must be transmitted through a third-party platform, and the configuration function modules are relatively fixed and less flexible.

[0043] 2. This application is highly embeddable. Currently, most common monitoring and alarm systems are third-party software that need to be installed and associated with the data warehouse. However, the monitoring and alarm configuration information table, data monitoring result table, and alarm information table in this application can be directly embedded in the data warehouse as nodes and data tables in the data warehouse, which is more suitable in the data warehouse.

[0044] 3. The rules of this application are flexible to configure. The current common monitoring and alarm systems often have configuration limitations, such as the configuration selection of monitoring indicators, the threshold configuration for triggering alarms, and the alarm information issued. In this application, the monitoring scope of the alarm is not limited to one indicator. Multiple indicators can be captured at the same time, including one main indicator and multiple auxiliary indicators, which effectively meets the scalability of monitoring; the threshold rules of the alarm are scalable. In addition to the existing threshold rules, new threshold rules can be added through customized development; in the alarm display content part, the alarm content can be customized to facilitate quick understanding of the information when the alarm is triggered. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0046] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0047] Figure 1 A method flow chart of an embedded data indicator monitoring and alarm method based on a data warehouse provided by an embodiment of the present invention;

[0048] Figure 2 A framework diagram of a system for an embedded data indicator monitoring and alarm system based on a data warehouse provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example 1

[0051] Combination Figure 1This embodiment describes an embedded data indicator monitoring and alarming method based on a data warehouse, and the method includes the following steps:

[0052] S1: Read the alarm configuration information in the monitoring alarm configuration information table;

[0053] S2: Parse the indicator rules in the alarm configuration information and obtain the indicator rule parameters; determine whether the indicator to be monitored already exists based on the is_exsit field. If the indicator to be monitored already exists, query the basic indicator value based on the indicator rule parameters and the spliced ​​SQL; if the indicator to be monitored does not exist, query the basic indicator value based on the SQL statement in the indicator rule;

[0054] S3: merge, organize and store the basic indicator values ​​of the query into the data monitoring result table;

[0055] S4: Obtain the values ​​of each basic indicator from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year;

[0056] S5: Read the alarm configuration information that meets the current monitoring frequency from the monitoring alarm configuration information table;

[0057] S6: Based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, one by one matching and comparing, determine whether the basic indicator value meets the alarm trigger threshold range, if it meets the alarm trigger threshold range, generate alarm content data, if it does not meet the alarm trigger threshold range, do not generate alarm content data;

[0058] S7: Based on the generated alarm content data, output an alarm information table.

[0059] In this implementation, the alarm configuration information in the monitoring alarm configuration information table is read; the indicator rules in the alarm configuration information are parsed to obtain the indicator rule parameters; based on the is_exsit field, it is determined whether the indicator to be monitored already exists. If the indicator to be monitored already exists, the basic indicator value is queried according to the indicator rule parameters and the splicing SQL; if the indicator to be monitored does not exist, the basic indicator value is queried according to the SQL statement in the indicator rule; the queried basic indicator values ​​are merged, sorted and stored in the data monitoring result table; each basic indicator value is obtained from the data monitoring result table and the daily or monthly year-on-year comparison is calculated; the alarm configuration information that meets the current monitoring frequency is read from the monitoring alarm configuration information table; based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, one by one matching and comparing, it is determined whether the basic indicator value meets the alarm trigger threshold range, if it meets the alarm trigger threshold range, the alarm content data is generated, if it does not meet the alarm trigger threshold range, the alarm content data is not generated; based on the generated alarm content data, the alarm information table is output. This solves the existing data indicator monitoring alarm technology has certain limitations, including the data must be passed through a third-party platform, the function configuration method is relatively fixed and the flexibility is low.

[0060] The monitoring alarm configuration information table in this embodiment is a data table filled in by manual configuration, which includes basic alarm information, alarm monitoring scope, alarm threshold rules, alarm display content and alarm access method. The basic alarm information includes the data entry date, alarm type grouping, alarm name, alarm cycle and whether the basic alarm information is valid. The alarm monitoring scope includes alarm indicators. The alarm threshold rules include alarm rules. The alarm display content includes alarm statements and alarm dimension names (can be empty). Alarm access methods include DingTalk group messages, DingTalk work orders and phone calls, among which DingTalk group messages include alarm group url and special reminder user information; DingTalk work orders include whether the platform work order alarm is, the problem type, the business line and the problem type number; the phone includes whether the phone alarm is.

[0061] In a preferred embodiment, this implementation method determines whether the indicator to be monitored already exists based on the is_exsit field. When is_exsit=1, it indicates that the indicator to be monitored already exists; when is_exsit=0, it indicates that the indicator to be monitored does not exist.

[0062] In a preferred embodiment, if the indicator to be monitored already exists, the SQL query basic indicator value is formed based on the table name parameter, dimension parameter and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field and table_indicator_name field.

[0063] In a preferred embodiment, this implementation method obtains each basic indicator value from the data monitoring result table and calculates the daily year-on-year or monthly year-on-year growth;

[0064] Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value;

[0065] Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

[0066] In a preferred embodiment, this implementation uses the alert_period field to filter the alarm configuration information that meets the current monitoring frequency;

[0067] For 30-minute frequency alarms, the filter condition is: alert_period = '30mi';

[0068] For hourly frequency alarms, the filter condition is: alert_period = '1h';

[0069] For daily frequency alarms, the filter condition is: alert_period = '1d';

[0070] For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

[0071] Example 2

[0072] Combination Figure 2 This embodiment is described as an embedded data indicator monitoring and alarm system based on a data warehouse. The system includes:

[0073] The first reading unit 10 is used to read the alarm configuration information in the monitoring alarm configuration information table;

[0074] The first parsing unit 20 is used to parse the indicator rule in the alarm configuration information and obtain the indicator rule parameters; based on the is_exsit field, determine whether the indicator to be monitored already exists; if the indicator to be monitored already exists, query the basic indicator value according to the indicator rule parameters and the spliced ​​SQL; if the indicator to be monitored does not exist, query the basic indicator value according to the SQL statement in the indicator rule;

[0075] The processing unit 30 is used to merge, sort and store the queried basic indicator values ​​into a data monitoring result table;

[0076] A calculation unit 40 is used to obtain each basic indicator value from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year;

[0077] The second reading unit 50 is used to read the alarm configuration information that meets the current monitoring frequency in the monitoring alarm configuration information table;

[0078] The second parsing unit 60 is used to perform one-by-one matching and comparison with the acquired basic indicator value based on the alarm trigger threshold rule in the alarm configuration information, and determine whether the basic indicator value meets the alarm trigger threshold range. If it meets the alarm trigger threshold range, alarm content data is generated; if it does not meet the alarm trigger threshold range, alarm content data is not generated;

[0079] The output unit 70 is used to output an alarm information table based on the generated alarm content data.

[0080] In this implementation, the alarm configuration information in the monitoring alarm configuration information table is read; the indicator rules in the alarm configuration information are parsed to obtain the indicator rule parameters; based on the is_exsit field, it is determined whether the indicator to be monitored already exists. If the indicator to be monitored already exists, the basic indicator value is queried according to the indicator rule parameters and the splicing SQL; if the indicator to be monitored does not exist, the basic indicator value is queried according to the SQL statement in the indicator rule; the queried basic indicator values ​​are merged, sorted and stored in the data monitoring result table; each basic indicator value is obtained from the data monitoring result table and the daily or monthly year-on-year comparison is calculated; the alarm configuration information that meets the current monitoring frequency is read from the monitoring alarm configuration information table; based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, one by one matching and comparing, it is determined whether the basic indicator value meets the alarm trigger threshold range, if it meets the alarm trigger threshold range, the alarm content data is generated, if it does not meet the alarm trigger threshold range, the alarm content data is not generated; based on the generated alarm content data, the alarm information table is output. This solves the existing data indicator monitoring alarm technology has certain limitations, including the data must be passed through a third-party platform, the function configuration method is relatively fixed and the flexibility is low.

[0081] In a preferred embodiment, this implementation method determines whether the indicator to be monitored already exists based on the is_exsit field. When is_exsit=1, it indicates that the indicator to be monitored already exists; when is_exsit=0, it indicates that the indicator to be monitored does not exist.

[0082] In a preferred embodiment, if the indicator to be monitored already exists, the SQL query basic indicator value is formed based on the table name parameter, dimension parameter and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field and table_indicator_name field.

[0083] In a preferred embodiment, the calculation unit 40 of this embodiment is used to obtain each basic indicator value from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year;

[0084] Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value;

[0085] Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

[0086] In a preferred embodiment, this implementation uses the alert_period field to filter the alarm configuration information that meets the current monitoring frequency;

[0087] For 30-minute frequency alarms, the filter condition is: alert_period = '30mi';

[0088] For hourly frequency alarms, the filter condition is: alert_period = '1h';

[0089] For daily frequency alarms, the filter condition is: alert_period = '1d';

[0090] For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

[0091] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An embedded data indicator monitoring and alarm method based on a data warehouse, characterized in that: The method comprises the following steps: Read the alarm configuration information in the monitoring alarm configuration information table; Parse the indicator rules in the alarm configuration information and obtain the indicator rule parameters; Based on the is_exsit field, determine whether the indicator to be monitored already exists. If the indicator to be monitored already exists, query the basic indicator value according to the indicator rule parameters and splicing SQL; If the indicator to be monitored does not exist, query the basic indicator value according to the SQL statement in the indicator rule; Merge, organize and store the basic indicator values ​​queried in the data monitoring result table; Get the values ​​of each basic indicator from the data monitoring result table and calculate the daily or monthly year-on-year comparison; Read the alarm configuration information that meets the current monitoring frequency from the monitoring alarm configuration information table; Based on the alarm trigger threshold rule in the alarm configuration information and the obtained basic indicator value, a one-to-one match is performed to determine whether the basic indicator value meets the alarm trigger threshold range. If it meets the alarm trigger threshold range, alarm content data is generated; if it does not meet the alarm trigger threshold range, no alarm content data is generated; Based on the generated alarm content data, output an alarm information table; Based on the is_exsit field, it is judged whether the indicator to be monitored already exists. When is_exsit = 1, it means that the indicator to be monitored already exists. When is_exsit=0, it means that the indicator to be monitored does not exist; If the indicator to be monitored already exists, the basic indicator value of the SQL query is formed based on the table name parameter, dimension parameter, and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field, and table_indicator_name field.

2. The embedded data indicator monitoring and alarm method based on data warehouse according to claim 1 is characterized in that: Get the values ​​of each basic indicator from the data monitoring result table and calculate the daily or monthly year-on-year comparison; Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value; Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

3. The embedded data indicator monitoring and alarm method based on data warehouse according to claim 1 is characterized in that: Use the alert_period field to filter the alarm configuration information that meets the current monitoring frequency; For 30-minute frequency alarms, the filter condition is: alert_period = '30mi'; For hourly frequency alarms, the filter condition is: alert_period = '1h'; For daily frequency alarms, the filter condition is: alert_period = '1d'; For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

4. An embedded data indicator monitoring and alarm system based on a data warehouse, characterized in that: The system comprises: The first reading unit is used to read the alarm configuration information in the monitoring alarm configuration information table; The first parsing unit is used to parse the indicator rules in the alarm configuration information and obtain the indicator rule parameters; based on the is_exsit field, determine whether the indicator to be monitored already exists. If the indicator to be monitored already exists, query the basic indicator value according to the indicator rule parameters and the spliced ​​SQL; if the indicator to be monitored does not exist, query the basic indicator value according to the SQL statement in the indicator rule; A processing unit, used to merge, organize and store the basic indicator values ​​queried into a data monitoring result table; A calculation unit, used to obtain the values ​​of each basic indicator from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year; The second reading unit is used to read the alarm configuration information that meets the current monitoring frequency in the monitoring alarm configuration information table; The second parsing unit is used to match and compare the alarm trigger threshold rule in the alarm configuration information with the obtained basic indicator value one by one, and determine whether the basic indicator value meets the alarm trigger threshold range. If it meets the alarm trigger threshold range, alarm content data is generated; if it does not meet the alarm trigger threshold range, alarm content data is not generated; An output unit, used to output an alarm information table based on the generated alarm content data; Based on the is_exsit field, it is judged whether the indicator to be monitored already exists. When is_exsit = 1, it means that the indicator to be monitored already exists; when is_exsit = 0, it means that the indicator to be monitored does not exist; If the indicator to be monitored already exists, the basic indicator value of the SQL query is formed based on the table name parameter, dimension parameter, and indicator parameter in the indicator rule parameter and the table_name field, dimension_value field, and table_indicator_name field.

5. The embedded data indicator monitoring and alarm system based on data warehouse according to claim 4 is characterized in that: The calculation unit is used to obtain the values ​​of each basic indicator from the data monitoring result table and calculate the daily year-on-year or monthly year-on-year; Daily year-on-year = (today's index value - yesterday's index value) / yesterday's index value; Month-on-month growth = (today's index value - index value for the same period last month) / index value for the same period last month.

6. The embedded data indicator monitoring and alarm system based on data warehouse according to claim 4 is characterized in that: Use the alert_period field to filter the alarm configuration information that meets the current monitoring frequency; For 30-minute frequency alarms, the filter condition is: alert_period = '30mi'; For hourly frequency alarms, the filter condition is: alert_period = '1h'; For daily frequency alarms, the filter condition is: alert_period = '1d'; For monthly frequency alerts, the filter condition is: alert_period = 'cm: current date'.

Images