Index calculation method and device based on data table and medium

Abstract

The invention provides an index calculation method and device based on a data table and a medium, and relates to the technical field of data processing. The method comprises the steps of determining a target data table for index calculation based on a to-be-calculated index; performing event monitoring on the target data table within a preset monitoring time range; if it is monitored that an event occurs in the target data table, verifying the target data table based on a preset verification rule; and if the verification is passed, a notification of downstream index calculation is triggered. The operation and maintenance cost can be reduced, the notification of downstream index calculation can be triggered in time, and the effectiveness of index calculation is improved.

Description

Background Art

[0002] With the development of technology, the indicator calculation model based on big data needs to meet the business requirements for indicator calculation accuracy and performance.

[0003] In related technologies, in the scenario of real-time calculation of indicators, a lot of external data will be relied on. This part of data may come from the offline calculation results of big data, or from the recommendation of intelligent algorithms, or from the manual import of business personnel. Only when the underlying data is fully imported into the system in the form of data tables can the relevant indicators be processed and calculated. However, the above method cannot trigger downstream calculations in time, and is prone to problems such as data missing interruptions and high maintenance costs.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a data table-based indicator calculation method, device and medium, which at least to a certain extent overcome the problems in related technologies of being unable to trigger downstream calculations in a timely manner, prone to data missing interruptions and high maintenance costs.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, there is provided a method for calculating an indicator based on a data table, comprising: determining a target data table for indicator calculation based on the indicator to be calculated; monitoring events on the target data table within a preset monitoring time limit; if an event is detected to occur in the target data table, verifying the target data table based on preset verification rules; if the verification passes, triggering a notification for downstream indicator calculation.

[0008] In one embodiment of the present disclosure, after monitoring the target data table for events within the preset monitoring time limit, the method further includes: if no event is detected in the target data table within the preset monitoring time limit, generating a first alarm message.

[0009] In one embodiment of the present disclosure, the preset verification rules include access verification rules, and the access verification rules are used to verify the compliance of the data in the target data table; wherein, the verification of the target data table based on the preset verification rules includes: performing access verification on the data in the target data table based on the access verification rules; if the access verification passes, saving the status of the target data table.

[0010] In one embodiment of the present disclosure, the preset verification rules also include timeliness verification rules; wherein, the method also includes: if the access verification fails, then based on the timeliness verification rules, performing timeliness verification on the target data table; if the timeliness verification fails, then generating a second alarm message.

[0011] In one embodiment of the present disclosure, the preset verification rules include the dependency verification rules; wherein, the verification of the target data table based on the preset verification rules includes: performing dependency verification on the target data table according to the preset verification rules and the status of the target data table; if the dependency verification passes, executing the operation of triggering the notification of downstream indicator calculation.

[0012] In one embodiment of the present disclosure, the preset verification rules also include timeliness verification rules; wherein, the method further includes: if the dependency verification fails, then based on the timeliness verification rules, performing timeliness verification on the target data table; if the timeliness verification fails, then generating a third alarm message.

[0013] In one embodiment of the present disclosure, the access check includes at least one of a data missing check, a data type check, a data boundary check, a data item number check, and a data row number check.

[0014] In one embodiment of the present disclosure, each of the target data tables is configured with multiple check fields, and each check field is configured with multiple access check rules.

[0015] According to another aspect of the present disclosure, there is provided an indicator calculation device based on a data table, including: a monitoring module, for determining a target data table for indicator calculation based on the indicator to be calculated; performing event monitoring on the target data table within a preset monitoring time limit; a verification module, for verifying the target data table based on preset verification rules if an event is detected in the target data table; and a notification module, for triggering a notification for downstream indicator calculation if the verification passes.

[0016] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned data table-based indicator calculation method by executing the executable instructions.

[0017] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned indicator calculation method based on the data table is implemented.

[0018] According to another aspect of the present disclosure, a computer program product is provided, which includes a computer program or computer instructions, and the computer program or the computer instructions are loaded and executed by a processor to enable a computer to implement the above-mentioned data table-based indicator calculation method.

[0019] In the disclosed embodiment, based on the indicator to be calculated, a target data table for indicator calculation is determined; within a preset monitoring time limit, event monitoring is performed on the target data table; if an event is detected to occur in the target data table, the target data table is verified based on preset verification rules; if the verification passes, a notification for downstream indicator calculation is triggered, thereby establishing indicator dependencies and time requirements, reducing operation and maintenance costs, and by monitoring events on the target data table used for indicator calculation, a notification for downstream indicator calculation can be triggered in a timely manner, thereby improving the effectiveness of indicator calculation.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0022] Figure 1 A system architecture diagram of an indicator calculation method based on a data table or an indicator calculation device based on a data table provided in an embodiment of the present disclosure is shown.

[0023] Figure 2 A flow chart of an indicator calculation method based on a data table provided in an embodiment of the present disclosure is shown.

[0024] Figure 3 A flow chart of another indicator calculation method based on a data table provided in an embodiment of the present disclosure is shown.

[0025] Figure 4 A flow chart of another indicator calculation method based on a data table provided in an embodiment of the present disclosure is shown.

[0026] Figure 5 A flow chart of another indicator calculation method based on a data table provided in an embodiment of the present disclosure is shown.

[0027] Figure 6 A flow chart of an example of indicator calculation based on a data table provided in an embodiment of the present disclosure is shown.

[0028] Figure 7 A schematic diagram of the structure of an indicator calculation device based on a data table provided in an embodiment of the present disclosure is shown.

[0029] Figure 8 A schematic diagram of the structure of another indicator calculation device based on a data table provided in an embodiment of the present disclosure is shown.

[0030] Fig. 9 A schematic diagram of data table dependency provided by an embodiment of the present disclosure is shown.

[0031] Fig.10 A structural block diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0034] Figure 1 An exemplary system architecture 100 is shown to which a method for calculating an indicator based on a data table or an apparatus for calculating an indicator based on a data table according to an embodiment of the present disclosure can be applied.

[0035] like Figure 1 As shown, the system architecture 100 may include a terminal device 110 , a network 120 and a server 130 .

[0036] The network 120 is a medium for providing a communication link between the terminal device 110 and the server 130 , and may be a wired network or a wireless network.

[0037] The user may use the terminal device 110 to interact with the server 130 through the network 120 to receive or send messages.

[0038] The terminal device 110 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a display screen, and a desktop computer.

[0039] The client of the application program that can be installed on the terminal device 110 is the same, or the client of the same type of application program based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application program can also be different, for example, the application program client can be a mobile client, a PC client, etc.

[0040] Exemplarily, the terminal device 110 is installed with various communication client applications, such as medical applications, web browser applications, search applications, etc. The communication client applications can collect data sources for indicator calculation and update the data sources to the target data table.

[0041] The server 130 may be a server that provides various services, such as a background management server that supports the device operated by the user using the terminal device 110. The background management server may analyze and process the received request data (such as the monitoring task of the target data table, etc.) to obtain the processing result (such as calculating the index if the target data table is verified or generating an alarm message if the target data table is not verified).

[0042] Optionally, server 130 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0043] Those skilled in the art will know that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0044] Based on this, an embodiment of the present disclosure provides a data table-based indicator calculation method, which determines the target data table used for indicator calculation based on the indicator to be calculated; performs event monitoring on the target data table within a preset monitoring time limit; if an event is detected to occur in the target data table, the target data table is verified based on preset verification rules; if the verification passes, a notification for downstream indicator calculation is triggered, thereby establishing indicator dependency relationships and time requirements, reducing operation and maintenance costs, and by monitoring events on the target data table used for indicator calculation, a notification for downstream indicator calculation can be triggered in a timely manner, thereby improving the effectiveness of indicator calculation.

[0045] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0046] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0047] First, the embodiment of the present disclosure provides an indicator calculation method based on a data table, which can be executed by any system with computing processing capabilities. In some embodiments, the indicator calculation method based on a data table provided in the embodiment of the present disclosure can be executed by a terminal device; in other embodiments, the indicator calculation method based on a data table provided in the embodiment of the present disclosure can be executed by a server, and accordingly, the indicator calculation device based on a data table is generally set in a terminal device or a server.

[0048] Figure 2 A flow chart of an indicator calculation method based on a data table provided in an embodiment of the present disclosure is shown, such as Figure 2 As shown, the indicator calculation method based on the data table provided in the embodiment of the present disclosure includes the following steps:

[0049] S202: Based on the indicator to be calculated, determine a target data table for indicator calculation.

[0050] In one embodiment, the above-mentioned indicators to be calculated are used as quantitative measurement standards for the business, for example, population, income, number of users, profit margin, coverage, etc.

[0051] Different data are needed to calculate different indicators. For example, when calculating production capacity targets, labor efficiency data and order data are needed. The data can be stored in a data table.

[0052] It should be noted that, for an indicator to be calculated, data of at least one target data table may correspond, and one target data table corresponds to one data source. When multiple target data tables are determined according to the indicator to be calculated, the relationship between the multiple target data tables needs to be determined. The relationship between the multiple target data tables may include a dependency relationship, and, or, etc. relationship.

[0053] It should be noted that the indicators to be calculated, the target data tables corresponding to the indicators to be calculated, and the relationships between the target data tables can be pre-configured in the configuration center. Once the indicators to be calculated are determined, the corresponding target data tables and the relationships between the target data tables can be obtained.

[0054] S204: Monitor events on the target data table within a preset monitoring time limit.

[0055] In one embodiment, the preset monitoring time range is used to limit the effective time range of event monitoring. Different target data tables can correspond to the same monitoring time range or different monitoring time ranges. When corresponding to different monitoring time ranges, CPU resources can be reasonably utilized.

[0056] The preset monitoring time validity range can be pre-configured in the configuration center, and the monitoring time validity range may include the latest data arrival time and the monitoring time period, etc. The monitoring time validity range can be set according to the target data table, and the present disclosure does not make specific limitations.

[0057] In one embodiment, for a target data table, at least one event may be registered, and monitoring may be performed for each event according to a monitoring period, a monitoring frequency, and the like.

[0058] Exemplarily, for a data update event, an event is triggered when there is an update within a specified time range by executing SQL logic.

[0059] It should be noted that, in addition to using SQL, other methods that can monitor events of the target data table are also applicable, and this disclosure does not make specific limitations.

[0060] S206: If an event is detected in the target data table, the target data table is verified based on a preset verification rule.

[0061] In one embodiment, the preset verification rule is used to represent the conditions for verifying the data in the target data table, and the preset verification rule can be pre-configured through the configuration center.

[0062] The preset verification rule may include at least one of an access verification rule, a timeliness verification rule, and a dependency verification rule.

[0063] S208. If the verification passes, a notification is triggered for the downstream to perform indicator calculation.

[0064] In one embodiment, when multiple target data tables all pass the verification, it is determined that the verification is passed; when any target data table fails the verification, it is determined that the verification is failed.

[0065] When the verification passes, the downstream pipeline information corresponding to the indicator to be calculated is determined, such as the downstream pipeline identifier, and the corresponding downstream pipeline is triggered to perform indicator calculation.

[0066] In the disclosed embodiment, based on the indicator to be calculated, a target data table for indicator calculation is determined; within a preset monitoring time limit, event monitoring is performed on the target data table; if an event is detected to occur in the target data table, the target data table is verified based on preset verification rules; if the verification passes, a notification for downstream indicator calculation is triggered, thereby establishing indicator dependencies and time requirements, reducing operation and maintenance costs, and by monitoring events on the target data table used for indicator calculation, a notification for downstream indicator calculation can be triggered in a timely manner, thereby improving the effectiveness of indicator calculation.

[0067] Figure 3 Another flow chart of an indicator calculation method based on a data table provided by an embodiment of the present disclosure is shown. Figure 2 Based on the embodiment, S205 is added after S204 to limit the situation where no event occurs in the target data table. Figure 3 As shown, the index calculation method based on the data table provided in the embodiment of the present disclosure includes S202 to S204, S205, and S206 to S208. The method includes:

[0068] S205: If no event is detected in the target data table within the preset monitoring time limit, a first alarm message is generated.

[0069] It should be noted that the specific implementation of S202 to S208 of this embodiment is the same as the specific implementation of S202 to S208 of the aforementioned embodiment, and will not be repeated here.

[0070] The first alarm information includes but is not limited to the alarm reason, alarm time, etc. The alarm reason is used to indicate the cause of the alarm, for example, no event is detected in the target data table within the preset monitoring time limit, and the alarm time is used to indicate the time when the alarm is generated.

[0071] In the embodiment of the present disclosure, when no event is detected in the target data table within the preset monitoring time limit, a first alarm message is generated to promptly remind relevant personnel upstream (data source collection end) and downstream (indicator calculation end) to handle the matter.

[0072] In one embodiment, the preset verification rules include access verification rules, and the access verification rules are used to verify the compliance of data in the target data table.

[0073] Figure 4 A flowchart of another indicator calculation method based on a data table provided by an embodiment of the present disclosure is shown. Figure 2 Based on the embodiment, S206 is further refined into S2061-S2062 to save the state of the target data table that has passed the access check. Figure 4 As shown, in one embodiment, the method provided by the embodiment of the present disclosure includes S202 to S204, S2061 to S2062, and S208. The method includes:

[0074] S2061. If an event is detected in the target data table, access verification is performed on the data in the target data table based on the access verification rule;

[0075] S2062: If the access check passes, save the state of the target data table.

[0076] It should be noted that the specific implementation of S202 to S208 in the embodiment of the present disclosure is the same as the specific implementation of S202 to S208 in the aforementioned embodiment, and will not be repeated here.

[0077] The above-mentioned access check rules are used to perform access check on the target data table. The access check may include but is not limited to data missing check, data type check, data boundary check, data item number check, data row number check, etc.

[0078] In one embodiment, each target data table is configured with multiple check fields, and each check field is configured with multiple access check rules.

[0079] The status of the target data table may include verification passed, verification failed, and others. By setting the status of the target data table based on whether the access verification is passed, tasks will no longer be generated repeatedly within the event cycle, thereby improving the efficiency of the access verification.

[0080] In the disclosed embodiment, by configuring access rules, quality control is performed on external data, thereby avoiding interruptions in the computing process due to missing data and improving CPU resource utilization.

[0081] Continue to refer Figure 4 In one embodiment, the preset verification rule also includes an aging verification rule. The indicator calculation method based on the data table provided by the embodiment of the present disclosure also includes:

[0082] S2063. If the access check fails, then based on the timeliness check rule, perform timeliness check on the target data table;

[0083] S2064: If the time validity check fails, a second alarm message is generated.

[0084] In one embodiment, the timeliness check rule is used to perform timeliness check on the target data table, and the timeliness check may include at least one of monitoring period check, latest data arrival time, monitoring time period, etc.

[0085] The second alarm information includes but is not limited to the alarm reason, alarm time, etc. The alarm reason is used to indicate the reason for the alarm, such as the admission check failed and the timeliness check failed. The alarm time is used to indicate the time when the alarm is generated.

[0086] In one embodiment, if the admission check fails but the timeliness check passes, the task is terminated.

[0087] In the disclosed embodiment, when the access check fails, the target data table continues to be checked for timeliness. If the timeliness check still fails, a second alarm message is generated to promptly remind upstream and downstream related personnel to handle it.

[0088] Figure 5 A flow chart of another indicator calculation method based on a data table provided by an embodiment of the present disclosure is shown. Figure 4 Based on the embodiment, S206 further includes S2065 to verify the dependency relationship of the target data table. In one embodiment, the preset verification rule includes a dependency verification rule. Figure 5 As shown, the method also includes:

[0089] S2065, performing dependency verification on the target data table according to preset verification rules and the state of the target data table;

[0090] S2082. If the dependency check passes, the operation of triggering notification for downstream indicator calculation is executed.

[0091] Dependency verification rules are used to perform dependency verification on the relationships between multiple target data tables.

[0092] Optionally, the preset verification rule also includes an aging verification rule; wherein the method further includes:

[0093] S2066. If the dependency check fails, then based on the timeliness check rule, perform timeliness check on the target data table;

[0094] S2067: If the time validity check fails, a third alarm message is generated.

[0095] It should be noted that the aging verification rules and aging verification here are similar to those in the aforementioned embodiment and will not be repeated here.

[0096] The third alarm information includes but is not limited to the alarm reason, alarm time, etc. The alarm reason is used to indicate the reason for the alarm, such as failure of access check, failure of dependency check and failure of timeliness check. The alarm time is used to indicate the time when the alarm is generated.

[0097] In one embodiment, if the admission check fails, the dependency check fails, but the timeliness check passes, the task is terminated.

[0098] In the disclosed embodiment, when the access check passes and the dependency check passes, the target data table continues to be time-effectively checked. If the time-effectiveness check still fails, a third alarm message is generated to promptly remind upstream and downstream related personnel to handle it.

[0099] In order to facilitate the understanding of the embodiments of the present disclosure, Figure 6 A specific example will be described.

[0100] like Figure 6 As shown, an indicator calculation method based on a data table provided in an embodiment of the present disclosure includes:

[0101] S601: Task generation.

[0102] In this process, the target data set for indicator calculation is determined according to the indicator to be calculated.

[0103] S602, determining whether an event occurs in the target data table within the preset monitoring time limit, if yes, executing S603; if no, executing S607;

[0104] S603, determine whether the access check is passed, if passed, execute S604; if not, execute S607;

[0105] S604, save the target data table status;

[0106] S605: Determine whether the dependency check is passed. If it is passed, execute S606; if it is not passed, execute S607;

[0107] S606, notification that there is calculation, end;

[0108] S607, determine whether the time validity check has passed, if passed, then end; if not, execute S608;

[0109] S608: Alarm notification.

[0110] It should be noted that the time validity verification rules in the above-mentioned different stages may be the same or different, and the alarm information in different stages carries the corresponding alarm reasons.

[0111] Based on the same inventive concept, the present disclosure also provides an indicator calculation device based on a data table, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0112] Figure 7 FIG. 1 is a schematic diagram showing a structure of an indicator calculation device based on a data table provided by an embodiment of the present disclosure. Figure 7 As shown, the indicator calculation device based on the data table of this embodiment includes a monitoring module 710, a verification module 720 and a notification module 730.

[0113] The monitoring module 710 is used to determine the target data table for the indicator calculation based on the indicator to be calculated; and monitor the events of the target data table within the preset monitoring time limit;

[0114] The verification module 720 is used to verify the target data table based on a preset verification rule if an event occurs in the target data table;

[0115] The notification module 730 is used to trigger the notification of downstream indicator calculation if the verification passes.

[0116] In one embodiment of the present disclosure, the verification module 720 is used to generate a first alarm message after monitoring the target data table for events within a preset monitoring time limit, if no event is detected in the target data table within the preset monitoring time limit.

[0117] In one embodiment, the preset verification rules include access verification rules, which are used to verify the compliance of data in the target data table; the verification module 720 is used to perform access verification on the data in the target data table based on the access verification rules; if the access verification passes, the state of the target data table is saved.

[0118] In one embodiment, the preset verification rules also include timeliness verification rules; the verification module 720 is used to perform timeliness verification on the target data table based on the timeliness verification rules if the access verification fails; if the timeliness verification fails, a second alarm message is generated.

[0119] In one embodiment, the preset verification rules include dependency verification rules; the verification module 720 is used to perform dependency verification on the target data table according to the preset verification rules and the status of the target data table; the notification module 730 is used to execute the notification operation that triggers the downstream to perform indicator calculation if the dependency verification passes.

[0120] In one embodiment, the preset verification rules also include timeliness verification rules; the verification module 720 is used to perform timeliness verification on the target data table based on the timeliness verification rules if the dependency verification fails; if the timeliness verification fails, a third alarm message is generated.

[0121] It should be noted that the access verification rules include at least one of data missing verification, data type verification, data boundary verification, data item number verification, and data row number verification.

[0122] like Figure 8 As shown, in one embodiment, the device further includes a configuration center 740, and the configuration center 740 is used to configure multiple check fields for each target data table, and each check field is configured with multiple access check rules.

[0123] In the disclosed embodiments, based on the indicator to be calculated, a target data table for indicator calculation is determined; within a preset monitoring time limit, event monitoring is performed on the target data table; if an event is detected in the target data table, the target data table is verified based on preset verification rules; if the verification passes, a notification is triggered for downstream indicator calculation, thereby establishing indicator dependency relationships and time requirements, reducing operation and maintenance costs, and by monitoring events on the target data table used for indicator calculation, a notification for downstream indicator calculation can be triggered in a timely manner, thereby improving the effectiveness of indicator calculation.

[0124] Continue to refer Figure 8 In one embodiment, the device includes a monitoring module 710 , a verification module 720 , a notification module 730 and a configuration center 740 .

[0125] In one embodiment, the configuration center 740 uniformly maintains the underlying dependencies of the indicators, and the configuration items of the configuration center 740 include one or more of the verification name, execution rules, timeliness requirements, access rules, downstream pipelines, task types, data boundaries, data source management, and alarm rules.

[0126] The verification name is used to define the name of the verification task, which can be the file name of the target data table or a name defined by the user. The execution rule is used to configure the event monitoring period, verification period, etc. The time requirement is used to configure the deadlines of different stages. The access rules (or access verification rules) include but are not limited to data missing verification, data type verification, data boundary verification, data number verification, data row number verification, etc. The downstream pipeline is used to configure the pipeline for indicator calculation, and the downstream pipeline can be identified by text, numbers, letters, etc. The task type is used to distinguish the type of data source, so as to determine the corresponding access verification rules according to the type of data source; the data boundary is used to limit the value range or value type of the data in the target data table.

[0127] Data source management is used to maintain basic data table information, such as data source type (such as Clickhouse, ElasticSearch, Mysql, Doris, etc.), database connection URL, account, password, data table name, maintainer, maintenance time, etc. The data table is the object of event monitoring.

[0128] Data sources may include detailed models, dimension table data, Big Data Platform (BDP) data push, JSF, file import, etc.

[0129] Alarm rules are used to configure alarm conditions.

[0130] Configuration center 740 is used for event monitoring configuration, including data table, event type, latest data arrival event, monitoring cycle, monitoring frequency, SQL execution, event access, etc. For each target data table, multiple events and monitoring event cycles can be registered. For example, the big data side writes T-1 data to the target data table at 3 a.m. every day, where T is the current date. For this event, the latest data arrival event and monitoring time period can be configured. When new data arrives, the event is triggered for subsequent processing. For data update events, SQL logic can be configured to trigger events when there is an update within the specified time range.

[0131] The configuration center 740 is also used for access rule configuration. The access rule configures the inspection strategy of the data table to determine whether the data in the data table is compliant, including rule coding, data table ID, field name, whether it is non-empty, data enumeration, data missing check, data type judgment, data boundary check, data number check, data row number check, etc. It should be noted that each data table can be configured with multiple check fields, and each check field can be configured with multiple access rules. If and only if all the check fields meet the conditions, the check is judged to pass; if any check field does not meet the conditions, the check is judged to fail.

[0132] The configuration center 740 is also used for dependency configuration. The calculation of an indicator needs to rely on multiple data tables. Each data table corresponds to a data source, such as the big data offline side, the intelligent algorithm side, the real-time data side, the business side, etc. The relationship between data tables can be abstracted as dependency, and, or or other relationships.

[0133] like Fig. 9 As shown, data table 3 depends on data table 1. When an event occurs in data table 1, the status of data table 3 is paid attention to. This dependency relationship is common in the status of the main table and the sub-table. The calculation requires the data of the sub-table, but the data of the sub-table depends on the changes of the main table.

[0134] Logical table A includes data table 2 and data table 4, and there is an or relationship between data table 2 and data table 4. When an event occurs in any of the two tables and passes the verification, logical table A is determined to have passed the verification. For example, when calculating the production capacity target, the efficiency data may come from the big data side or be imported by business personnel. As long as an event occurs in one data table, downstream calculations can be performed. At this time, you can define the or relationship of multiple data tables, set the priority strategy internally, and when events occur in multiple data tables at the same time, obtain the data in the data table with the higher priority according to the priority strategy.

[0135] There is an AND relationship between logical table A and data table 3. Indicator calculation can be performed only after both data tables pass verification. For example, when calculating production capacity targets, both labor efficiency data and order data are needed.

[0136] Configuration center 740 is also used to configure the timeliness of indicators. The calculation of indicators needs to meet the timeliness of the business. For example, the experience indicator data must be calculated at 8 o'clock every morning. By configuring the calculation timeliness of indicators and the latest event of the data table, in the detection practice process, according to the timeliness requirements, timely alarms are sent to the upstream data end or relevant personnel for intervention and processing to meet the timeliness requirements.

[0137] The configuration center 740 is also used to configure the notification module 730, and the configuration content includes abnormal alarm configuration and downstream pipeline information configuration. The abnormal alarm configuration is used to notify relevant personnel through alarm information when the indicator calculation does not meet the timeliness requirements or the data verification does not meet the access rules. The downstream pipeline information configuration is used to trigger the downstream calculation by configuring the pipeline when all the target data tables of the indicator calculation pass the verification.

[0138] The monitoring module 710 mainly ensures the timeliness of data and triggers downstream actions. It is used for configuration analysis, state management and subtask generation. The monitoring module 710 is responsible for monitoring events of the target data table. When an event occurs, a task is generated and handed over to the verification module 720 for verification processing. When no event occurs within the timeliness requirement range (preset monitoring timeliness range), the notification module 730 is triggered in time for alarm processing.

[0139] The verification module 720 is used for data acquisition, rule parsing, and result merging. It is responsible for data compliance verification, receives messages from the monitoring module 710, performs data verification on the target data table, and maintains the verification status. When all target data tables meet the requirements (that is, when all target data tables pass the verification), the downstream indicator calculation is triggered. Among them, the verification includes non-empty verification, type verification, boundary verification, row number verification, enumeration verification, and time verification.

[0140] The notification module 730 is used for abnormal alarms, on-site snapshots, downstream notifications, etc., and is responsible for triggering downstream actions.

[0141] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Various aspects of the present invention may be specifically implemented in the following forms, a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as a "circuit", "module" or "system".

[0142] Refer to the following Fig.10 The electronic device 1000 according to this embodiment of the present invention will be described. Fig.10 The electronic device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0143] like Fig.10 As shown, the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010, the at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0144] The storage unit stores program codes, which can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1010 can perform the following steps: Figure 2 As shown in the figure, based on the indicator to be calculated, the target data table for indicator calculation is determined; within the preset monitoring time range, the target data table is monitored for events; if an event is detected in the target data table, the target data table is verified based on the preset verification rules; if the verification passes, the notification for indicator calculation is triggered downstream.

[0145] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202 , and may further include a read-only storage unit (ROM) 10203 .

[0146] The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205, such program modules 10205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0147] Bus 1030 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0148] The electronic device 1000 may also communicate with one or more external devices 1040 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the system, and / or any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1050. Furthermore, the system may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1060. Fig.10 As shown, the network adapter 1060 communicates with other modules of the electronic device 1000 via the bus 1030. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0149] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0150] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. In an exemplary embodiment of the present disclosure, a computer program product is also provided, which includes a computer program or a computer instruction, and the computer program or the computer instruction is loaded and executed by a processor to enable the computer to implement the steps of the method disclosed in the above embodiment.

[0151] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0152] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0153] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0154] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0155] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0156] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0157] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0158] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for calculating an indicator based on a data table, characterized in that: include: Based on the indicator to be calculated, determine the target data table for indicator calculation; Perform event monitoring on the target data table within a preset monitoring time limit; If an event is detected in the target data table, the target data table is verified based on a preset verification rule; If the verification passes, the downstream is notified to calculate the indicator.

2. The index calculation method based on the data table according to claim 1 is characterized in that: After monitoring the target data table for events within the preset monitoring time limit, the method further includes: If no event is detected in the target data table within the preset monitoring time limit, a first alarm message is generated.

3. The index calculation method based on the data table according to claim 1 is characterized in that: The preset verification rules include access verification rules, and the access verification rules are used to verify the compliance of the data in the target data table; The verifying of the target data table based on the preset verification rules includes: Based on the access verification rule, performing access verification on the data in the target data table; If the access check passes, the state of the target data table is saved.

4. The index calculation method based on the data table according to claim 3 is characterized in that: The preset verification rules also include timeliness verification rules; Wherein, the method further comprises: If the access check fails, then based on the timeliness check rule, a timeliness check is performed on the target data table; If the time validity check fails, a second alarm message is generated.

5. The index calculation method based on the data table according to claim 3 is characterized in that: The preset verification rules include dependent verification rules; The verifying of the target data table based on the preset verification rules includes: Performing dependency verification on the target data table according to the preset verification rule and the state of the target data table; If the dependency check passes, the operation of triggering the notification of the downstream indicator calculation is executed.

6. The index calculation method based on the data table according to claim 5 is characterized in that: The preset verification rules also include timeliness verification rules; Wherein, the method further comprises: If the dependency check fails, then based on the timeliness check rule, a timeliness check is performed on the target data table; If the time validity check fails, a third warning message is generated.

7. The index calculation method based on the data table according to claim 3 is characterized in that: The access verification rule includes at least one of data missing verification, data type verification, data boundary verification, data item number verification, and data row number verification.

8. The index calculation method based on the data table according to claim 7 is characterized in that: Each of the target data tables is configured with a plurality of check fields, and each check field is configured with a plurality of access check rules.

9. An electronic device, characterized in that: include: processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the indicator calculation method based on the data table as described in any one of claims 1-8 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for calculating an indicator based on a data table as described in any one of claims 1 to 8 is implemented.

Images