Multi-dimensional index calculation method and device, equipment and storage medium

Through the multi-dimensional indicator calculation method, direct acquisition and composite calculation indicators are configured to generate different types of scheduling tasks, solving the bottleneck of indicator calculation performance in the context of big data, realizing the timeliness and integrity of indicator data, and improving query efficiency and system stability.

CN119065838APending Publication Date: 2024-12-03CHINA REALTIME DATABASE CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411120404.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the context of big data, traditional indicator calculation methods are prone to performance bottlenecks when facing complexity and diversity, which makes it difficult to guarantee the timeliness and integrity of indicator data and cannot meet users' real-time query needs.

Method used

Using multi-dimensional indicator calculation method, different types of scheduling tasks (timed, quasi-real-time, buried points, compensation) are generated by configuring direct acquisition indicators and compound calculation indicators to allocate computing resources, perform task slices and classification calculations, and pass the table Store calculation results to ensure timely updates and integrity of indicator data.

Benefits of technology

It improves computing performance, reduces computing time, meets multiple query needs of users, ensures the accuracy and reliability of indicator data, and supports real-time query and efficient query of historical data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065838A_ABST
    Figure CN119065838A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-dimensional index calculation method, device and equipment and a storage medium. The method comprises the steps that a direct collection index and a composite calculation index are configured, and an SQL for index calculation is assembled; according to the information of the index configuration and the type of the calculation demand initiated by the user, generating an index calculation scheduling task of a corresponding type; a corresponding calculation process is triggered through the index calculation scheduling task, and calculation resources are distributed; according to an index calculation scheduling task and a triggered calculation process, obtaining configuration information and a calculation SQL of an index needing to be calculated, and executing task slicing and classified calculation; monitoring an index calculation result, and after calculation succeeds, storing a new index value obtained by the calculation result in a sub-table manner; when a user needs to query, obtaining a dimension required to be queried by the user and a dimension value of the dimension, and querying an index value under the dimension value from the index value table and the index historical value table; according to the method, the calculation performance in multi-dimensional index calculation is improved, and the calculation time is shortened.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an indicator calculation method, device, equipment and storage medium, and in particular to a multi-dimensional indicator calculation method, device, equipment and storage medium, belonging to the field of big data technology. Background Art

[0002] In the context of big data, indicators refer to specific values ​​or status indicators used to measure, evaluate and analyze specific information or business conditions in big data sets. These indicators play an important role in understanding business operations, optimizing processes, improving efficiency and supporting decision-making. In addition to standardized management of indicators, how to ensure the timeliness and integrity of indicator data and meet the diverse query needs of users is an important goal of the indicator platform.

[0003] Enterprises hope to use data to guide business decisions and reduce subjective assumptions and blindness. Indicators can provide quantitative business performance and help management quickly understand business conditions, so as to make more scientific and reasonable decisions. Therefore, the timeliness and integrity of indicator data affect the efficiency of the company's business decision-making under the background of big data. However, as the complexity of indicator construction becomes higher and higher, and the data calculation link continues to lengthen, traditional temporary queries are unable to cope with the complexity and diversity of big data. In the real-time calculation process, various performance bottlenecks may be encountered, such as I / O performance bottlenecks and network performance bottlenecks. These bottlenecks will seriously affect the performance and stability of real-time indicator queries. In the context of inconsistent data sources, deep indicator compound levels, and inconsistent indicator dimensions, it is difficult to ensure the timeliness and integrity of data, and it is difficult to meet users' various query needs such as indicator detail query, aggregate query, high concurrent access, and query efficiency. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a multi-dimensional indicator calculation method, device, equipment and storage medium that can improve computing performance and reduce computing time.

[0005] Technical solution: A multi-dimensional index calculation method described in the present invention includes:

[0006] Configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information;

[0007] Generate corresponding types of indicator calculation scheduling tasks according to the indicator configuration information and the type of computing requirements initiated by the user; the indicator calculation scheduling task types include: scheduled scheduling tasks, quasi-real-time computing scheduling tasks, embedded point scheduling tasks and compensation computing scheduling tasks;

[0008] The corresponding computing process is triggered by the indicator calculation scheduling task to allocate computing resources;

[0009] According to the indicator calculation scheduling task and triggered calculation process, obtain the configuration information and calculation SQL of the indicator to be calculated, and perform task slicing and classification calculation;

[0010] Monitor the calculation results of indicators. After the calculation is successful, the new indicator values ​​obtained by the calculation results are stored in separate tables;

[0011] When the user needs to make a query, the dimension that the user needs to query and the dimension value of the dimension are obtained, and the indicator value under the dimension value is queried from the indicator value table and the indicator historical value table.

[0012] Furthermore, the configuration content of the direct purchase indicator includes: indicator name, data accuracy, indicator unit, indicator characteristics, data source information, data set information, data field, aggregation method, data format, dimension information, time dimension, screening condition, statistical range and statistical frequency;

[0013] The configuration content of the composite calculation indicator includes: indicator name, data precision, indicator unit, indicator characteristics, calculation formula, dependent indicators, data format, dimension information, time dimension, screening conditions, statistical range and statistical frequency;

[0014] The SQL for assembling the indicator calculation according to the indicator configuration information is specifically as follows:

[0015] Get the field name corresponding to the indicator and concatenate it after select;

[0016] Obtaining the indicator aggregation method and converting it into an SQL formula; the indicator aggregation method includes: latest value, maximum value, minimum value, sum, average and number;

[0017] Get the statistical period of the indicator and convert it into an SQL calculation formula based on the time dimension information of the indicator; the statistical period of the indicator includes: 180 days and before, the most recent natural week, the most recent natural month, the most recent natural quarter, as of the current day, the beginning of the year as of the current day, from midnight to the current day, the most recent fiscal year, and the most recent hour.

[0018] Get the filter conditions of the indicator, and concatenate the filter field and filter value after where;

[0019] Get the dimension information of the indicator and calculate all the permutations and combinations of the dimensions as the grouping conditions of the SQL statement.

[0020] Further, the generating of the scheduled scheduling task specifically comprises: obtaining the statistical frequency information in the indicator configuration information, classifying the indicators according to the categories of the statistical frequency information by day, month, quarter and year, and storing the Cologne expressions of various types of scheduled tasks in the database respectively, so as to calculate the corresponding classified indicator calculation tasks according to the Cologne expressions on a scheduled basis;

[0021] Generate quasi-real-time computing scheduling tasks, specifically, set a button that can be manually triggered to trigger the corresponding specified indicator computing tasks;

[0022] Generate a tracking point scheduling task, specifically, insert the tracking point code after the code block where the user adds a new indicator and / or updates an indicator, so as to trigger the indicator calculation task when the user adds a new indicator and / or updates an indicator;

[0023] Generate a compensation calculation scheduling task, specifically, set a compensation queue, the compensation queue is used to record the indicator calculation tasks that failed to calculate, and wait for the next recalculation according to a preset calculation frequency.

[0024] Furthermore, the corresponding computing process is triggered by the indicator calculation scheduling task to allocate computing resources, specifically including: timing scheduling task triggering computing process, quasi-real-time computing scheduling task triggering computing process, buried point scheduling task triggering computing process and compensation computing scheduling task triggering computing process;

[0025] The timing scheduling task trigger calculation process is specifically as follows: the daily calculation timing task is triggered at 00:00:00 every day, and all statistical frequencies are calculated as daily indicators; the monthly calculation timing task is triggered at 00:00:00 on the 1st of each month, and all statistical frequencies are calculated as monthly indicators; the quarterly calculation timing task is triggered at 00:00:00 on the first day of each quarter, and all statistical frequencies are calculated as quarterly indicators; the annual calculation timing task is triggered at 00:00:00 on January 1 of each year, and all statistical frequencies are calculated as annual indicators;

[0026] The quasi-real-time computing scheduling task triggers the computing process, specifically: the user selects one or more indicators that need to be calculated immediately, manually triggers the specified indicator calculation process, adds the calculation task of the indicator to the message queue and broadcasts it, and the currently idle service obtains the distributed lock of the calculation task to perform the calculation;

[0027] The tracking point scheduling task triggers the calculation process, specifically inserting the tracking point code after the code block where the user adds a new indicator and / or updates an indicator. When the user adds a new indicator and / or updates an indicator, the tracking point calculation task is triggered, and the calculation task of the indicator is added to the message queue and broadcasted. The currently idle service will obtain the distributed lock of the calculation task to perform calculations;

[0028] The compensation calculation scheduling task triggers the calculation process, obtains the indicators to be recalculated in the compensation queue according to the preset calculation frequency, and performs calculation.

[0029] Furthermore, the calculation process of scheduling tasks and triggering according to the indicator calculation, obtaining the configuration information and calculation SQL of the indicator to be calculated, and performing task slicing and classification calculation specifically includes:

[0030] Slice the indicator calculation scheduling tasks that need to be calculated, group the preset number of indicators, obtain the calculation SQL of each indicator, and perform batch calculation;

[0031] When the calculated indicator is a sub-indicator, after the sub-indicator is calculated, the parent indicator of the sub-indicator is queried, and the indicator value of the parent indicator is calculated and updated.

[0032] Furthermore, after the calculation is successful, the new indicator value obtained by the calculation result is stored in a separate table. Specifically, after the indicator calculation task is successfully calculated, the indicator value of the indicator is stored in a preset indicator value storage table; if the historical value of the indicator already exists in the indicator value storage table, the historical value is dumped to the preset indicator historical value table.

[0033] Furthermore, the monitoring indicator calculation result also includes, when the calculation fails, adding the failed indicator calculation task to the compensation calculation scheduling task.

[0034] Based on the same inventive concept, the present invention also provides a multi-dimensional index calculation device, comprising:

[0035] The indicator configuration module is used to configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information;

[0036] The task generation module is used to generate various corresponding types of indicator calculation scheduling tasks according to the indicator configuration information and the type of calculation requirements initiated by the user; the indicator calculation scheduling task types include: timed scheduling tasks, quasi-real-time calculation scheduling tasks, embedded point scheduling tasks and compensation calculation scheduling tasks;

[0037] The computing scheduling module is used to schedule tasks through indicator computing and trigger the corresponding computing process to allocate computing resources;

[0038] The indicator calculation module is used to calculate the scheduling tasks and triggered calculation processes according to the indicators, obtain the configuration information and calculation SQL of the indicators to be calculated, and perform task slicing and classification calculations;

[0039] The monitoring and processing module is used to monitor the calculation results of the indicators. After the calculation is successful, the new indicator value obtained by the calculation result is stored in a separate table;

[0040] The indicator query module is used to obtain the dimension that the user needs to query and the dimension value of the dimension when the user needs to query, and query the indicator value under the dimension value from the indicator value table and the indicator historical value table.

[0041] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multi-dimensional indicator calculation method described in any one of the above items are implemented.

[0042] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the multi-dimensional indicator calculation method described in any one of the above items.

[0043] Beneficial effects: Compared with the prior art, the present invention can flexibly deal with complex scenarios such as inconsistent data sources, deep composite levels of indicators, and inconsistent indicator dimensions by configuring direct acquisition indicators and composite calculation indicators, thereby ensuring the accuracy and flexibility of indicator calculation; through a multi-type task scheduling mechanism, computing tasks are reasonably allocated to computing resources, thereby achieving balanced use of resources and maximum utilization; at the same time, through task slicing and classified calculations, computing performance is improved and computing time is reduced; through various methods such as scheduled tasks, embedded point calculation tasks, and quasi-real-time calculation tasks, timely updating and integrity of indicator data are ensured, meeting users' high requirements for data timeliness; users are supported to perform various query requirements such as indicator detail query and aggregation query, thereby improving the convenience and efficiency of user queries; at the same time, by storing historical data in separate tables, the query efficiency of the latest data is improved; through the compensation calculation mechanism of failed calculation tasks and the function of recursively triggering the parent indicator calculation task, the reliability and stability of the calculation tasks are ensured, and data errors or missing problems caused by calculation failures are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] Enterprises hope to use data to guide business decisions and reduce subjective assumptions and blindness. Indicators can provide quantitative business performance and help management quickly understand business conditions, so as to make more scientific and reasonable decisions. Therefore, the timeliness and completeness of indicator data affect the efficiency of the company's business decision-making under the background of big data. However, as the complexity of indicator construction becomes higher and higher, and the data calculation link continues to lengthen, traditional temporary queries are unable to cope with the complexity and diversity of big data. In the real-time calculation process, various performance bottlenecks may be encountered, such as I / O performance bottlenecks, network performance bottlenecks, etc. These bottlenecks will seriously affect the performance and stability of real-time indicator queries.

[0048] As attached Figure 1 As shown, the multi-dimensional index calculation method of this embodiment includes:

[0049] Configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information;

[0050] Generate corresponding types of indicator calculation scheduling tasks according to the indicator configuration information and the type of computing requirements initiated by the user; the indicator calculation scheduling task types include: scheduled scheduling tasks, quasi-real-time computing scheduling tasks, embedded point scheduling tasks and compensation computing scheduling tasks;

[0051] The corresponding computing process is triggered by the indicator calculation scheduling task to allocate computing resources;

[0052] According to the indicator calculation scheduling task and triggered calculation process, obtain the configuration information and calculation SQL of the indicator to be calculated, and perform task slicing and classification calculation;

[0053] Monitor the calculation results of indicators. After the calculation is successful, the new indicator values ​​obtained by the calculation results are stored in separate tables;

[0054] When the user needs to make a query, the dimension that the user needs to query and the dimension value of the dimension are obtained, and the indicator value under the dimension value is queried from the indicator value table and the indicator historical value table.

[0055] The failed calculation task is added to the compensation queue of the compensation calculation scheduling task and waits for recalculation.

[0056] In this way, the bottleneck problem encountered in real-time calculation and query of indicators can be better solved. Through various scheduling tasks, the indicator values ​​can be pre-calculated and stored, ensuring the timeliness and integrity of the data while improving the performance and stability of the system to meet business needs.

[0057] Specifically, create a new direct purchase indicator A, generate an indicator number SG24_1001, fill in the indicator name (A), indicator type (direct purchase indicator), data precision (integer), indicator unit (points), indicator characteristics (positive indicator), data source information, data set information, data field (security level), aggregation method (maximum value), data format (ordinary value), time dimension (creation time), dimension information (province code, department code), screening information (province code is 02, 03), statistical interval (as of the day), and statistical frequency (monthly).

[0058] Similarly, create a new direct purchase indicator B, generate an indicator number SG24_1002, fill in the indicator name (B), indicator type (direct purchase indicator), data precision (integer), indicator unit (points), indicator characteristics (positive indicator), data source information, data set information, data field (responsibility frequency support and guarantee capability), aggregation method (maximum value), data format (ordinary value), time dimension (creation time), dimension information (province code, department code), screening information (province code is 02, 03), statistical interval (as of the day), and statistical frequency (monthly).

[0059] Create a new composite calculation indicator C, generate the indicator number SG24_1003, fill in the indicator name (C), indicator type (composite calculation indicator), data precision (integer), indicator unit (points), indicator characteristics (positive indicator), calculation formula (A+B), data format (ordinary value), time dimension (creation time), public dimension (province code, department code), statistical interval (as of today), and statistical frequency (monthly).

[0060] According to the indicator configuration information, configure the indicator calculation SQL:

[0061] Directly collected indicator A: The data field of indicator A is security level, and the English name of the corresponding field in the database is clomn1, and the aggregation method is maximum value. Therefore, the SQL statement generated starts with select MAX(clomn1); according to the dataset information, the source table name of the indicator value is obtained, which is dataset1 here, and the SQL statement from dataset1 is spliced; according to the screening information that "province code is 02, 03", the SQL statement where province_code in(02,03) is spliced; according to the statistical interval as of the day and the time field as the creation time, the where condition of the SQL statement and curdate()>date(create_time) is continued to be spliced; according to the dimension information of province code and department code, the SQL grouping statement group by province_code,dep_code is spliced; therefore, the final indicator calculation SQL is obtained as select MAX(clomn1)from dataset1whereprovince_code in(02,03)and curdate()>date(create_time)group by province_code,dep_code;

[0062] Indicator B: The data field of indicator B is the responsibility frequency support and assurance capability. The English name of the corresponding field in the database is clomn2, and the aggregation method is the maximum value. Therefore, the SQL statement generated starts with select MAX(clomn2); according to the dataset information, the source table name of the indicator value is obtained, which is dataset2 here, and the SQL statement from dataset2 is spliced; according to the screening information that "province codes are 02, 03", the SQL statement where province_code in(02,03) is spliced; according to the statistical interval as of the current day and the time field as the creation time, the where condition of the SQL statement and curdate()>date(create_time) is continued to be spliced; according to the dimension information of province code and department code, the SQL grouping statement group byprovince_code,dep_code is spliced; therefore, the final indicator calculation SQL is obtained as select MAX(clomn2)fromdataset2where province_code in(02,03)and curdate()>date(create_time)group byprovince_code,dep_code;

[0063] Indicator C: Indicator C is a composite calculation indicator with the calculation formula A+B. It is necessary to obtain the indicator values ​​of indicators A and B from the indicator result table for calculation. Therefore, it is necessary to query the indicator values ​​of indicators A and B first. Take the SQL statement for querying indicator A as an example: select indicator_value fromindicator_value_definition where indicator_code="SG24_1001"group by dimension_value, where indicator_value is the indicator value column, indicator_value_definition is the indicator value table, indicator_code is the indicator code, and dimension_value is the indicator dimension value. The values ​​of indicators A and B obtained by the query are grouped and calculated according to the common dimension information to obtain the indicator values ​​of each dimension of indicator C.

[0064] Generate the indicator calculation scheduled scheduling task based on the indicator configuration information, including the statistical period, statistical date, etc.:

[0065] In this embodiment, the statistical period of the three indicators is the same, and they are all monthly statistics. Therefore, the three indicators are bound to the monthly calculation scheduling task, and the cron expression is 0 0 0 1*?.

[0066] According to the indicator calculation task scheduling information, the indicator calculation process is triggered:

[0067] In this embodiment, the scheduled task cycles of the three indicators are all executed on a monthly basis. Therefore, at 00:00:00 on the 1st of each month, the calculation tasks of the three indicators are automatically triggered, the corresponding calculation SQL is obtained, and the indicator values ​​of the three indicators are calculated;

[0068] In this embodiment, if the calculation of the three indicators fails due to an unexpected situation, the failed task will be recorded in the compensation calculation queue, and the compensation calculation scheduling task will schedule the indicator to be recalculated at the 0th minute of each hour. If the calculation is successful, it will be deleted from the compensation calculation queue, otherwise it will wait for the next compensation calculation;

[0069] As an optional implementation, the tracking point calculation scheduling task can be triggered when creating and editing indicators. In this embodiment, when new indicators A, B, and C are created, the calculation process is immediately executed to calculate the indicator values ​​of these three indicators;

[0070] As an optional implementation, a manual quasi-real-time computing scheduling task triggered by a button can be set. In this embodiment, any one or more of indicators A, B, and C can be selected for quasi-real-time computing. The computing task of the indicator can be added to the message queue and broadcast through the message queue. The currently idle service will obtain the distributed lock of the computing task and perform the calculation.

[0071] Slice the indicator task stream, obtain the indicator calculation SQL for calculation, and the failed calculation tasks enter the compensation calculation queue and wait for the compensation calculation scheduling task to be recalculated:

[0072] In this embodiment, all the indicator calculation scheduling tasks received by the indicator calculation process are sliced, with 100 indicators as a group, and the calculation SQL of each indicator is obtained for batch calculation, and the tasks that failed to be calculated are added to the compensation calculation queue;

[0073] As an optional implementation, after the calculation of the sub-indicator is completed, the parent indicator of the indicator is automatically queried, and the indicator value of the parent indicator is calculated and updated to ensure the timeliness and integrity of the data. In this embodiment, after the calculation task of indicator A or indicator B is completed, the indicator value of indicator C is automatically calculated.

[0074] After the calculation is successful, the new indicator value is stored:

[0075] In this embodiment, the common dimensions of the three indicators are the province code and the unit code, where the province codes are 02 and 03, and the department codes are 101 and 102. Since the province code is set as a filtering condition, there are 6 cases of dimension grouping. A total of 18 indicator data are calculated for the three indicators, as shown in Table 1.

[0076] Table 1 Index value results table

[0077]

[0078]

[0079] Query the indicator value according to the dimension conditions selected by the user:

[0080] In this embodiment, the dimension and dimension value combination of the indicator selected by the user is obtained, and the indicator value table is queried and returned to the user. For example, if the user queries indicator A, the province code is selected as 02, and the indicator value obtained by the query is 96; if the user queries indicator C, the province code is selected as 03 and the department code is 101, and the indicator value obtained by the query is 135.

[0081] Based on the same inventive concept, this embodiment also provides a multi-dimensional index calculation device, including:

[0082] The indicator configuration module is used to configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information;

[0083] The task generation module is used to generate the corresponding type of indicator calculation scheduling tasks according to the indicator configuration information and the type of calculation requirements initiated by the user;

[0084] The computing scheduling module triggers the corresponding computing process through the indicator computing scheduling task and allocates computing resources;

[0085] The indicator calculation module is used to calculate the scheduling tasks and triggered calculation processes according to the indicators, obtain the configuration information and calculation SQL of the indicators to be calculated, and perform task slicing and classification calculations;

[0086] The monitoring and processing module is used to monitor the calculation results of the indicators. After the calculation is successful, the new indicator value obtained by the calculation result is stored in a separate table;

[0087] The indicator query module is used to obtain the dimension that the user needs to query and the dimension value of the dimension when the user needs to query, and query the indicator value under the dimension value from the indicator value table and the indicator historical value table.

[0088] The indicator configuration module is a module for configuring direct-collection indicators and compound-calculation indicators. It provides a visual interface for configuring indicator name, data precision, indicator unit and other attribute information. This module includes four sub-modules: data source configuration, direct-collection indicator configuration, compound-calculation indicator configuration, and calculation SQL assembly.

[0089] Next, the above submodules are described:

[0090] The data source configuration submodule configures the data source information of the direct acquisition indicator, including the link address, port number, database name, user name, and password. The direct acquisition indicator obtains the target data source for the SQL execution by binding to the configured data source.

[0091] The direct acquisition indicator configuration submodule configures the direct acquisition indicator attributes, including: indicator name, data precision, indicator unit, indicator characteristics, data source information, data set information, data field, aggregation method, data format, dimension information, time dimension, screening conditions, statistical range, and statistical frequency;

[0092] The composite calculation indicator configuration submodule configures the composite calculation indicator properties, including: indicator name, data precision, indicator unit, indicator characteristics, calculation formula, dependent indicators, data format, dimension information, time dimension, filtering conditions, statistical range, and statistical frequency.

[0093] The calculation SQL assembly submodule assembles the calculation SQL by reading the configuration information of the indicator, which depends on the data field, aggregation method, time dimension, filtering conditions and other attributes of the indicator.

[0094] The task generation module is used to generate various corresponding types of indicator calculation scheduling tasks according to the indicator configuration information and the type of calculation requirements initiated by the user; the indicator calculation scheduling task types include: timed scheduling tasks, quasi-real-time calculation scheduling tasks, embedded point scheduling tasks and compensation calculation scheduling tasks;

[0095] The generating of the scheduled scheduling task specifically comprises: obtaining the statistical frequency information in the indicator configuration information, classifying the indicators according to the categories of the statistical frequency information by day, month, quarter and year, and storing the Cologne expressions of various types of scheduled tasks in the database respectively, so as to calculate the corresponding classified indicator calculation tasks according to the Cologne expressions on a scheduled basis;

[0096] Generate quasi-real-time computing scheduling tasks, specifically, set a button that can be manually triggered to trigger the corresponding specified indicator computing tasks;

[0097] Generate a tracking point scheduling task, specifically, insert the tracking point code after the code block where the user adds a new indicator and / or updates an indicator, so as to trigger the indicator calculation task when the user adds a new indicator and / or updates an indicator;

[0098] Generate a compensation calculation scheduling task, specifically, set a compensation queue, the compensation queue is used to record the indicator calculation tasks that failed to calculate, and wait for the next recalculation according to a preset calculation frequency.

[0099] The calculation scheduling module is a module for indicator calculation scheduling tasks. This module is responsible for allocating computing resources through indicator calculation scheduling tasks and triggering corresponding computing processes. At the same time, it realizes real-time monitoring of tasks and abnormal warnings to ensure efficient and stable indicator calculation. This module includes four sub-modules: timed scheduling tasks, quasi-real-time calculation scheduling tasks, embedded point scheduling tasks, and compensation calculation scheduling tasks.

[0100] Next, the above submodules are described:

[0101] The scheduled scheduling task submodule is divided into daily scheduled tasks, monthly scheduled tasks, quarterly scheduled tasks and annual scheduled tasks. According to the statistical frequency information of each indicator in the indicator configuration module, the indicators are allocated to four types of scheduled tasks. The daily calculation scheduled task is triggered at 00:00:00 every day, and all indicators with daily statistical frequencies are calculated; the monthly calculation scheduled task is triggered at 00:00:00 on the 1st of each month, and all indicators with monthly statistical frequencies are calculated; the quarterly calculation scheduled task is triggered at 00:00:00 on the first day of each quarter, and all indicators with quarterly statistical frequencies are calculated; the annual calculation scheduled task is triggered at 00:00:00 on January 1 of each year, and all indicators with annual statistical frequencies are calculated.

[0102] The quasi-real-time computing scheduling task module executes the user's request to immediately calculate the indicator value through the message queue.

[0103] The tracking point scheduling task module monitors the indicator configuration module, adds and edits indicators, and triggers indicator calculation tracking points and calculates indicator values ​​when the operations occur.

[0104] The compensation calculation scheduling task module performs calculations at 00:00 every hour and schedules the indicators waiting for compensation calculation in the indicator compensation calculation queue to the indicator calculation module for calculation.

[0105] The indicator calculation module is a module that executes indicator calculation tasks and calculates indicator values. This module obtains the calculation SQL of the indicator to be calculated and calculates through indicator slicing. This module includes three sub-modules: calculation task flow slicing, multi-dimensional direct acquisition indicator calculation, and multi-dimensional composite indicator calculation.

[0106] Next, the above submodules are described:

[0107] The calculation task flow slicing submodule slices the indicator calculation scheduling tasks dispatched by all current task scheduling systems. The number of indicators contained in each slice can be set by configuring parameters. By slicing, the calculation efficiency and success rate of indicators can be improved.

[0108] The multi-dimensional direct-purchase index calculation submodule obtains the configuration information of the direct-purchase index and takes data from the data source for calculation. All configured dimensions are fully arranged and grouped to ensure data integrity.

[0109] The multi-dimensional composite calculation indicator submodule calculates by obtaining the direct purchase indicator value in the indicator value storage module. According to the composite indicator calculation formula set in the indicator configuration module, all sub-indicators are grouped according to the common dimensions, and the composite calculation indicator value under each common dimension is calculated.

[0110] The monitoring and processing module is a module for monitoring the calculation results of indicators, including two sub-modules: the storage of the current value of the indicator and the storage of the historical value of the indicator.

[0111] Next, the above submodules are described:

[0112] The indicator current value storage module stores the latest indicator value obtained after the indicator calculation module successfully calculates. In the current value storage module for all indicators, each dimension has only one indicator value, which is convenient for improving the efficiency of users querying the latest indicator value.

[0113] The indicator historical value storage module stores the indicator values ​​that are expired and replaced in the indicator current value storage module.

[0114] The indicator query module is a module for users to query indicator values. The user sends a query request, which includes the queried indicator and the dimension of the indicator value to be queried. After receiving the request, the module queries the indicator value storage module for the corresponding indicator value and returns it to the user. This module includes a submodule for multi-dimensional compound query.

[0115] Based on the same inventive concept, this embodiment also provides a computing device, including: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multi-dimensional indicator calculation method described in any one of the above items are implemented.

[0116] Based on the same inventive concept, see Figure 2 , which is a schematic diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 300 stores a computer program 311, which includes program instructions, and when the program instructions are executed by the processor, the processor implements the steps of the above-mentioned multi-dimensional indicator calculation method.

[0117] It should be noted that in the context of the present application, a machine-readable storage medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0119] Although the subject matter has been described in a language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms for implementing the claims. Although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0120] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present application is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept.

Claims

1. A multi-dimensional index calculation method, characterized in that: include: Configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information; Generate corresponding indicator calculation scheduling tasks based on the indicator configuration information and the type of computing requirements initiated by the user; The index calculation scheduling task types include: scheduled scheduling tasks, quasi-real-time calculation scheduling tasks, embedded point scheduling tasks and compensation calculation scheduling tasks; The corresponding computing process is triggered by the indicator calculation scheduling task to allocate computing resources; According to the indicator calculation scheduling task and triggered calculation process, obtain the configuration information and calculation SQL of the indicator to be calculated, and perform task slicing and classification calculation; Monitor the calculation results of indicators. After the calculation is successful, the new indicator values ​​obtained by the calculation results are stored in separate tables; When the user needs to make a query, the dimension that the user needs to query and the dimension value of the dimension are obtained, and the indicator value under the dimension value is queried from the indicator value table and the indicator historical value table.

2. The multi-dimensional index calculation method according to claim 1, characterized in that: The configuration content of the direct purchase indicator includes: indicator name, data accuracy, indicator unit, indicator characteristics, data source information, data set information, data field, aggregation method, data format, dimension information, time dimension, screening condition, statistical scope and statistical frequency; The configuration content of the composite calculation indicator includes: indicator name, data precision, indicator unit, indicator characteristics, calculation formula, dependent indicators, data format, dimension information, time dimension, screening conditions, statistical range and statistical frequency; The SQL for assembling the indicator calculation according to the indicator configuration information is specifically as follows: Get the field name corresponding to the indicator and concatenate it after select; Obtaining the indicator aggregation method and converting it into an SQL formula; the indicator aggregation method includes: latest value, maximum value, minimum value, sum, average and number; Get the statistical period of the indicator and convert it into an SQL calculation formula based on the time dimension information of the indicator; the statistical period of the indicator includes: 180 days and before, the most recent natural week, the most recent natural month, the most recent natural quarter, as of the current day, the beginning of the year as of the current day, from midnight to the current day, the most recent fiscal year, and the most recent hour. Get the filter conditions of the indicator, and concatenate the filter field and filter value after where; Get the dimension information of the indicator and calculate all permutations and combinations of the dimensions as the grouping conditions of the SQL statement.

3. The multi-dimensional index calculation method according to claim 1, characterized in that: The generating of the scheduled scheduling task specifically comprises: obtaining the statistical frequency information in the indicator configuration information, classifying the indicators according to the categories of the statistical frequency information by day, month, quarter and year, and storing the Cologne expressions of various types of scheduled tasks in the database respectively, so as to calculate the corresponding classified indicator calculation tasks according to the Cologne expressions on a scheduled basis; Generate quasi-real-time computing scheduling tasks, specifically, set a button that can be manually triggered to trigger the corresponding specified indicator computing tasks; Generate a tracking point scheduling task, specifically, insert the tracking point code after the code block where the user adds a new indicator and / or updates an indicator, so as to trigger the indicator calculation task when the user adds a new indicator and / or updates an indicator; Generate a compensation calculation scheduling task, specifically, set a compensation queue, the compensation queue is used to record the indicator calculation tasks that failed to calculate, and wait for the next recalculation according to a preset calculation frequency.

4. The multi-dimensional index calculation method according to claim 1, characterized in that: The corresponding computing process is triggered by the indicator calculation scheduling task to allocate computing resources, specifically including: the timing scheduling task triggers the computing process, the quasi-real-time computing scheduling task triggers the computing process, the buried point scheduling task triggers the computing process and the compensation computing scheduling task triggers the computing process; The timing scheduling task trigger calculation process is specifically as follows: the daily calculation timing task is triggered at 00:00:00 every day, and all statistical frequencies are calculated as daily indicators; the monthly calculation timing task is triggered at 00:00:00 on the 1st of each month, and all statistical frequencies are calculated as monthly indicators; the quarterly calculation timing task is triggered at 00:00:00 on the first day of each quarter, and all statistical frequencies are calculated as quarterly indicators; the annual calculation timing task is triggered at 00:00:00 on January 1 of each year, and all statistical frequencies are calculated as annual indicators; The quasi-real-time computing scheduling task triggers the computing process, specifically: the user selects one or more indicators that need to be calculated immediately, manually triggers the specified indicator calculation process, adds the calculation task of the indicator to the message queue and broadcasts it, and the currently idle service obtains the distributed lock of the calculation task to perform the calculation; The tracking point scheduling task triggers the calculation process, specifically inserting the tracking point code after the code block where the user adds a new indicator and / or updates an indicator. When the user adds a new indicator and / or updates an indicator, the tracking point calculation task is triggered, and the calculation task of the indicator is added to the message queue and broadcasted. The currently idle service will obtain the distributed lock of the calculation task to perform calculations; The compensation calculation scheduling task triggers the calculation process, obtains the indicators to be recalculated in the compensation queue according to the preset calculation frequency, and performs calculation.

5. The multi-dimensional index calculation method according to claim 1, characterized in that: The calculation process of scheduling tasks and triggering according to the indicator calculation, obtaining the configuration information and calculation SQL of the indicator to be calculated, and performing task slicing and classification calculation specifically includes: Slice the indicator calculation scheduling tasks that need to be calculated, group the preset number of indicators, obtain the calculation SQL of each indicator, and perform batch calculation; When the calculated indicator is a sub-indicator, after the sub-indicator is calculated, the parent indicator of the sub-indicator is queried, and the indicator value of the parent indicator is calculated and updated.

6. The multi-dimensional index calculation method according to claim 1, characterized in that: The monitoring indicator calculation result, after the calculation is successful, the new indicator value obtained by the calculation result is stored in a separate table. Specifically, after the indicator calculation task is successfully calculated, the indicator value of the indicator is stored in a preset indicator value storage table; if the historical value of the indicator already exists in the indicator value storage table, the historical value is dumped to the preset indicator historical value table.

7. The multi-dimensional index calculation method according to claim 1, characterized in that: The monitoring indicator calculation result also includes, when the calculation fails, adding the failed indicator calculation task to the compensation calculation scheduling task.

8. A multi-dimensional index calculation device, characterized in that: include: The indicator configuration module is used to configure direct acquisition indicators and composite calculation indicators, and assemble the SQL for indicator calculation based on the indicator configuration information; The task generation module is used to generate indicator calculation scheduling tasks of corresponding types according to the indicator configuration information and the type of calculation requirements initiated by the user; The index calculation scheduling task types include: scheduled scheduling tasks, quasi-real-time calculation scheduling tasks, embedded point scheduling tasks and compensation calculation scheduling tasks; The computing scheduling module is used to schedule tasks through indicator computing and trigger the corresponding computing process to allocate computing resources; The indicator calculation module is used to calculate the scheduling tasks and triggered calculation processes according to the indicators, obtain the configuration information and calculation SQL of the indicators to be calculated, and perform task slicing and classification calculations; The monitoring and processing module is used to monitor the calculation results of the indicators. After the calculation is successful, the new indicator value obtained by the calculation result is stored in a separate table; The indicator query module is used to obtain the dimension that the user needs to query and the dimension value of the dimension when the user needs to query, and query the indicator value under the dimension value from the indicator value table and the indicator historical value table.

9. A computing device, characterized in that include: One or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the multidimensional indicator calculation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the multi-dimensional indicator calculation method according to any one of claims 1 to 7.