A multi-level index calculation system and method based on configuration

Abstract

The application provides a multi-level index calculation system and method based on configuration, and relates to the technical field of data processing, and comprises an acquisition module, a rule configuration module and an index calculation module; the acquisition module is configured with a metadata model; the acquisition module is used for acquiring a plurality of heterogeneous source data and extracting field metadata by using the metadata model; the rule configuration module is used for configuring a calculation rule for a target index based on the index type of the target index; the index calculation module is used for matching the field metadata of the plurality of heterogeneous source data according to each calculation rule, and determining the heterogeneous source data with matched field metadata as index calculation data required for calculating the target index; and in the order from high to low of the calculation priority, the statistical calculation method of the corresponding calculation rule is sequentially executed on the index calculation data to obtain index result data of the target index. The whole process dynamic generation from original data to multi-level indexes is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-level index calculation system and method based on configuration. BACKGROUND

[0002] In a big data driven information simulation scenario, efficient generation and dynamic update of situation indicators are the core link to support real-time decision-making. Traditional index calculation methods usually adopt a hierarchical processing mode: atomic indicators are extracted from heterogeneous source data through customized code, composite indicators are combined by rule engines, and derivative indicators are calculated based on preset formulas.

[0003] However, as the simulation scale expands and real-time requirements improve, the customized development of related technologies for the unique format of each data source cannot meet the new indicator demand that frequently emerges in the simulation process. Due to the field heterogeneity and real-time conflict of multi-source data, the development process requires repeated writing of data cleaning, association mapping and calculation logic code, resulting in long development cycle and high labor cost. Therefore, the traditional index configuration or calculation method cannot adapt to the dynamic changes of indicator demand in large-scale simulation scenarios, resulting in significant bottlenecks in the expansion of the indicator system. SUMMARY

[0004] The embodiments of the present application provide a multi-level index calculation system and method based on configuration, aiming at solving the problems in the above background technology.

[0005] In order to solve the above technical problems, the present application is implemented as follows:

[0006] In a first aspect, the embodiments of the present application provide a multi-level index calculation system based on configuration, which comprises a collection module, a rule configuration module and an index calculation module.

[0007] The collection module is configured with a metadata model; the collection module is used to obtain a plurality of heterogeneous source data, and extract the field metadata of each heterogeneous source data by using the metadata model;

[0008] The rule configuration module is used to configure at least one calculation rule for the target indicator based on the index type of the target indicator, and the calculation rule comprises a statistical field, a field constraint symbol, a field constraint value, a statistical calculation method and a calculation priority;

[0009] The index calculation module is configured to match field metadata of the plurality of heterogeneous source data according to a statistical field, a field constraint symbol and a field constraint value of each calculation rule, and determine, as index calculation data required for calculating the target index, the heterogeneous source data for which the matching of the field metadata is successful; and sequentially execute a statistical calculation method of the corresponding calculation rule on the index calculation data in descending order of calculation priority, to obtain index result data of the target index.

[0010] Optionally, the metadata model is configured to extract a field code, a field name and a field description of each heterogeneous source data, and take the field code, the field name and the field description of each heterogeneous source data as field metadata of the heterogeneous source data.

[0011] The index calculation module is configured to:

[0012] The field constraint value of each calculation rule is parsed according to a type of the field constraint symbol of each calculation rule, to obtain a target constraint interval of each calculation rule, and the type of the field constraint symbol includes equal to, numerical interval, contains, greater than, less than, greater than or equal to, less than or equal to and not equal to.

[0013] In the plurality of heterogeneous source data, the heterogeneous source data containing the statistical field in the field code or the field name is determined as candidate calculation data, and a data value of each candidate calculation data is determined according to a field description of each candidate calculation data.

[0014] The candidate calculation data with the data value in the target constraint interval is determined as the heterogeneous source data for which the matching of the field metadata is successful.

[0015] Optionally, the system further comprises an index definition module and an index recommendation module.

[0016] The index definition module is configured to acquire the plurality of heterogeneous source data, and generate index definition data corresponding to the target index according to the plurality of heterogeneous source data, the index definition data being configured to describe the target index in different dimensions.

[0017] The index recommendation module is configured to acquire index definition data corresponding to each of a plurality of indexes, and index result data of each of the plurality of indexes; determine an association relationship between the indexes according to the index definition data and the index result data, and output an index recommendation result representing the association relationship between the indexes.

[0018] Optionally, the index recommendation module is configured with a graph database and a graph convolution network model, each node in the graph database representing a one-to-one corresponding index, and the graph convolution network model being pre-trained according to index definition data corresponding to each of the plurality of indexes and index result data corresponding to each of the plurality of indexes.

[0019] The index definition module is configured to associate the index definition data corresponding to each of the plurality of indexes with the calculation rule corresponding to each of the plurality of indexes to obtain an associated calculation rule identifier corresponding to each of the plurality of indexes.

[0020] The index recommendation module is configured to determine an association relationship between each index according to the associated calculation rule identifier corresponding to each of the plurality of indexes, calculate an association weight between each index according to the association relationship between each index, and establish an edge between each node in the graph database based on the association weight between each index.

[0021] The index recommendation module is configured to calculate an association strength between each index according to the nodes and edges in the graph database through the graph convolution network model, determine at least one associated index set with an association strength greater than a preset threshold, and output a recommended index result including the at least one associated index set.

[0022] Optionally, the index type includes an atomic index, a composite index, and a derived index.

[0023] The rule configuration module is configured to, in a case where the index type of the target index is an atomic index, configure the target index with a first type calculation rule set including one calculation rule, and configure a statistical calculation method of the first type calculation rule set as an atomic level operator, the atomic level operator being any one of an original value, a count, a maximum value, a minimum value, and an average value.

[0024] The index calculation module is configured to, in a case where the index type of the target index is an atomic index, execute a statistical calculation method of a corresponding calculation rule on the index calculation data to obtain index result data of the target index.

[0025] Optionally, the rule configuration module is configured to, in a case where the index type of the target index is a composite index, configure the target index with a second type calculation rule set including at least two calculation rules, and configure a statistical calculation method of each calculation rule as an atomic level operator or a composite level operator, the composite level operator including at least one of addition, subtraction, multiplication, and division.

[0026] The rule configuration module is further configured to configure a corresponding calculation priority for each calculation rule in the second type calculation rule set, and bind the calculation result of the calculation rule of the previous stage calculation priority as the input data of the calculation rule of the next stage calculation priority.

[0027] The index calculation module is configured to, in a case where the index type of the target index is a composite index, sequentially execute the statistical calculation methods of the calculation rules on the index calculation data in a descending order of the calculation priorities, and cache the intermediate result data into the memory mapping structure until the index result data of the target index is generated.

[0028] Optionally, the rule configuration module is configured to, in a case where the index type of the target index is a derivative index, configure a third type calculation rule set containing at least two calculation rules for the target index, wherein the statistical calculation method of at least one calculation rule is configured as a derivative level operator, and the derivative level operator includes a de-duplication count.

[0029] The rule configuration module is further configured to configure a corresponding calculation priority for each calculation rule in the third type calculation rule set, and configure an additional field list for extracting specified additional field information from the plurality of heterogeneous source data.

[0030] The index calculation module is configured to, in a case where the index type of the target index is a derivative index, sequentially execute the statistical calculation methods of the calculation rules on the index calculation data in a descending order of the calculation priorities, and cache the intermediate result data and the additional field information in the form of key-value pairs into the memory mapping structure until the index result data of the target index is generated.

[0031] Optionally, the system further comprises a data preprocessing module, and the data preprocessing module comprises a data cleaning submodule, a data parsing submodule and a data mapping submodule.

[0032] The data cleaning submodule is configured to obtain a plurality of original heterogeneous source data from a plurality of data sources, perform data cleaning on the plurality of original heterogeneous source data, filter out abnormal value data and duplicate data in the plurality of original heterogeneous source data, and filter out non-key feature data in the plurality of original heterogeneous source data.

[0033] The data parsing submodule is configured to parse the plurality of original heterogeneous source data output by the data cleaning submodule according to a predefined format, and the predefined format includes a table format and a JSON format.

[0034] The data mapping submodule is configured to map the multiple original heterogeneous source data output by the data parsing submodule according to the data format matched by the metadata model, and generate multiple heterogeneous source data in a standardized format.

[0035] Optionally, the acquisition module is configured to acquire the multiple heterogeneous source data when it is detected that the multiple heterogeneous source data in the standardized format are generated, and extract field metadata from the multiple heterogeneous source data through the metadata model.

[0036] The acquisition module is further configured to compare the field metadata of the target indicator with the field metadata in the metadata model when it is detected that the indicator definition data corresponding to the target indicator are generated, and replace the metadata model based on the version label of the field metadata when the field metadata of the target indicator is inconsistent with the field metadata in the metadata model.

[0037] In a second aspect, an embodiment of the present application provides a multi-level indicator calculation method based on configuration, applied to the system of the first aspect, and the method comprises the following steps:

[0038] acquiring multiple heterogeneous source data, and extracting field metadata of each heterogeneous source data by using a metadata model;

[0039] configuring at least one calculation rule for the target indicator based on the indicator type of the target indicator, wherein the calculation rule comprises a statistical field, a field constraint symbol, a field constraint value, a statistical calculation method, and a calculation priority;

[0040] matching the field metadata of the multiple heterogeneous source data according to the statistical field, the field constraint symbol, and the field constraint value of each calculation rule, and determining the heterogeneous source data with matched field metadata as indicator calculation data required for calculating the target indicator;

[0041] sequentially executing the statistical calculation method of the corresponding calculation rule on the indicator calculation data in the order from high to low of the calculation priority, to obtain indicator result data of the target indicator.

[0042] The technical scheme provided by the embodiment of the present application at least brings the following beneficial effects:

[0043] The application can automatically extract field metadata of heterogeneous data sources by the metadata model built in the collection module, and map the original data to a standardized field identification system. The burden of repeatedly writing cleaning and parsing codes for different data sources in traditional methods is eliminated, so that when a new data source is added, only the metadata needs to be dynamically updated to complete the adaptation, solving the conflict between field heterogeneity and real-time performance. The rule configuration module defines a general rule model containing statistical fields, constraint conditions, calculation operators and priorities, and for the first time realizes the configuration generation of atomic indicators without customizing the development of data filtering logic. For composite indicators, dynamic association calculation between atomic indicators is supported through priority rule chain, breaking the bottleneck of fragmented multi-level indicator calculation process in traditional solutions. It can be seen that through the configuration architecture of multiple modules, the application realizes the dynamic generation of the whole process from raw data to multi-level indicators, significantly improving the real-time performance and scalability of the indicator system. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0045] Figure 1 is an architecture diagram of a multi-level indicator calculation system based on configuration provided by an embodiment of the present application;

[0046] Figure 2 is a complete application process schematic diagram of a multi-level indicator calculation system based on configuration provided by an embodiment of the present application;

[0047] Figure 3 is a step schematic diagram of a multi-level indicator calculation method based on configuration provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention. In the description of the embodiments of this invention, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. In this invention, "at least one" refers to one or more, and "more" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0049] In big data-driven simulation scenarios, dynamically changing situational indicators require rapid responses to complex and ever-changing decision-making needs. Traditional indicator calculation methods suffer from the following core defects: First, atomic indicators rely on customized development, with deep coupling between calculation logic and data sources, resulting in high development costs and poor scalability; second, composite / derived indicators lack a unified calculation framework, making it difficult to flexibly combine multi-level indicators through configuration; finally, indicators are calculated in isolation, failing to build a correlation network to mine deep data value. These defects are particularly prominent in large-scale real-time simulation scenarios, severely restricting situational awareness and decision support. The core concept of this invention lies in constructing a multi-level indicator calculation system that integrates data processing, rule decoupling, and hierarchical calculation. This invention's embodiments achieve plug-and-play functionality for multi-source data through a metadata-driven heterogeneous data dynamic adaptation mechanism. A three-level configuration rule model covering atomic, composite, and derived indicators is designed, abstracting the calculation logic into programmable rule components. A priority-based streaming calculation architecture is adopted to support multi-level derivation of complex indicators, effectively solving the technical bottlenecks of traditional methods.

[0050] Figure 1 This is an architectural block diagram of a configurable multi-level index calculation system provided in one embodiment of the present invention, such as... Figure 1 As shown, the system includes: a data acquisition module, a rule configuration module, and an indicator calculation module;

[0051] The acquisition module is configured with a metadata model; the acquisition module is used to acquire multiple heterogeneous source data and use the metadata model to extract field metadata for each heterogeneous source data.

[0052] The collection module supports real-time collection of multiple heterogeneous source data from databases, APIs, log files and other data sources. Considering that the related art has the defect of calculation ambiguity caused by field conflicts or inconsistencies, the metadata model is pre-configured in the collection module, which manages the heterogeneous source data collected by the collection module according to multiple dimensions. Specifically, the metadata model can be managed according to six dimensions of field code, field name, field description, field type, field position and data source. The field position is recorded in the form of a path to generate a unique identifier for the field, eliminating ambiguity in the calculation process.

[0053] The rule configuration module is configured to configure at least one calculation rule for the target indicator based on the indicator type of the target indicator, and the calculation rule includes a statistical field, a field constraint symbol, a field constraint value, a statistical calculation method and a calculation priority.

[0054] The calculation rule is used to indicate how to filter and calculate the indicator calculation data required by the target indicator, and to indicate the calculation logic of the target indicator. The calculation rule is pre-configured by the rule configuration module according to the specified indicator type of the target indicator. In this embodiment, the indicator type includes an atomic indicator, a composite indicator and a derived indicator. The following describes the three types of indicators:

[0055] The atomic indicator is the most basic indicator and is an indicator without cross-rule dependency, such as daily sales and daily attendance of real people. The composite indicator is an indicator generated by arithmetic operations on two or more atomic indicators, such as quarterly sales growth rate. The derived indicator is an indicator further obtained by arithmetic operations or conditional filtering on two or more atomic indicators or composite indicators, such as quarterly profit rate, which is calculated based on quarterly sales (composite indicator) and cost (atomic indicator), or customer satisfaction index, which is calculated based on multiple atomic indicators (such as customer rating, service timeliness, customer feedback, etc.), and involves weighted average or conditional filtering of different indicators.

[0056] As Figure 1As shown, the field metadata collected by the collection module is equivalent to providing a template of configuration items for the rule configuration module to indicate that the rule configuration module specifically configures the calculation rule from which level of configuration items. Each calculation rule contains configuration items of the following parts: statistical field, field constraint symbol, field constraint value, statistical calculation method and calculation priority. Among them, the statistical field refers to the specific data field that needs to be counted or calculated in the calculation process. For example, in a sales data set, the statistical field can be "sales" or "sales quantity". If you want to calculate the total sales, "sales" is the statistical field; the field constraint symbol is a symbol or keyword used to define the limit condition of the statistical field; the field constraint value is a specific value used in cooperation with the field constraint symbol to define the filtering condition. If the field constraint symbol is "greater than", the field constraint value can be "1000". When performing statistical filtering in multiple heterogeneous source data, only the orders with sales greater than 1000 will be counted; the statistical calculation method refers to the specific statistical method or algorithm used in the calculation process of the target index; the calculation priority refers to the number that determines the execution order in multiple calculation rules. The smaller the number of calculation priority, the higher the priority. If there are two rules, the priority of rule A is 1, and the priority of rule B is 2. When performing calculation, rule A will be executed first, and then rule B will be executed.

[0057] The index calculation module is configured to match the field metadata of the plurality of heterogeneous source data according to the statistical field, the field constraint symbol and the field constraint value of each calculation rule, and determine the heterogeneous source data with matched field metadata as index calculation data required for calculating the target index; and sequentially execute the statistical calculation method of the corresponding calculation rule on the index calculation data in descending order of calculation priority, to obtain index result data of the target index.

[0058] The index calculation module is configured to implement data filtering and data operation. According to the calculation rule provided by the rule configuration module, the plurality of heterogeneous source data is statistically calculated, and index result data of the target index is generated. The index calculation module first filters out the field metadata meeting the condition from the plurality of heterogeneous source data as at least one index calculation data required for calculating the target index according to the statistical field, the field constraint symbol and the field constraint value specified in the calculation rule, and the index calculation data is used as a calculation factor participating in subsequent operation.

[0059] After the data screening is completed, the index calculation module executes the statistical calculation method in the corresponding calculation rule on the at least one index calculation data according to the calculation priority from high to low in sequence according to the definition of the calculation rule, for example, performs total, average, weighted average, maximum, minimum and other operations on the index calculation data. The stream calculation based on the calculation priority not only ensures the correct order of the execution of multiple calculation rules, but also avoids calculation conflicts or errors caused by complex inter-rule dependency relationships.

[0060] After all the calculation rules are executed, the index calculation module finally generates index result data. The index result data not only includes the calculation results of each level index (atomic index, composite index and derived index), but also contains intermediate result data in the calculation process, facilitating subsequent decision support and data visualization work. The index result data can be output in a predetermined format and transmitted to a decision support system or a visualization platform for further analysis and display.

[0061] The present application can automatically extract field metadata of heterogeneous data sources through the metadata model built in the acquisition module, and uniformly map the original data to a standardized field identification system. The burden of repeatedly writing cleaning and parsing codes for different data sources in traditional methods is eliminated, so that when a new data source is added, only the metadata needs to be dynamically updated to complete the adaptation, solving the conflict between field heterogeneity and real-time performance. The rule configuration module defines a general rule model including statistical fields, constraint conditions, calculation operators and priorities, and for the first time realizes the configuration generation of atomic indexes without the need for customized development of data filtering logic. For composite indexes, dynamic association calculation between atomic indexes is supported through priority rule chains, breaking through the bottleneck of fragmented multi-level index calculation process in traditional solutions. It can be seen that the present application realizes the full-process dynamic generation from raw data to multi-level indexes through the configuration architecture of multiple modules, significantly improving the real-time performance and expandability of the index system.

[0062] In an optional embodiment, the metadata model is used to extract the field encoding, field name and field description of each heterogeneous source data, and the field encoding, field name and field description of each heterogeneous source data are taken as the field metadata of the heterogeneous source data.

[0063] In the management dimension of the metadata model, field codes, field names and field descriptions can be included. Specifically, the field code is a unique identifier of each field of the heterogeneous source data, for example, "F001", used to locate the field to avoid conflicts of fields with the same name in different data sources (such as "ID" fields in different databases may have different meanings); the field name is used to record the business name of the field (such as "sales" and "timestamp"), facilitating the user to intuitively understand the meaning of the field; the field description is used to supplement the business background, value range or calculation logic of the field (such as "timestamp format is UTC millisecond" and "sales unit is RMB"), providing semantic support for subsequent data filtering and calculation.

[0064] The index calculation module is configured to: parse the field constraint value of each calculation rule according to the type of the field constraint symbol of each calculation rule to obtain a target constraint interval of each calculation rule, and the type of the field constraint symbol includes equal, numerical interval, contains, greater than, less than, greater than or equal to, less than or equal to, and not equal to.

[0065] The type of the field constraint symbol includes equal, numerical interval, contains, greater than, less than, greater than or equal to, less than or equal to, and not equal to. The field constraint symbol and the field constraint value jointly determine the target constraint interval used for filtering index calculation data. For example, in the case of the type of the field constraint symbol being equal (=), the field constraint value is directly used as a matching condition (such as "entity classification = ship"); in the case of the type of the field constraint symbol being numerical interval (∈), the field constraint value is parsed as an interval range (such as "damage percentage ∈ (0, 20]"); and in the case of the type of the field constraint symbol being contains, the field constraint value is parsed as a set (such as "target ID contains A001, B002").

[0066] As described above, the logic of parsing the field constraint value according to the type of the field constraint symbol is to dynamically convert the format of the constraint value according to the type of the constraint symbol to obtain the target constraint interval of each calculation rule, which is equivalent to determining the matching range of each calculation rule.

[0067] In the plurality of heterogeneous source data, the heterogeneous source data containing the statistical field in the field code or the field name is determined as candidate calculation data, and the data value of each candidate calculation data is determined according to the field description of each candidate calculation data.

[0068] For example, if the statistical field is "damage percentage", all heterogeneous source data is traversed to match field codes (such as "F005") or field names (such as "damage_percent"), and the heterogeneous source data whose field code or field name matches the statistical field is determined as candidate calculation data, thereby obtaining at least one candidate calculation data. This embodiment allows the user to locate the field in any way through the code or the name, reduces the risk of configuration failure caused by inconsistent naming of data sources, and improves the success rate of field location through the mechanism of double-path matching. The present application can adapt to the naming specifications of different data sources without the need to forcibly unify the field names.

[0069] The candidate calculation data with the data value in the target constraint interval is determined as the heterogeneous source data with which the field metadata matches successfully.

[0070] For example, the field description defines that the "timestamp format is UTC milliseconds", and "1630454400000" in the candidate calculation data is parsed as the time value "2021-09-01 00:00:00". The parsed data value is compared with the target constraint interval. For example, if the statistical field is "timestamp" and the target constraint interval is [9:00, 10:00], only the candidate calculation data with the value of the timestamp in [9:00, 10:00] is retained, that is, the heterogeneous source data with which the timestamp matches successfully in [9:00, 10:00] is screened out from the at least one candidate calculation data.

[0071] Through accurate screening and effective calculation of data, the present application significantly improves the efficiency of situation awareness and decision support, and provides more accurate and timely analysis results for the user.

[0072] In an optional implementation, the system further comprises an indicator definition module and an indicator recommendation module.

[0073] The indicator definition module is configured to acquire the plurality of heterogeneous source data, and generate indicator definition data corresponding to the target indicator according to the plurality of heterogeneous source data, the indicator definition data being used to describe the target indicator in different dimensions.

[0074] Figure 2 is a complete application flowchart of a multi-level indicator calculation system based on configuration provided by an embodiment of the present application, please refer to Figure 2The index definition module is configured to generate index definition data of the target index according to the plurality of heterogeneous source data. The index definition data is structured information describing the index, aiming to provide a comprehensive and accurate description of the index. Through the configuration mode, the index definition module can dynamically generate the index definition data of each index without manually writing complex codes. The index definition data includes eight dimensions of definition items, namely, index unique code, index name, index abbreviation, measurement unit, index grouping, associated calculation rule identifier, index type, and source data. The associated calculation rule identifier represents which calculation rules the index is associated with, indicating at least one index related to the index.

[0075] The index recommendation module is configured to obtain index definition data corresponding to each of the plurality of indexes and index result data of each of the plurality of indexes; determine the association relationship between the indexes according to the index definition data and the index result data, and output an index recommendation result representing the association relationship between the indexes.

[0076] The index recommendation module is configured to generate an index recommendation result with high correlation based on the existing index definition data and index result data during the calculation and analysis of the target index, to assist users to obtain the information required for decision-making more quickly.

[0077] According to the association calculation rule identifier of the index definition data of each index, at least one calculation rule corresponding to the index is determined. According to the at least one calculation rule corresponding to the index, the calculation process of the index and the related data involved in the calculation process are determined, and the mutual association between a plurality of indexes is analyzed in turn. For example, the quarterly sales may be associated with customer satisfaction, market share and other indexes, and these association relationships are identified by analyzing the index definition data and the index result data obtained through historical calculation. Specifically, according to the association calculation rule identifier of the index definition data of each index, it is determined that the index is obtained through which calculation rule. Each calculation rule defines the relationship between indexes and the calculation order. If a derivative index (such as quarterly profit rate) depends on a plurality of atomic indexes (such as quarterly sales and cost), the index structure data of the index is directly associated with these atomic indexes. By backtracking analysis on the index definition data and the index result data obtained through historical calculation of the target index, it is identified that which indexes often change together or have statistical correlation in a certain decision-making scenario. For example, assuming that the user queries the quarterly profit rate index, according to the association calculation rule identifier, it is determined that the quarterly profit rate depends on the quarterly sales (composite index) and the cost (atomic index) for calculation. According to the index result data, it is determined that there is a strong correlation between the quarterly sales and the customer satisfaction, and there is an association relationship between the cost and the production efficiency. Therefore, the related indexes such as “quarterly sales”, “quarterly cost” and “customer satisfaction” are recommended for the quarterly profit rate index, so as to help the user further analyze these indexes.

[0078] Finally, the index recommendation result presented in the form of a table or a report is output, and all related indexes and their calculation results are clearly listed and sorted according to the association degree. Through the index recommendation result, the user can quickly identify and access other key indexes closely related to the target index, so as to realize more efficient and more comprehensive decision support.

[0079] In an optional implementation, the index recommendation module is configured with a graph database and a graph convolution network model, each node in the graph database represents a one-to-one corresponding index, and the graph convolution network model is obtained by pre-training according to the index definition data corresponding to a plurality of indexes and the index result data of the plurality of indexes.

[0080] In the index recommendation module, the graph database represents the association relationship between indexes. Each index has a one-to-one corresponding node in the graph database, and the nodes are connected through edges to represent the association relationship between them. Specifically, each index corresponds to a node in the graph database, and the attributes of the node include each definition item of the index definition data of the index.

[0081] The index definition module is configured to associate the index definition data of each of the plurality of indexes with the calculation rule corresponding thereto, to obtain an associated calculation rule identifier of each of the plurality of indexes.

[0082] As described above, the index definition module is further configured to associate the index definition data of the plurality of indexes with the calculation rule corresponding thereto, and obtain the associated calculation rule identifier of each of the indexes, the associated calculation rule identifier being one of the definition items of the index definition data.

[0083] The index recommendation module is configured to determine the association relationship between the indexes according to the associated calculation rule identifier of each of the plurality of indexes, calculate the association weight between the indexes according to the association relationship between the indexes, and establish the edge between the nodes in the graph database based on the association weight between the indexes.

[0084] The edge between the index nodes represents the association relationship therebetween. The weight of the edge reflects the correlation degree between the indexes. By means of the associated calculation rule identifier, the dependency relationship chain between the calculation rules is analyzed, the association weight between the indexes is further calculated, and the association relationship is embodied in the graph database through the weight of the edge. For example, if there is a strong association between the quarterly sales and the quarterly profit rate (for example, the calculation of the profit rate depends on the sales), there will be a connecting edge between the two nodes in the graph database, and the weight of the edge will be high.

[0085] The index recommendation module is configured to calculate the association strength between the indexes according to the nodes and the edges in the graph database by means of the graph convolution network model, determine at least one associated index set with an association strength greater than a preset threshold, and output a recommended index result including the at least one associated index set.

[0086] The graph convolution network model is a method of deep learning based on the data structure of the graph database. The graph convolution network model learns the relationship between the nodes through the graph structured data, so as to propagate the information between the nodes and make predictions and recommendations. In the embodiment of the present application, the graph convolution network model calculates the association strength between the indexes according to the nodes and the edges in the graph database. Specifically, the input of the graph convolution network model is the nodes and the edges in the graph database. Through the graph convolution network model, the interaction between the nodes in the graph is learned, the information between the nodes is propagated, and then the association strength between the indexes is calculated. The graph convolution network aggregates the information of adjacent nodes through multi-layer convolution operation, so as to enhance the relationship features between the nodes. For example, if there is an association relationship between the quarterly sales and the quarterly profit rate, the graph convolution network can determine the association strength therebetween and give a higher weight in subsequent calculation.

[0087] After the graph convolution network model calculates the correlation strength between all indicators, the correlation strength between the indicators is calculated through the operation of the graph convolution network, representing the correlation tightness between two or more indicators. The higher the correlation strength, the more reference value the interrelated indicators have in decision support. Only when the correlation strength between two or more indicators is greater than a preset threshold, it is considered that there is a significant correlation between them. Based on the above judgment, one or more correlation indicator sets are determined, each of which includes two or more indicators.

[0088] Through the indicator recommendation module, a recommended indicator result including at least one correlation indicator set is output.

[0089] The present application provides an intelligent and dynamically adaptive indicator recommendation function by combining a graph database and a graph convolution network model. It can identify the correlation between indicators according to the definition data, calculation rules and calculation results of the indicators, and make accurate indicator recommendations based on this. Not only does it improve the decision-making efficiency in a large-scale data environment, but it also provides advanced intelligent decision-making support means for various industries.

[0090] In an optional implementation, the indicator types include atomic indicators, composite indicators and derived indicators.

[0091] The rule configuration module is configured to, in the case that the indicator type of the target indicator is an atomic indicator, configure the target indicator with a first type calculation rule set containing one calculation rule, and the statistical calculation method of the first type calculation rule set is configured as an atomic level operator, and the atomic level operator is any one of original value, count, maximum value, minimum value and average value.

[0092] When the target indicator is an atomic indicator, the rule configuration module configures a first type calculation rule set containing only one calculation rule. The indicator calculation module will perform the corresponding statistical calculation method on the target indicator calculation data according to the first type calculation rule set. During the calculation process, basic atomic level operators such as original value, count, maximum value, minimum value and average value are performed on the indicator calculation data to obtain the final indicator result data. As mentioned earlier, the atomic level operator is a basic calculation unit that can quickly and efficiently process atomic indicator data. The original value is suitable for indicators that do not need to be calculated, such as the number of goods in stock; the count is suitable for count data, such as calculating the number of users accessing a website within a certain period of time; the maximum value and the minimum value are suitable for scenarios where the extreme value of the data range needs to be found, such as the highest temperature or the lowest temperature; the average value is suitable for scenarios where the data fluctuation is small and the overall trend needs to be understood, such as the average value of monthly sales.

[0093] The index calculation module is configured to, in a case where the index type of the target index is an atomic index, perform a statistical calculation method of a corresponding calculation rule on the index calculation data to obtain index result data of the target index.

[0094] The index calculation module generates the index result data of the target index by performing a corresponding calculation rule, i.e., performing a calculation rule in the first type data set on the index calculation data.

[0095] In an optional embodiment, the rule configuration module is configured to, in a case where the index type of the target index is a composite index, configure the target index with a second type calculation rule set containing at least two calculation rules, wherein a statistical calculation method of each calculation rule is configured as an atomic level operator or a composite level operator, and the composite level operator includes at least one of addition, subtraction, multiplication, and division.

[0096] In this embodiment, the rule configuration module configures the target index with a second type calculation rule set containing at least two calculation rules in a case where the target index is a composite index. The statistical calculation method of each calculation rule can be an atomic level operator or a composite level operator, which is used to realize the combination calculation of multiple atomic indexes or composite indexes, and supports more complex business requirements and analysis. The composite level operator includes basic mathematical operations such as addition (+), subtraction (-), multiplication (x), and division (÷). The composite level operator can flexibly combine multiple atomic indexes or composite indexes to calculate new index results. For example, the results of multiple atomic indexes are summed by addition, or the ratio of a certain index is calculated by division, to meet the complex calculation requirements.

[0097] The rule configuration module is further configured to configure each calculation rule in the second type calculation rule set with a corresponding calculation priority, and bind the calculation result of a calculation rule of a previous stage calculation priority as input data of a calculation rule of a next stage calculation priority.

[0098] Since multiple calculation rules are involved in the calculation process of a composite index, the execution order of the calculation rules also needs to be strictly specified. The rule configuration module further configures each calculation rule with a calculation priority, which indicates the execution order of each calculation rule in the calculation process. The calculation priority of each calculation rule is identified by a number, and the smaller the priority number, the higher the execution priority of the rule. In execution, the calculation rules are executed in order from high to low according to the calculation priority. In addition, the rule configuration module binds the calculation result of each stage calculation rule with the input data of the next stage calculation rule. Specifically, the calculation result of the previous stage calculation rule is used as the input data of the next stage calculation rule.

[0099] For example, assume that the target indicator is the sales growth rate of a certain quarter, which depends on two composite indicators: the sales of the current quarter and the sales of the previous quarter, and a second type of calculation rule set containing three calculation rules is pre-configured, where rule A is used to calculate the sales of the current quarter, rule B is used to calculate the sales of the previous quarter, and rule C is used to calculate (sales of the current quarter - sales of the previous quarter) ÷ sales of the previous quarter × 100%. In this example, the calculation priorities of rule A and rule B can be the same or different, and do not affect the calculation result of the final aggregation, but the calculation priority of rule C should be higher than that of rule A and rule B. The calculation results of rule A and rule B will be used as input data for rule C.

[0100] The indicator calculation module is configured to, in the case that the target indicator is a composite indicator, sequentially perform statistical calculation methods of each calculation rule on the indicator calculation data in descending order of the calculation priorities, and cache intermediate result data into the memory mapping structure until the indicator result data of the target indicator is generated.

[0101] In order to improve the calculation efficiency and ensure that the subsequent calculation is not affected by the errors of the previous calculation, the indicator calculation module caches the intermediate result data after each calculation into the memory mapping structure. This can effectively reduce the time of repeated data calculation and ensure the data integrity and consistency during the calculation process. The cached intermediate result data will be used by the subsequent calculation rules in order according to the dependency relationship. After all the calculation rules are sequentially executed, the indicator calculation module will generate the indicator result data of the target indicator, which not only contains the final calculation result of the composite indicator, but also contains all the intermediate result data during the calculation process.

[0102] In an optional embodiment, the rule configuration module is configured to, in the case that the target indicator is a derived indicator, configure a third type of calculation rule set containing at least two calculation rules for the target indicator, wherein the statistical calculation method of at least one calculation rule is configured as a derived level operator, and the derived level operator includes a distinct count.

[0103] In the calculation of a derived indicator, the rule configuration module configures a third type of calculation rule set containing at least two calculation rules according to the specific requirements of the target indicator. The calculation rules in the third type of calculation rule set involve complex data processing, including derived level operators such as distinct count. The distinct count operation is used to count the number of unique values in a field. For example, when analyzing the number of unique customers who purchase a certain product, the distinct count can ensure that each customer is counted only once, rather than being counted multiple times due to multiple purchases. In this case, the role of the derived level operator is not limited to common mathematical operations, but is also extended to operations such as further filtering, grouping, and statistics on data, such as de-duplication, conditional filtering, etc.

[0104] The rule configuration module is further configured to configure a corresponding calculation priority for each calculation rule in the third type calculation rule set, and configure an additional field list used to extract specified additional field information from the plurality of heterogeneous source data.

[0105] For the calculation of derivative indicators, the rule configuration module needs to not only configure the calculation priority, but also configure the additional field list. The additional field list specifies which additional data fields (additional field information) need to be extracted from the plurality of heterogeneous source data and used as auxiliary data in the calculation process. The additional field information involves more context information or provides important background data support for derivative indicators. For example, when calculating the derivative indicator of customer satisfaction, the additional field list can include additional field information such as "customer feedback date" and "customer service score" to assist in calculation or as a filtering condition to affect the calculation result. The additional field information makes the calculation process more accurate, while providing more data levels and perspectives and enhancing the interpretability of the results.

[0106] The index calculation module is configured to, in a case where the index type of the target index is a derivative index, sequentially execute statistical calculation methods of each calculation rule on the index calculation data in a descending order of the calculation priority, and cache intermediate result data and additional field information in the form of key-value pairs in the memory mapping structure until generating the index result data of the target index.

[0107] Similar to the calculation logic of composite atomic indexes, in the calculation process of the target index as a derivative index, the index calculation module sequentially executes statistical calculation methods of each calculation rule in a descending order of the configured calculation priority. In the execution of each calculation rule, in addition to processing the main calculation data, the additional field information related to the rule is also processed. The additional field information is stored in the form of key-value pairs and cached in the memory mapping structure, where the intermediate result data is the key of the key-value pair and the additional field information is the value of the key-value pair. The intermediate result data and the additional field information after each calculation are stored according to the execution order of the calculation rules. When all the calculation rules in the third type calculation rule set corresponding to the derivative index type target index are executed, the final target index result data is generated and contains each intermediate result data and additional field information, facilitating subsequent decision support and data display.

[0108] In an optional embodiment, the system further comprises a data preprocessing module, and the data preprocessing module comprises a data cleaning submodule, a data parsing submodule, and a data mapping submodule.

[0109] The data cleaning submodule is configured to acquire a plurality of original heterogeneous source data from a plurality of data sources, and clean the plurality of original heterogeneous source data to filter out abnormal value data and repeated data in the plurality of original heterogeneous source data, and filter out non-key feature data in the plurality of original heterogeneous source data.

[0110] Please refer to Figure 2 In this embodiment, the data cleaning submodule is configured to acquire original heterogeneous data from a plurality of data sources, and the data sources include various devices or platforms that generate data required for computing indicators, such as flight logs, sensors, and background monitoring programs. The data cleaning submodule identifies and filters out abnormal value data in the plurality of original heterogeneous source data, such as input errors, data points that are out of the normal range or obviously unreasonable. In addition, the data cleaning submodule eliminates repeated data to prevent the same data from being calculated multiple times, thereby ensuring the accuracy and effectiveness of the calculation results. At the same time, the data cleaning submodule deletes non-key feature data from the plurality of original heterogeneous source data, that is, data fields that have no substantial impact on subsequent calculation and decision support, thereby reducing the interference of irrelevant information and improving the efficiency of subsequent data analysis. Optionally, the processing method of the abnormal value can be a statistical method, such as the standard deviation method, the quartile method, etc.; the repeated data can be removed by matching the unique identifiers of the data to ensure that each data record is processed only once; and the selection of non-key feature data can be realized based on feature selection techniques, which are not limited by the present application.

[0111] The data parsing submodule is configured to parse the plurality of original heterogeneous source data output by the data cleaning submodule according to a predefined format, and the predefined format includes a table format and a JSON format.

[0112] After data cleaning, the plurality of original heterogeneous source data output by the data cleaning submodule is input into the data parsing submodule. The data parsing submodule is configured to perform format conversion on the cleaned data to make it conform to a predefined standard format. The predefined format includes a table format (such as CSV, Excel table, etc.) and a JSON format. Irregular data formats (such as log files, XML format, binary data, etc.) in the plurality of original heterogeneous source data are converted into standard formats that are easy to process and calculate, thereby realizing the structured management of data.

[0113] The data mapping submodule is configured to map the plurality of original heterogeneous source data output by the data parsing submodule according to a data format matched by the metadata model, to generate a plurality of heterogeneous source data in a standardized format.

[0114] After data parsing is completed, the data mapping submodule takes over and further converts the parsed data into a data format conforming to a unified standard. According to the metadata model as described above, the parsed data is subjected to a mapping operation to generate heterogeneous source data in a standardized format. For example, assume that the original heterogeneous source data contains a "user ID" field and a "purchase amount" field, but in different original heterogeneous source data, these two fields can use different naming rules or units (such as "User_ID" and "Amount"). The data mapping submodule unifies these fields to a standard field name and processes the conversion of different units according to the dimensions of the metadata model, ensuring the consistency and comparability of the data. In this way, the multi-level index calculation system provided in this embodiment can process heterogeneous source data from different data sources. In specific implementation, for example, in a simulation scenario relying on a large amount of simulation data, one frame of original heterogeneous source data is collected per second following the simulation process, and the original heterogeneous source data collected in real time is subjected to the steps of data cleaning, data parsing, and data mapping described above to form standardized heterogeneous source data that can be used for configuration calculation.

[0115] In an optional implementation, the collection module is configured to, in a case where the plurality of heterogeneous source data in the standardized format is detected to be generated, acquire the plurality of heterogeneous source data, and extract field metadata from the plurality of heterogeneous source data through the metadata model.

[0116] It can be understood that the collection process of the collection module in this embodiment is dynamic, and the collection opportunity is triggered at different times according to different collection sources. In a first aspect, in a case where the plurality of heterogeneous source data in the standardized format is detected to be generated, the heterogeneous source data is automatically acquired (such as Figure 2 the path of metadata active collection).

[0117] The collection module is further configured to, in a case where the definition data of the target index corresponding to the target index is detected to be generated, compare the field metadata of the target index with the field metadata in the metadata model; and in a case where the field metadata of the target index is inconsistent with the field metadata in the metadata model, replace the metadata model based on the version label of the field metadata.

[0118] In a second aspect, in a case where the definition data of the target index is detected to be generated, the collection module automatically acquires the definition data of the target index, and further compares the target index data with the field metadata in the metadata model based on the target index data.

[0119] If the acquisition module finds that the field metadata of the target indicator is inconsistent with the field metadata in the metadata model in the comparison process, a further correction operation is started. Specifically, the acquisition module partitions and replaces the metadata model according to the version label in the field metadata. The version label of the field metadata is used to identify different versions of the field. In the case where the field metadata of the target indicator is inconsistent with the field metadata in the metadata model, the acquisition module determines whether the corresponding field needs to be replaced or updated according to the indication of the version label. At this time, only the replacement operation is performed on the inconsistent field metadata, instead of reconstructing the entire metadata model.

[0120] It can be seen that the acquisition module can not only accurately extract and process heterogeneous source data, but also ensure that the metadata model is timely adapted and updated when new target indicators are introduced, thereby maintaining the efficient operation of the overall system and the consistency of the data.

[0121] Figure 3 is a step schematic diagram of a multi-level indicator calculation method based on configuration provided by an embodiment of the application, applied to a multi-level indicator calculation system based on configuration as described above. The method comprises:

[0122] In step S101, a plurality of heterogeneous source data is acquired, and the field metadata of each heterogeneous source data is extracted by using a metadata model.

[0123] The acquisition module acquires heterogeneous source data from a plurality of data sources. The data sources can include but are not limited to databases, APIs, log files and various forms. The metadata model built-in the acquisition module automatically processes the plurality of heterogeneous source data acquired, and extracts the field metadata in each heterogeneous source data.

[0124] In step S102, at least one calculation rule is configured for the target indicator based on the type of the target indicator, and the calculation rule comprises a statistical field, a field constraint symbol, a field constraint value, a statistical calculation method and a calculation priority.

[0125] Based on the type of the target indicator, the rule configuration module pre-designs the calculation rule for the target indicator. The calculation rule is used to define how to filter and calculate the indicator calculation data required for the target indicator from the plurality of heterogeneous source data, and how to operate the indicator calculation data, that is, to specify the calculation logic of the indicator calculation data.

[0126] In step S103, the field metadata of the plurality of heterogeneous source data is matched according to the statistical field, the field constraint symbol and the field constraint value of each calculation rule, and the heterogeneous source data whose field metadata is matched successfully is determined as the indicator calculation data required for calculating the target indicator.

[0127] The index calculation module filters out index calculation data meeting the conditions from the plurality of heterogeneous source data according to the calculation rules configured by the rule configuration module. First, the field metadata of the plurality of collected heterogeneous source data is matched according to the statistical field, field constraint symbol and field constraint value of each calculation rule. The data meeting the conditions is taken as the index calculation data by analyzing the combination conditions of the field constraint symbol and the field constraint value. The successfully matched heterogeneous source data is taken as the data available for target index calculation and enters the subsequent calculation stage.

[0128] In step S104, the statistical calculation method of the corresponding calculation rule is sequentially performed on the index calculation data in the order from high to low of the calculation priority, so as to obtain the index result data of the target index.

[0129] The index calculation module performs calculation processing on the data according to the filtered index calculation data and the calculation rule, so as to obtain the final result of the target index. All the calculation rules are sorted according to the calculation priority defined in the rule configuration module, and the rules with high priority are preferentially executed. The data meeting the conditions is statistically operated according to the statistical calculation method defined in the calculation rule. After the calculation is performed one by one, the index result data of the target index is output. The index result data not only includes the calculation result of each level index (atomic index, composite index and derived index), but also contains the intermediate result data in the calculation process, so as to facilitate further decision support and data analysis.

[0130] In the embodiment of the application, the field metadata of the heterogeneous data source can be automatically extracted by the metadata model built in the acquisition module, and the original data is uniformly mapped to a standardized field identification system. The burden of repeatedly writing cleaning and parsing codes for different data sources in the traditional method is eliminated, so that when a new data source is added, the adaptation can be completed by only updating the metadata dynamically, and the conflict problem between field heterogeneity and real-time performance is solved. The rule configuration module defines a general rule model including a statistical field, a constraint condition, a calculation operator and a priority, and first realizes the configuration generation of the atomic index without customizing the development of data filtering logic. For the composite index, the dynamic association calculation between the atomic indexes is realized through the priority rule chain, and the bottleneck of the fragmented multi-level index calculation process in the traditional scheme is broken. It can be seen that, through the configuration architecture of the multiple modules, the dynamic generation of the whole process from the original data to the multi-level index is realized, and the real-time performance and the expansibility of the index system are significantly improved.

[0131] Those skilled in the art will appreciate that embodiments of the application can be devised for a variety of electronic devices. It is therefore intended that the embodiments of the application can be carried out in other specific forms than those set forth herein without departing from the spirit and essential characteristics of the application. Accordingly, the drawings and descriptions should be regarded as illustrative in nature rather than restrictive. Although the application is described with regard to only a few embodiments, variations of the embodiments and modifications not described can be devised by those skilled in the art who want to apply the perceived application. Therefore, the application that is intended to be limited only by the claims that follow.

[0132] Embodiments of the application are described with reference to the drawings, in which are shown flowcharts and / or block diagrams of methods and apparatus according to embodiments of the application. It is understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 Figure 1

[0133] While the preferred embodiments of the application have been described, additional variations and modifications can be devised by those skilled in the art who intend to apply the perceived application. It is therefore intended that the appended claims shall be construed to include all such variations and modifications as falling within the true spirit and scope of the application. Accordingly, while the preferred embodiments of the application have been described, it will be understood that further modifications can be devised by those skilled in the art who intend to apply the perceived application. It is therefore intended that the appended claims shall be construed to include all such variations and modifications as falling within the true spirit and scope of the application.

[0134] ​​​​​​Finally, it needs to be pointed out that in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of additional same elements in the process, method, article or terminal device including the elements. The above describes in detail the configuration-based multi-level index calculation system and method provided by the present application, and the principles and implementation modes of the present application are described by applying specific examples in this article. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the present application should not be understood as a limitation of the present application.

Claims

1. A configuration-based multi-level index calculation system, characterized in that, The system includes: a data acquisition module, a rule configuration module, and an indicator calculation module; The acquisition module is configured with a metadata model; the acquisition module is used to acquire multiple heterogeneous source data and extract field metadata of each heterogeneous source data using the metadata model; when the acquisition module detects that multiple heterogeneous source data in a standardized format have been generated, it acquires the heterogeneous source data; and when the acquisition module detects that definition data of a target indicator has been generated, it acquires the definition data of the target indicator and adapts and updates the metadata model based on the definition data of the target indicator. The rule configuration module is used to configure at least one calculation rule for the target indicator based on its indicator type. The calculation rule includes statistical fields, field constraints, field constraint values, statistical calculation methods, and calculation priorities. When the target indicator's indicator type is an atomic indicator, a first type of calculation rule set containing one calculation rule is configured for the target indicator, and the statistical calculation method of the first type of calculation rule set is configured as an atomic-level operator. When the target indicator's indicator type is a composite indicator, a second type of calculation rule set containing at least two calculation rules is configured for the target indicator, wherein the statistical calculation method of each calculation rule is configured as an atomic-level operator or a composite-level operator. When the target indicator's indicator type is a derived indicator, a third type of calculation rule set containing at least two calculation rules is configured for the target indicator, wherein the statistical calculation method of at least one calculation rule is configured as a derived-level operator. The indicator calculation module is used to match the field metadata of the multiple heterogeneous source data according to the statistical fields, field constraint symbols and field constraint values ​​of each calculation rule, and determine the heterogeneous source data whose field metadata is successfully matched as the indicator calculation data required to calculate the target indicator; and execute the statistical calculation method of the corresponding calculation rule on the indicator calculation data in order of calculation priority from high to low to obtain the indicator result data of the target indicator.

2. The system according to claim 1, characterized in that, The metadata model is used to extract the field code, field name, and field description of each heterogeneous source data, and uses the field code, field name, and field description of each heterogeneous source data as the field metadata of that heterogeneous source data; The indicator calculation module is used for: Based on the type of the field constraint symbol for each calculation rule, the field constraint value of each calculation rule is parsed to obtain the target constraint range for each calculation rule. The types of the field constraint symbols include equal to, numerical range, contain, greater than, less than, greater than or equal to, less than or equal to, and not equal to. Among the multiple heterogeneous source data, heterogeneous source data whose field codes or field names contain the statistical field are identified as candidate computation data, and the data value of each candidate computation data is determined according to the field description of each candidate computation data. Candidate computational data whose data values ​​fall within the target constraint range are identified as heterogeneous source data whose field metadata has been successfully matched.

3. The system according to claim 2, characterized in that, The system also includes an indicator definition module and an indicator recommendation module; The indicator definition module is used to acquire the multiple heterogeneous source data and generate indicator definition data corresponding to the target indicator based on the multiple heterogeneous source data. The indicator definition data is used to describe the target indicator in different dimensions. The indicator recommendation module is used to obtain the indicator definition data corresponding to each of the multiple indicators, as well as the indicator result data of each of the multiple indicators. Based on the indicator definition data and the indicator result data, the correlation between each indicator is determined, and indicator recommendation results representing the correlation between each indicator are output.

4. The system according to claim 3, characterized in that, The indicator recommendation module is configured with a graph database and a graph convolutional network model. Each node in the graph database represents a corresponding indicator. The graph convolutional network model is pre-trained based on the indicator definition data corresponding to each of the multiple indicators and the indicator result data of each of the multiple indicators. The indicator definition module is used to associate the indicator definition data corresponding to each of the multiple indicators with the calculation rules of each of the multiple indicators to obtain the association calculation rule identifier of each of the multiple indicators. The indicator recommendation module is used to determine the association relationship between the indicators according to the association calculation rule identifiers of the multiple indicators; calculate the association weight between the indicators according to the association relationship between the indicators; and establish the edges between the nodes in the graph database based on the association weight between the indicators. The indicator recommendation module is used to calculate the correlation strength between indicators based on the nodes and edges in the graph database using the graph convolutional network model, and to determine at least one set of related indicators with a correlation strength greater than a preset threshold; and outputs a recommendation indicator result including at least one set of related indicators.

5. The system according to claim 1, characterized in that, The indicator types include atomic indicators, composite indicators, and derived indicators; The rule configuration module is used to configure a first type of calculation rule set containing one calculation rule for the target indicator when the indicator type of the target indicator is an atomic indicator. The statistical calculation method of the first type of calculation rule set is configured as an atomic-level operator, and the atomic-level operator is any one of the original value, count, maximum value, minimum value and average value operators. The indicator calculation module is used to perform statistical calculations on the indicator calculation data according to the corresponding calculation rules when the indicator type of the target indicator is an atomic indicator, so as to obtain the indicator result data of the target indicator.

6. The system according to claim 5, characterized in that, The rule configuration module is used to configure a second type of calculation rule set containing at least two calculation rules for the target indicator when the indicator type of the target indicator is a composite indicator. The statistical calculation method of each calculation rule is configured as an atomic-level operator or a composite-level operator, and the composite-level operator includes at least one of addition, subtraction, multiplication and division. The rule configuration module is also used to configure a corresponding calculation priority for each calculation rule in the second type of calculation rule set, and to bind the calculation result of the calculation rule with the previous calculation priority as the input data of the calculation rule with the next calculation priority. The indicator calculation module is used to, when the indicator type of the target indicator is a composite indicator, sequentially execute the statistical calculation methods of each calculation rule on the indicator calculation data in descending order of calculation priority, and cache the intermediate result data in a memory-mapped structure until the indicator result data of the target indicator is generated.

7. The system according to claim 5, characterized in that, The rule configuration module is used to configure a third type of calculation rule set containing at least two calculation rules for the target indicator when the indicator type of the target indicator is a derived indicator. The statistical calculation method of at least one calculation rule is configured as a derived-level operator, and the derived-level operator includes deduplication counting. The rule configuration module is also used to configure a corresponding calculation priority for each calculation rule in the third type of calculation rule set, and to configure an additional field list, which is used to extract specified additional field information from the multiple heterogeneous source data. The indicator calculation module is used to, when the indicator type of the target indicator is a derived indicator, sequentially execute the statistical calculation methods of each calculation rule on the indicator calculation data in descending order of calculation priority, and cache the intermediate result data and additional field information in the form of key-value pairs in a memory-mapped structure until the indicator result data of the target indicator is generated.

8. The system according to claim 1, characterized in that, The system also includes a data preprocessing module, which includes a data cleaning submodule, a data parsing submodule, and a data mapping submodule. The data cleaning submodule is used to acquire multiple raw heterogeneous source data from multiple data sources; Data cleaning is performed on the multiple original heterogeneous source data to filter out outlier data and duplicate data, as well as non-critical feature data. The data parsing submodule is used to parse multiple raw heterogeneous source data output by the data cleaning submodule according to a predefined format, including table format and JSON format; The data mapping submodule is used to perform data mapping on multiple raw heterogeneous source data output by the data parsing submodule according to the data format matched by the metadata model, and generate multiple heterogeneous source data in a standardized format.

9. The system according to claim 3 or 8, characterized in that, The acquisition module is used to acquire the multiple heterogeneous source data when multiple heterogeneous source data in a standardized format are detected, and to extract field metadata from the multiple heterogeneous source data through the metadata model. The acquisition module is also used to compare the field metadata of the target indicator with the field metadata in the metadata model when it detects that the indicator definition data corresponding to the target indicator has been generated. If the field metadata of the target metric is inconsistent with the field metadata in the metadata model, the metadata model is partitioned and replaced based on the version tag of the field metadata.

10. A configuration-based multi-level index calculation method, characterized in that, Applied to the system as described in any one of claims 1-9, the method comprises: The acquisition module acquires multiple heterogeneous source data and extracts field metadata for each heterogeneous source data using a metadata model. When the acquisition module detects that multiple heterogeneous source data in a standardized format have been generated, it acquires the heterogeneous source data. When the acquisition module detects that definition data for a target indicator has been generated, it acquires the definition data for the target indicator and adapts and updates the metadata model based on the definition data for the target indicator. Based on the target indicator's indicator type, at least one calculation rule is configured for the target indicator. The calculation rule includes a statistical field, field constraint, field constraint value, statistical calculation method, and calculation priority. If the target indicator's indicator type is an atomic indicator, a first type of calculation rule set containing one calculation rule is configured for the target indicator, and the statistical calculation method of the first type of calculation rule set is configured as an atomic-level operator. If the target indicator's indicator type is a composite indicator, a second type of calculation rule set containing at least two calculation rules is configured for the target indicator, wherein the statistical calculation method of each calculation rule is configured as an atomic-level operator or a composite-level operator. If the target indicator's indicator type is a derived indicator, a third type of calculation rule set containing at least two calculation rules is configured for the target indicator, wherein the statistical calculation method of at least one calculation rule is configured as a derived-level operator. Based on the statistical fields, field constraint operators, and field constraint values ​​of each calculation rule, the field metadata of the multiple heterogeneous source data is matched, and the heterogeneous source data whose field metadata is successfully matched is determined as the indicator calculation data required to calculate the target indicator; The statistical calculation methods of the corresponding calculation rules are executed sequentially on the indicator calculation data according to the calculation priority from high to low, so as to obtain the indicator result data of the target indicator.

Images