Data index generation method and device and storage medium

Abstract

The invention relates to the field of data processing, in particular to a data index generation method and device and a storage medium, and the method comprises the steps that definition information input by a user is acquired, and the definition information comprises a data source, service logic, dimensions and measurement of a data index; according to the data source, the business logic, the dimension and the measurement of the data index, selecting a processing node with a preset processing rule, and arranging the processing node into a description model of the data index; generating description fields of the data indexes through the description model; traversing processing nodes of the description model to obtain operation information of each processing node in a description field generation process; compiling the operation information, and generating a calculation instruction of the data index; according to the technical scheme, the description model can be arranged based on the processing nodes of the preset processing rules, description of the data indexes is unified, the calculation instruction of the data indexes is generated, the calculation result is rapidly obtained, and the development efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a method, device and storage medium for generating data indicators. Background Art

[0002] In the era of big data, a large number of data indicators have become necessary conditions for decision-making. From business decisions to personal purchases, most behaviors are affected by data indicators. Generally, data indicator configuration comes from the needs of upper-level decision-making. After business review, the indicator definition, data acquisition logic, indicator calculation formula, indicator dimension and measurement are obtained, and then handed over to technical personnel for development. According to the indicator definition and indicator calculation formula, the technical personnel write specific query SQL (Structured Query Language, database language) instruction statements, and obtain the final indicator calculation results through the execution of SQL instruction statements.

[0003] However, different business personnel have different descriptions of indicator definitions, formulas, dimensions, and metrics. Different technicians also have different calculation logics for the same business indicator. Different configurations are performed on business indicators each time, which leads to low work efficiency. At present, there is still a lack of a data indicator construction method to unify the description of data indicators, quickly generate the calculation results of data indicators, and improve the development efficiency of data indicators. Summary of the invention

[0004] One of the purposes of this application is to provide a method for generating data indicators to solve the problem in the prior art that different business personnel have inconsistent descriptions when constructing data indicators, resulting in high communication costs and low development efficiency; the second purpose is to provide a data indicator generating device; the third purpose is to provide a storage medium.

[0005] In order to achieve the above purpose, the technical solution adopted in this application is as follows:

[0006] A data indicator generation method, comprising:

[0007] Obtaining definition information input by a user, the definition information including: data source, business logic, dimension and measurement of data indicators;

[0008] According to the data source, business logic, dimension and metric of the data indicator, a processing node with a pre-set processing rule is selected and arranged into a description model of the data indicator, wherein the processing rule is: each processing node corresponds to an operation; the operations include filtering, intersection, union, aggregation, function call, counting, deduplication and sorting;

[0009] Generate a description field of the data indicator through the description model;

[0010] Traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field;

[0011] Compile the operation information to generate calculation instructions for the data indicators;

[0012] Run the calculation instruction to obtain the result of the data indicator.

[0013] Optionally, the description field includes a dimension field and a metric field of the data indicator; and generating the description field of the data indicator through the description model includes:

[0014] Calling a data source node to connect to the data source of the data indicator according to the data source of the data indicator;

[0015] Calling a data filtering node to filter out the data required by the data indicator from the data source;

[0016] According to the business logic, dimension and metric of the data indicator, the corresponding processing node is called to operate the data to generate the dimension field and metric field of the data indicator; the processing node includes: a data connection node, a data aggregation node, and a data calculation node;

[0017] Call the description output node to output the description field of the data indicator.

[0018] Optionally, the data calculation node includes a calculation formula, which is obtained according to the dimension of the data indicator. The corresponding processing node is called to operate the data according to the business logic, dimension and metric of the data indicator to generate the dimension field and metric field of the data indicator, including:

[0019] The data calculation node is called to calculate the received data according to the calculation formula to obtain the dimension field of the data indicator.

[0020] Optionally, compiling the operation information to generate a calculation instruction for the data indicator includes:

[0021] The operation information is compiled according to the logical order in which each processing node processes the data to generate calculation instructions for the data indicators.

[0022] Optionally, the description model includes at least a data source node and a description output node, and traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field includes:

[0023] Starting from the data source node, traversing until the description output node, sequentially obtaining the operation record of each processing node of the description model;

[0024] The operation records of each of the processing nodes are parsed to obtain the operation information of each of the processing nodes.

[0025] Optionally, the executing the calculation instruction to generate the data indicator includes:

[0026] transmitting the computing instructions to a computing environment;

[0027] Connecting to the data source of the data indicator;

[0028] In the computing environment, the computing instructions are executed to generate results of the data indicators.

[0029] A data indicator generating device, the device comprising:

[0030] The acquisition module is used to acquire definition information input by the user, and the definition information includes:

[0031] The data source, business logic, dimension and measurement of the data indicator; an arrangement module, which is used to select processing nodes with pre-set processing rules according to the data source, business logic, dimension and measurement of the data indicator, and arrange them into a description model of the data indicator, wherein the processing rules are: each processing node corresponds to an operation; the operations include filtering, intersection, union, aggregation, function call, counting, deduplication and sorting;

[0032] A description field generation module, used to generate the description field of the data indicator through the description model;

[0033] A traversal module, used for traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field;

[0034] A compiling module, used to compile the operation information and generate calculation instructions for the data indicators;

[0035] The running module is used to run the calculation instruction to obtain the result of the data indicator.

[0036] Optionally, the description field includes dimension fields and measurement fields of the data indicator; the description field generation module is also used to: call the data source node to connect the data source of the data indicator according to the data source of the data indicator; call the data filtering node to filter out the data required for the data indicator from the data source; call the corresponding processing node to operate on the data according to the business logic, dimension and measurement of the data indicator to generate the dimension field and measurement field of the data indicator; the processing node includes: data connection node, data aggregation node, data calculation node; call the description output node to output the description field of the data indicator.

[0037] Optionally, the compilation module is further used to compile the operation information according to the logical order in which each of the processing nodes processes the data, and generate calculation instructions for the data indicators.

[0038] A computer-readable storage medium has a computer program installed thereon, and when the computer program is executed by a processor, the above-mentioned data indicator generation method is implemented.

[0039] Beneficial effects of this application:

[0040] The present application selects processing nodes with pre-set processing rules according to the data source, business logic, dimension and measurement of the data indicator, and arranges them into a description model of the data indicator; generates description fields of the data indicator through the description model; traverses the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field; compiles the operation information to generate calculation instructions for the data indicator; runs the calculation instructions to obtain the results of the data indicator. Through the technical solution of the present application, the description model can be arranged based on the processing nodes with pre-set processing rules, the description of the data indicator can be unified, and the calculation instructions of the data indicator can be generated to quickly obtain the calculation results, thereby improving development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of a method for generating data indicators in one embodiment of the present application;

[0042] Figure 2 A logical diagram describing model data processing in an embodiment of the present application;

[0043] Figure 3 It is another flow chart of the data indicator generation method described in one embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of the framework of the data indicator generating device described in one embodiment of the present application.

[0045] Among them, 201-data source node 1, 202-data source node 2, 203-data filtering node, 204-data connection node, 205-data calculation node, 206-data aggregation node, 207-description output node, 400-data indicator generation device, 401-acquisition module, 402-orchestration module, 403-description field generation module, 404-traversal module, 405-compilation module, 406-operation module. DETAILED DESCRIPTION

[0046] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0047] The technical concept of the present application is to call the corresponding processing nodes to be arranged into a description model according to the definition information of the data indicator, use the description model to generate the description field of the data indicator, and parse the operation of each node of the description model in the process of generating the description field, compile the operation of each node according to the logical relationship, generate the calculation instruction of the data indicator, and run the calculation instruction to generate the calculation result of the data indicator. Among them, the description model of the data indicator is a DSL (Domain Specific Language) description model, which can output the description field of the data indicator and unify the description of the data indicator. Business personnel only need to pay attention to the field definition and the implementation of the DSL description model of the data indicator, and do not need to understand the specific underlying execution logic of the data indicator, and further realize the business personnel to construct the data indicator without perceiving the underlying calculation logic of the data indicator. In the present application, the DSL description model is constructed by processing nodes with processing rules pre-set.

[0048] Specifically, refer to Figure 1 , Figure 1 : is a flow chart of a method for generating a data indicator in an embodiment of the present application. In an embodiment of the present application, the method for generating a data indicator specifically includes the following steps:

[0049] Step S101, obtaining definition information input by a user, wherein the definition information includes: data source, business logic, dimension and measurement of a data indicator.

[0050] Specifically, the definition information input by the user is a specific text description of the data indicator that needs to be generated. Based on the data source, business logic, dimension and measurement text information of the data indicator in the definition information, the description model of the data indicator can be further compiled.

[0051] Step S102, according to the data source, business logic, dimension and measurement of the data indicator, select the processing nodes with pre-set processing rules, and arrange them into a description model of the data indicator, the processing rules are: each processing node corresponds to an operation; the operations include filtering, intersection, union, aggregation, function call, counting, deduplication, and sorting.

[0052] Specifically, in the present application, each processing node with processing rules can be pre-set on the data indicator description model construction platform, and each processing node corresponds to an operation, that is, each processing node only performs one operation. In this way, when arranging the description model corresponding to each data indicator based on the data indicator, different processing nodes can be called according to different needs to combine into a targeted and complete data indicator description model. Specifically, it can be determined how to call the processing node to arrange the description model of the data indicator based on the data source, business logic, dimension and measurement in the definition information of the data indicator.

[0053] Since each processing node corresponds to an operation, when selecting, you can easily understand the operations performed by each node through the visual operation functions of the processing node. The operations involved in the processing nodes are diverse, including filtering, intersection, union, aggregation, function call, counting, deduplication, sorting, and other operation types, so as to ensure that the executable operations of each processing node meet the requirements of the data indicator as much as possible. By selecting the required processing node, you can easily build a description model for the data indicator and unify the description of the data indicator.

[0054] Step S103: Generate a description field of the data indicator through the description model.

[0055] Specifically, the description model of the data indicator can generate the description field of the data indicator, which is the field information of the data indicator at the definition level, so that calculation instructions based on the description field can be generated later.

[0056] Step S104, traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field.

[0057] Specifically, the description model of the data indicator has obtained the execution logic of how to generate the specific result of the data indicator through the processing of each node in the process of generating the description field of the data indicator. Therefore, if you need to obtain the specific result of the data indicator, first traverse each node in the description model of the data indicator to obtain the specific operation information of each processing node in the process of generating the data indicator, so as to obtain the processing logic for generating the data indicator based on the operation information.

[0058] Step S105, compile the operation information and generate calculation instructions for the data indicators.

[0059] Specifically, by using data indicators to describe the operation data of each processing node in the model, the processing logic corresponding to the data indicator can be obtained. By compiling the operation information of each processing node, the calculation instructions of the data indicator can be generated, and the calculation instructions are used to calculate the specific results of the data indicator.

[0060] Step S106, executing the calculation instruction to obtain the result of the data indicator.

[0061] Specifically, the data instruction can be run in the required computing environment, and after the calculation is completed, the specific result of the data indicator can be obtained. That is, the specific value of the data indicator is calculated, so that the decision can be made intuitively through the specific result of the data indicator.

[0062] In this embodiment, by obtaining definition information, the definition information includes the data source, business logic, dimension and measurement of the data indicator. According to the data source, business logic, dimension and measurement of the data indicator, a processing node with a pre-set processing rule is selected and arranged into a description model of the data indicator, wherein the processing rule corresponds to an operation for each processing node; through the description model of the data indicator, a description field of the data indicator is generated, and then each node in the description model is traversed to obtain the operation information of each node in the process of generating the description field of the data indicator, and the operation information is compiled to generate the calculation instruction of the data indicator. By running the calculation instruction, the result of the data indicator can be obtained. Based on the processing node with unified processing rules, the description model of each data indicator is conveniently arranged, the description field is output, the description of the data indicator is unified, and the calculation instruction of the data indicator is generated through the description model of the data indicator, so that the specific result of the required data indicator can be quickly obtained, and the development efficiency can be improved so that decision-making can be made based on the specific result of the data indicator.

[0063] In a feasible implementation manner, the description field includes a dimension field and a metric field of the data indicator; step S103, generating the description field of the data indicator through the description model, may specifically include the following steps:

[0064] Step S1031, calling the data source node, and connecting to the data source of the data indicator according to the data source of the data indicator.

[0065] Specifically, if a data indicator is to be calculated, the metadata of the data indicator must be obtained first, and the table name, field and other specific data containing the required data must be selected from the data source. Therefore, the data source node is first called, and this processing node is used to connect to the data source of the data indicator. Specifically, the data source node first connects to the data source of the data indicator based on the data source of the data indicator, such as the IP address, connection port, database and other information of the data source of the data indicator, as well as the table name and field name of the specific query, and obtains data related to the data indicator from the data source.

[0066] Step S1032, calling a data filtering node to filter out data required by the data indicator from the data source.

[0067] Specifically, after obtaining data related to the data indicator, these data need to be filtered. The filtering conditions can be data that exceeds a fixed limit value, or it can be recorded data within a certain time period. The specific conditions can be pre-set according to actual needs to filter out the data required for the data indicator from the data source.

[0068] Step S1033, according to the business logic, dimension and measurement of the data indicator, call the corresponding processing node to operate on the data to generate the dimension field and measurement field of the data indicator; the processing node includes: data connection node, data aggregation node, data calculation node.

[0069] Specifically, after determining the data required for the data indicator, these data can be processed to obtain the description field of the data indicator. By calling the corresponding processing nodes, such as data connection nodes, data aggregation nodes, data calculation nodes, etc., according to the business logic, dimensions and metrics of the data indicator, the data is operated accordingly, and finally the dimension fields and metric fields of the data indicator are generated, and the name field is defined for the data indicator at the data connection node. In this application, the business logic of the data indicator includes the relationship between the data indicator and other data, so as to understand how to process other data through the business logic, and finally obtain the processing logic of the data indicator. The dimensions and metrics of data indicators are common features in data analysis and will not be repeated here.

[0070] Step S1034, calling the description output node to output the description field of the data indicator.

[0071] Specifically, after the dimension fields and metric fields of the data indicator have been generated, the description output node can be called to output the description fields of the data indicator, which indicates that the description model corresponding to the data indicator has been compiled and can be used normally.

[0072] In this embodiment, by calling the corresponding processing node, the description field of the data indicator is generated, and the description field includes the dimension field and the measurement field of the data indicator. The specific field of the description field can be referenced to generate instructions related to the data indicator. In the process of generating the description field, the data received by each processing node is generally the data output by the previous processing node. After performing the corresponding processing operation, the description field of the data indicator can be obtained.

[0073] In a feasible implementation, specifically, the data calculation node includes a calculation formula, the calculation formula is obtained according to the dimension of the data indicator, and the corresponding processing node is called to operate the data according to the business logic, dimension and metric of the data indicator to generate the dimension field and metric field of the data indicator, including:

[0074] The data calculation node is called to calculate the received data according to the calculation formula to obtain the dimension field of the data indicator.

[0075] Specifically, in the process of generating the description field of the data indicator, the data calculation node can be called to process the received data, and the data calculation node should include a calculation formula obtained according to the dimension of the data indicator. For example, if the data indicator is the number of goods sold by a store S on a certain day, then the dimension is each product of the store S, and the measurement is the sales quantity of the product. The corresponding calculation formula should be to add up the sales quantity of the products S1, S2, S3...S10 of the store S on a certain day.

[0076] In this embodiment, the specific calculation method of the data computing node on the received data is determined through the dimension information of the data indicator, and the dimension field of the data indicator can be accurately obtained by processing the data using the data computing node.

[0077] In one embodiment of the present application, referring to Figure 2 , Figure 2 This is a logical diagram describing the model data processing according to an embodiment of the present application. Figure 2 , further explains the construction of the description model of data indicators. In actual applications, it can be set as needed. The processing nodes in the figure are only examples, showing the local data processing logic.

[0078] Assume that, based on the definition information of a data indicator, the processing nodes such as data source nodes 201 and 202, data filtering node 203, data connection node 204, data calculation node 205, data aggregation node 206, and description output node 207 are selected to compile the description model corresponding to the data indicator. Figure 2 As shown:

[0079] First, connect to the data source of the data indicator according to the data source of the data indicator. Generally speaking, the data source in the definition information of the data indicator includes the data source connection information, such as the IP address of the data storage device, the connection port, the database, the table name and the field name of the specific query.

[0080] Currently, two data sources need to be connected, one is the data source Hive (a data warehouse tool), and the other is the data source MySql (an open source relational database management system). Two data source nodes 201 and 202 are called to connect to the data sources, and the corresponding table name and specific field name in the data source are found.

[0081] The data filtering node 203 is called to filter the data found from the connected data source. Specifically, a filtering operator can be set, and the filtering operator includes but is not limited to >, <, <=, >=, IN, NOT IN, !=, to perform data filtering. The limit value represents the limit of the filter field. Assuming that the filtering condition at this time is to filter out the field A whose specific value in the data source Hive table A is greater than the limit value 100, the field A in the table A is filtered, and the filtered data is transmitted to the data connection node 204. The data selected from the above data source MySql does not need to pass through the data filtering node 204, and the output data is transmitted to the data connection node 204 together with the data output by the data filtering node 203.

[0082] The data connection node 204 includes a left node and a right node. The connection condition represents the connection mode of the left node and the right node. The data connection node 204 receives the output data of the data filtering node 203 and the output data of the data source node 202 .

[0083] The data processed by the data connection node 204 is transmitted to the data calculation node 205. At this time, the calculation formula of the data calculation node 205 is to add field A and field B to obtain field D, and transmit fields C and D to the next processing node.

[0084] After processing the data, the data calculation node 205 outputs the data to the data aggregation node 206. The aggregation types include SUM (select field sum), MAX (select field maximum value), MIN (select field minimum value), COUNT (to find the number of rows recorded in the current module), etc. In this embodiment, the aggregation type is sum, and the data aggregation node 206 aggregates field D to obtain field E, and transmits fields C and E to the description output node 207, that is, outputs the description field of the data indicator.

[0085] In this way, after the description model of the data indicator is arranged, the description field of the data indicator is output. The indicator name of the data indicator is text-index, the dimension field is E, and the metric field is C. In this embodiment, the data input and output of each processing node may be the same as the field in the data source, or it may be different. Fields A, B, C, D, and E are all examples and are not intended to limit this application. In this way, after the description model is arranged, the description field of the data indicator is output through the description model to unify the description of the data indicator.

[0086] In a feasible implementation manner, compiling the operation information of each node of the description model of the data indicator in the process of generating the description field of the data indicator to generate the calculation instruction of the data indicator includes:

[0087] The operation information is compiled according to the logical order in which each processing node processes the data, and calculation instructions for the data indicators are generated.

[0088] In this embodiment, the operation information of each processing node is compiled according to the logical order in which each processing node processes data. It can be understood that the input data received by the current processing node is the data output by the previous processing node of the current processing node. Therefore, the operation information needs to be compiled according to the logical order in which the processing node processes data, and finally the calculation instructions of the data indicators are generated.

[0089] In a feasible implementation manner, the description model of the data indicator includes at least a data source node and a description output node, and the traversing of the processing nodes of the description model to obtain the operation information of each processing node in the process of generating the description field includes:

[0090] Starting from the data source node, traversal begins until the description output node, and operation records of each processing node of the description model are obtained in turn; the operation records of each processing node are parsed to obtain the operation information of each processing node.

[0091] In this embodiment, the processing nodes of a description model of a data indicator include at least a data source node and a description output node. The data source node is used to connect to the data source of the data indicator, and the description output node is used to output the description field of the data indicator. When obtaining the operation information of each processing node of the description model in the process of generating the description field of the data indicator, the various processing nodes of the description model are traversed, with the data source node as the starting point and the description output node as the end point. According to the logical order in which each processing node processes data, the nodes are traversed in sequence to obtain the operation records of each processing node. The operation records of each processing node are parsed to obtain the specific operation information of the processing node from the operation records.

[0092] In a feasible implementation manner, the executing the computing instruction to generate the data indicator includes:

[0093] The computing instructions are transmitted to a computing environment; the data source of the data indicator is connected; and in the computing environment, the computing instructions are executed to generate the data indicator result.

[0094] In practical applications, the compilation of instructions and the execution of instructions are performed in different environments. In this embodiment, the compiled calculation instructions are first transmitted to the calculation environment, the data source of the data indicator is connected in the calculation environment, the relevant data required for calculating the data indicator is obtained, and then the calculation instructions of the data indicator are executed in the calculation environment to obtain the specific results of the data indicator.

[0095] In one embodiment of the present application, referring to Figure 3 , Figure 3 is another flow chart of the data indicator generation method described in one embodiment of the present application, such as Figure 3 As shown, the data indicator generation method includes the following steps:

[0096] Step S301, input definition information of the data indicator, and receive the arranged data indicator description model.

[0097] Specifically, users can input description information of data indicators in the description model orchestration platform, and after the description model is orchestrated, it is returned to the user terminal.

[0098] Step S302, taking the data source node as the starting node, breadth-wise traversing the description model, and parsing to obtain relevant information of each model node.

[0099] Specifically, after receiving the description model, the description model of the data indicator is used to output the description field of the data indicator. After the description model runs normally, the data source node is used as the starting node, and the description model is traversed in breadth to parse and obtain the relevant information of each model node. The relevant information includes the operation information of each processing node of the description model in the process of generating the data indicator description field.

[0100] Step S303, constructing calculation instructions for data indicators based on the relevant information of each model node.

[0101] Specifically, the relevant information of each model node (i.e., the processing node that describes the model) includes the operation information of the processing node, so as to generate calculation instructions for data indicators based on the operation information.

[0102] Step S304: submit the obtained computing instruction to the computing environment.

[0103] The specific details of this embodiment can be found in the above embodiments and will not be described again here.

[0104] In this embodiment, the user can receive a description model of a data indicator, call a node with pre-set processing rules based on a unified data indicator description model orchestration platform, orchestrate the description model, obtain the corresponding data indicator description model, and finally construct a calculation instruction for the data indicator to calculate the specific result of the data indicator.

[0105] In one embodiment of the present application, referring to Figure 4 , Figure 4 is a schematic diagram of the framework of the data indicator generating device 400 in an implementation of the present application, such as Figure 4 As shown, in this embodiment, a data indicator generating device includes:

[0106] The acquisition module 401 is used to acquire definition information input by the user, wherein the definition information includes: data source, business logic, dimension and measurement of the data indicator;

[0107] The orchestration module 402 is used to select processing nodes with pre-set processing rules according to the data source, business logic, dimension and measurement of the data indicator, and arrange them into a description model of the data indicator, wherein the processing rule is: each processing node corresponds to an operation; the operation includes filtering, intersection, union, aggregation, function call, counting, deduplication and sorting;

[0108] A description field generation module 403 is used to generate a description field of the data indicator through the description model;

[0109] A traversal module 404 is used to traverse the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field;

[0110] A compiling module 405, used to compile the operation information and generate calculation instructions for the data indicators;

[0111] The running module 406 is used to run the calculation instruction to obtain the result of the data indicator.

[0112] In this embodiment, by selecting processing nodes with pre-set processing rules according to the data source, business logic, dimension and measurement of the data indicator in the definition information, and arranging them into a description model of the data indicator, wherein the processing rule corresponds to an operation for each processing node; through the description model of the data indicator, a description field of the data indicator is generated, and then each node in the description model is traversed to obtain the operation information of each processing node in the process of generating the description field of the data indicator, compile these operation information, generate the calculation instructions of the data indicator, and run the calculation instructions to obtain the result of the data indicator. Based on the processing nodes of the processing rules, the description models of various data indicators are arranged, so that the description models of the data indicators can be obtained conveniently and efficiently, and the calculation instructions of the data indicators can be generated, so as to quickly obtain the specific results of the required data indicators, so as to facilitate decision-making based on the specific results of the data indicators.

[0113] Optionally, the description field includes a dimension field and a metric field of the data indicator; the description field generating module 403 is further used to:

[0114] Call the data source node to connect the data source of the data indicator according to the data source of the data indicator; call the data filtering node to filter out the data required by the data indicator from the data source; call the corresponding processing node to operate on the data according to the business logic, dimension and measurement of the data indicator to generate the dimension field and measurement field of the data indicator; the processing node includes: data connection node, data aggregation node, data calculation node; call the description output node to output the description field of the data indicator.

[0115] Optionally, the compiling module 405 is further used for:

[0116] The operation information is compiled according to the logical order in which each processing node processes the data to generate calculation instructions for the data indicators.

[0117] Optionally, the data calculation node includes a calculation formula, and the calculation formula is obtained according to the dimension of the data indicator. The description field generation module 403 is further used to:

[0118] The data calculation node is called to calculate the received data according to the calculation formula to obtain the dimension field of the data indicator.

[0119] Optionally, the description model at least includes a data source node and a description output node, and the traversal module 404 is further used to:

[0120] Starting from the data source node, traversal begins until the description output node, and operation records of each processing node of the description model are obtained in turn; the operation records of each processing node are parsed to obtain the operation information of each processing node.

[0121] Optionally, the operation module 406 is further configured to:

[0122] The computing instruction is transmitted to a computing environment; the data source of the data indicator is connected; and the computing instruction is executed in the computing environment to generate the result of the data indicator.

[0123] In this embodiment, the description of the related device embodiment is relatively short, and specific reference may be made to the above method embodiment.

[0124] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is installed. When the computer program is executed by a processor, the data indicator generation method described in the above embodiment is implemented.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0127] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0131] The above is a detailed introduction to a data indicator generation method, device and storage medium provided. This article uses specific examples to illustrate the principles and embodiments of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific embodiments and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating data indicators, characterized in that: The method comprises: Obtaining definition information input by a user, the definition information including: data source, business logic, dimension and measurement of data indicators; According to the data source, business logic, dimension and metric of the data indicator, a processing node with a pre-set processing rule is selected and arranged into a description model of the data indicator, wherein the processing rule is: each processing node corresponds to an operation; the operations include filtering, intersection, union, aggregation, function call, counting, deduplication and sorting; Generate a description field of the data indicator through the description model; Traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field; Compile the operation information to generate calculation instructions for the data indicators; Run the calculation instruction to obtain the result of the data indicator.

2. The data indicator generation method according to claim 1, characterized in that: The description field includes the dimension field and the measurement field of the data indicator; the description field of the data indicator is generated by the description model, including: Calling a data source node to connect to the data source of the data indicator according to the data source of the data indicator; Calling a data filtering node to filter out the data required by the data indicator from the data source; According to the business logic, dimension and metric of the data indicator, the corresponding processing node is called to operate the data to generate the dimension field and metric field of the data indicator; the processing node includes: a data connection node, a data aggregation node, and a data calculation node; Call the description output node to output the description field of the data indicator.

3. The data indicator generation method according to claim 2, characterized in that: The data calculation node includes a calculation formula, which is obtained according to the dimension of the data indicator. According to the business logic, dimension and metric of the data indicator, the corresponding processing node is called to operate the data to generate the dimension field and metric field of the data indicator, including: The data calculation node is called to calculate the received data according to the calculation formula to obtain the dimension field of the data indicator.

4. The data indicator generation method according to claim 1, characterized in that: The compiling the operation information to generate the calculation instruction of the data indicator includes: The operation information is compiled according to the logical order in which each processing node processes the data to generate calculation instructions for the data indicators.

5. The data indicator generation method according to claim 1, characterized in that: The description model at least includes a data source node and a description output node, and the traversing of the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field includes: Starting from the data source node, traversing until the description output node, sequentially obtaining the operation record of each processing node of the description model; The operation records of each of the processing nodes are parsed to obtain the operation information of each of the processing nodes.

6. The data indicator generation method according to claim 1, characterized in that: The step of running the calculation instruction to generate the data indicator includes: transmitting the computing instructions to a computing environment; Connecting to the data source of the data indicator; In the computing environment, the computing instructions are executed to generate results of the data indicators.

7. A data indicator generating device, characterized in that: The device comprises: An acquisition module is used to acquire definition information input by a user, wherein the definition information includes: data source, business logic, dimension and measurement of data indicators; An orchestration module is used to select processing nodes with pre-set processing rules according to the data source, business logic, dimension and measurement of the data indicator, and arrange them into a description model of the data indicator, wherein the processing rules are as follows: each processing node corresponds to an operation; the operations include filtering, intersection, union, aggregation, function call, counting, deduplication and sorting; A description field generation module, used to generate the description field of the data indicator through the description model; A traversal module, used for traversing the processing nodes of the description model to obtain operation information of each processing node in the process of generating the description field; A compiling module, used to compile the operation information and generate calculation instructions for the data indicators; The running module is used to run the calculation instruction to obtain the result of the data indicator.

8. The data indicator generating device according to claim 7, characterized in that: The description field includes the dimension field and the measurement field of the data indicator; the description field generation module is further used to: call the data source node to connect the data source of the data indicator according to the data source of the data indicator; call the data filtering node to filter out the data required by the data indicator from the data source; According to the business logic, dimension and metric of the data indicator, the corresponding processing node is called to operate the data to generate the dimension field and metric field of the data indicator; the processing node includes: a data connection node, a data aggregation node, and a data calculation node; Call the description output node to output the description field of the data indicator.

9. The data indicator generating device according to claim 7, characterized in that: The compiling module is further used to compile the operation information according to the logical order in which each of the processing nodes processes the data, and generate calculation instructions for the data indicators.

10. A computer-readable storage medium, characterized in that: A computer program is installed thereon, and when the computer program is executed by a processor, the data indicator generation method as described in any one of claims 1-6 above is implemented.

Images