A method, apparatus, equipment and storage medium for index processing

Abstract

This invention discloses a method, apparatus, device, and storage medium for index processing, applied in the field of index processing. Based on a product code package, the data processing flow of the index is organized into data processing steps, and these steps are configured in a data processing step configuration table. An index lineage table is generated based on the product code package. The index to be processed is input into the main data processing entry point, and the index processing interface is called through the main data processing entry point to obtain the index lineage relationship of the index to be processed from the index lineage relationship table. The data processing step configuration table is updated based on the index lineage relationship. The processing interface of the scheduling tool is called through the index processing interface to execute the data processing steps according to the data processing step configuration table to obtain the index result of the index to be processed. By determining the data processing steps to be executed through the index to be processed and its index lineage relationship, the processing of the index to be processed is carried out, avoiding waste of computing and storage resources and improving computing efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of index processing, and in particular to an index processing method, an index processing apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In the development of indicator-based data analysis products, data processing procedures are written for all indicators according to business requirements. This results in a single data processing step encompassing the indicators and their common data, which is then scheduled by a processing interface of a scheduling tool. However, in product applications, users may only need specific indicators. To meet personalized indicator needs, existing technologies typically employ two methods: full-processing for personalized display and manual selection of indicator processing steps. Full-processing wastes significant time on unnecessary work, wasting both computational and storage resources. Manual processing is error-prone and inefficient. Summary of the Invention

[0003] The purpose of this invention is to provide an index processing method, apparatus, device, and storage medium, which are applied in the field of index processing. This method determines the data processing steps to be performed by the index to be processed and the index lineage, and then processes the index to be processed, avoiding the waste of computing and storage resources, and improving computing efficiency and accuracy.

[0004] To solve the above-mentioned technical problems, the present invention provides an index processing method, comprising:

[0005] The product code package based on indicator-based data analysis products organizes the data processing flow of indicators into data processing steps, and configures the data processing steps into the data processing step configuration table of the scheduling tool.

[0006] A metric lineage table is generated based on the product code package. The metric to be processed is input into the data processing entry point, and the metric processing interface is called through the data processing entry point.

[0007] The indicator lineage relationship of the indicator to be processed is obtained from the indicator lineage relationship table through the indicator processing interface, and the data processing step configuration table is updated based on the indicator lineage relationship;

[0008] The indicator processing interface calls the processing interface of the scheduling tool to execute the data processing steps according to the data processing step configuration table to obtain the indicator result of the indicator to be processed.

[0009] Optionally, a metric lineage table is generated based on the product code package, including:

[0010] Parsing the product code package yields all the developed metrics and the data processing procedures for all the metrics;

[0011] Identify the data tables used in the data processing of all the aforementioned indicators, and determine the dependencies between the data processing processes based on the data tables;

[0012] Starting with the data processing process of each indicator, the indicator lineage relationship of each indicator is obtained by retrieval based on the dependency relationship, and the indicator lineage relationship table is constructed based on the indicator lineage relationship.

[0013] Optionally, determining the dependencies between the data processing procedures based on the data table includes:

[0014] Determine the operation type of the data table during the data processing; the operation type includes write operations and read operations.

[0015] The dependencies between the data processing procedures are determined based on the operation type.

[0016] Optionally, the data processing step configuration table includes a data processing procedure field and a valid flag field;

[0017] The data processing process field is used to record the name of the data processing process in the data processing steps;

[0018] The validity flag field is used to characterize whether the data processing process is valid.

[0019] Optionally, updating the data processing step configuration table based on the indicator lineage includes:

[0020] When the indicator to be processed is a full indicator, then all the valid flag fields in the data processing step configuration table are set to valid flags;

[0021] When the indicator to be processed is not a full indicator, the valid flag field of the data processing process in the indicator lineage is set to the valid flag.

[0022] Optionally, the indicators to be processed are input into the overall data processing entry point, including:

[0023] Enter the list of indicator codes into the main data processing entry point;

[0024] If the indicator code list is empty, then all the indicators will be identified as the indicators to be processed.

[0025] If the indicator code list is not empty and contains a positive selection flag, then the indicator that matches the indicator code list is determined as the indicator to be processed.

[0026] If the indicator code list is not empty and contains a reverse selection flag, then the indicator that does not match the indicator code list is identified as the indicator to be processed.

[0027] Optionally, the method further includes:

[0028] When the indicator processing interface identifies a non-indicator data analysis product, it directly calls the processing interface of the scheduling tool.

[0029] To solve the above-mentioned technical problems, the present invention provides an index processing method, comprising:

[0030] The first module is used to organize the data processing flow of indicators into data processing steps based on the product code package of indicator-based data analysis products, and configure the data processing steps into the data processing step configuration table of the scheduling tool.

[0031] The second module is used to generate an indicator lineage table based on the product code package, input the indicators to be processed into the data processing main entry point, and call the indicator processing interface through the data processing main entry point.

[0032] The third module is used to obtain the indicator lineage relationship of the indicator to be processed from the indicator lineage relationship table through the indicator processing interface, and update the data processing step configuration table based on the indicator lineage relationship;

[0033] The fourth module is used to call the processing interface of the scheduling tool through the indicator processing interface to execute the data processing steps according to the data processing step configuration table to obtain the indicator result of the indicator to be processed.

[0034] To solve the above-mentioned technical problems, the present invention provides an electronic device, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor is used to implement the above-described index processing method when executing the computer program.

[0037] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned index processing method.

[0038] As can be seen, this invention, based on the product code package of an indicator-based data analysis product, organizes the data processing flow of indicators into data processing steps, and configures these steps into the data processing step configuration table of the scheduling tool. It generates an indicator lineage table based on the product code package, inputs the indicator to be processed into the main data processing entry point, and calls the indicator processing interface through this entry point. The indicator processing interface retrieves the indicator lineage relationship of the indicator to be processed from the indicator lineage relationship table, and updates the data processing step configuration table based on this relationship. Finally, the indicator processing interface calls the processing interface of the scheduling tool to execute the data processing steps according to the data processing step configuration table, obtaining the indicator result for the indicator to be processed. This invention determines the data processing steps to be executed by identifying the indicator to be processed and its lineage relationship, thereby processing the indicator and avoiding waste of computing and storage resources, while improving computational efficiency and accuracy. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 This is an example diagram of an indicator data retrieval process provided in an embodiment of the present invention;

[0041] Figure 2 This is an example diagram illustrating a data processing procedure call provided in an embodiment of the present invention;

[0042] Figure 3 A flowchart of an index processing method provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of an index processing scheme provided in an embodiment of the present invention;

[0044] Figure 5 This is a structural block diagram of an index processing device provided in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Metric-based data analytics products can contain numerous metrics. Each metric's business logic is implemented through a written SQL (Structured Query Language) script that runs on a database management system. Each metric's SQL script is encapsulated as a data processing procedure using a stored procedure. Then, all the stored procedures for all metrics within a product are packaged into a single product code package. This creates a one-to-one correspondence between metric-based data analytics products and product code packages. Subsequent data processing is performed by calling the data processing procedures within the product code package.

[0047] like Figure 1 As shown, some indicators can be directly obtained from the raw data of systems such as data warehouses, while others, for the sake of consistency of definition, data consistency, and processing performance, will be obtained from the pre-processed public data of the current product.

[0048] Risk-related and other data analysis products are similar to indicator-based data analysis products in terms of processing and implementation. They also use stored procedures and packages to implement specific business logic. The main difference lies in the final deliverables of the products. Unlike indicator-based products, which output a result for each indicator, risk-related products only have one fixed final result. All the data processing in the early stages is ultimately aimed at outputting whether the data is "risky" or "risk-free".

[0049] Regardless of the type of data analysis product, there are two methods for invoking the data processing procedures within the product: First, manually calling each step one by one within the database management system. Second, after development, the sequential data processing steps of the product can be obtained according to data lineage, with each step corresponding to a data processing procedure. These steps are then configured into the scheduling tool via a configuration interface (usually a configuration table). During data processing, the product's main data processing entry point is first called, passing in specific parameters, which then calls the processing interface provided by the scheduling tool to perform the data processing. Figure 2 As shown.

[0050] The advantage of the second method is that during product development, you only need to focus on writing the corresponding data processing procedures based on the business requirements of the indicators. The scheduling and processing operations are completed by the scheduling tool. The scheduling tool can support different products and realizes the separation of business logic and scheduling processing. It is currently the main way to implement data processing.

[0051] A special case exists with indicator-based data analysis products: during product development, data processing procedures are written for all indicators according to business requirements. This results in a data processing set that includes all data processing steps for the indicators and their common data, which is then scheduled and processed by the processing interface of the scheduling tool.

[0052] However, during product deployment, some users often don't need all the metrics; they request that only specific metrics be processed. This demand for personalized metric processing becomes even more prominent in the pre-sales stage. Users carefully select a few metrics and compare the processed results with their own experience data to verify the creativity of the metric's business design and whether it truly identifies problems. Therefore, the main problem this invention aims to solve is to flexibly respond to user requests by processing only certain metrics without increasing workload.

[0053] Existing technologies typically employ two methods to meet users' personalized metric processing needs:

[0054] (1) Full processing: This means that in the data processing stage, the processing interface of the scheduling tool is directly called to process all indicators according to the complete data processing steps. However, in the presentation stage, only the results of the indicators selected by the user are displayed. Full processing wastes both computing and storage resources.

[0055] (2) Manual selection of data processing procedures: This involves manually identifying the data processing procedures corresponding to the indicators selected by the user and the common data used by these indicators. Then, these data processing procedures are either manually called one by one, or the data processing steps in the scheduling tool are manually modified before calling the scheduling tool's processing interface to process the selected indicators. Even with manual methods, if the data processing procedures are called one by one manually, the common data used by many indicators will inevitably be called repeatedly when there are many selected indicators, resulting in redundant processing and wasted computing resources. Finding the corresponding data processing procedures for the selected indicators and then either calling them one by one or modifying the data processing steps is a very tedious and error-prone process. When the user selects many indicators, or when the processing of the indicators involves multiple common data, the efficiency of manual methods is undoubtedly very low.

[0056] The following combination Figure 3 , Figure 3 A flowchart of an index processing method provided in an embodiment of the present invention, the method may include:

[0057] S101: The product code package for indicator-based data analysis products organizes the data processing flow of indicators into data processing steps and configures the data processing steps into the data processing step configuration table of the scheduling tool.

[0058] like Figure 4 As shown, this embodiment can organize all data processing processes in the product code package into data processing steps with a sequential order according to the data processing dependencies, and configure them into the data processing step configuration table through the interface provided by the scheduling tool.

[0059] The data processing step configuration table includes two key fields: the data processing procedure field and the validity flag field. The data processing procedure field (SJJGGC) records the name of the data processing procedure in the data processing steps. The validity flag field (YXBZ) indicates whether the data processing procedure is valid. The processing interface of the scheduling tool will only call the data processing procedure of the valid data processing step.

[0060] S102: Generate an indicator lineage table based on the product code package, input the indicators to be processed into the main data processing entry point, and call the indicator processing interface through the main data processing entry point.

[0061] like Figure 4 As shown, this embodiment can generate an indicator lineage table based on the product code package. The indicator lineage table records the data processing process of each indicator from the most original data warehouse data to the intermediate indicator public data to the final indicator result data.

[0062] by Figure 1 For example, the indicators of blood relations are shown in the table below:

[0063] Table 1: Examples of Blood Relationships

[0064]

[0065] This embodiment does not limit the specific method of generating the indicator lineage table. Generally, the product code package can be parsed to obtain all developed indicators and the data processing process of all indicators.

[0066] The data tables used in the data processing of all indicators are determined, and the dependencies between data processing processes are determined based on the data tables. Specifically, the operation types of the data tables in the data processing process can be determined, including write operations and read operations. The dependencies between data processing processes are determined based on the operation types.

[0067] Starting with the data processing of each indicator, the indicator lineage relationship of each indicator is obtained by searching based on the dependency relationship, and an indicator lineage relationship table is constructed based on the indicator lineage relationship.

[0068] In other words, during the generation of the metric lineage table, the first step is to obtain the developed metrics. For example, by reading the Oracle (a database) data dictionary `user_procedures`, the names of all data processing procedures for the current product can be obtained, and by reading the Oracle data dictionary `user_dependencies`, all tables used by the current product can be obtained. Combining these two sets of data, the tables involved in each data processing procedure and the operation type of these tables in that data processing procedure—whether it's a "write operation" or a "read operation"—can be obtained. Then, in the current product package, all data processing procedures whose operation type is "write operation" for tables with "read operations" are searched, thus obtaining the data processing procedures that each data processing procedure depends on.

[0069] Furthermore, starting with the data processing process of each indicator in "Obtaining Developed Indicators", we search upwards layer by layer from the data processing process dependency relationship obtained above, thus obtaining the complete data processing lineage of each indicator.

[0070] The indicator lineage configuration involves writing the data obtained during indicator lineage generation into the indicator lineage table (WD_ZBXYGX). The structure of the indicator lineage table can be shown in Table 2.

[0071] Table 2: Examples of Blood Relationships

[0072]

[0073] by Figure 1 Taking indicator 1 as an example, its record in the indicator kinship table is shown in Table 3:

[0074] Table 3: Bloodline Relationship of Indicator 1

[0075]

[0076] In this embodiment, the indicator to be processed can be input into the main data processing entry point, and the indicator processing interface can be called through the main data processing entry point.

[0077] Each data analytics product has a central entry point for data processing, which accepts input parameters that differ from those of other products, such as the analysis period and the analysis object. For indicator-based data analytics products, this can also be a list of indicator codes for indicators to be executed (or not to be executed). After processing the input parameters, the product obtains the indicator to be processed and then calls the indicator processing interface.

[0078] The indicator processing interface, acting as an intermediate layer, encapsulates the processing interface of the scheduling tool, enabling flexible and universal data processing. The indicator processing interface first determines whether the current product is an indicator based on an inherent attribute of each product. If not, it directly calls the scheduling tool's processing interface. In other words, when the indicator processing interface identifies a non-indicator data analysis product, it directly calls the scheduling tool's processing interface; otherwise, before calling the scheduling tool's processing interface, it sequentially calls two processes: obtaining the indicator lineage and modifying the scheduling tool's data processing steps through the configuration interface.

[0079] In this embodiment, the indicator to be processed can be determined by the indicator code list input at the main data processing entry point. The input parameters of the indicator code list determine which indicators to process. This process is used to obtain the data processing lineage of these indicators.

[0080] Specifically, in this embodiment, the list of indicator codes can be input into the main data processing entry point.

[0081] If the indicator code list is empty, all indicators will be designated as indicators to be processed.

[0082] If the indicator code list is not empty and contains a positive selection flag, the indicator that matches the indicator code list will be identified as the indicator to be processed.

[0083] If the indicator code list is not empty and contains a reverse selection flag, then indicators that do not match the indicator code list will be identified as indicators to be processed.

[0084] For example, the indicator code list can be a string concatenated with indicator codes separated by "|" (if there is only a single indicator, simply pass in the indicator code for that single indicator). The following three different input formats support three different indicator processing rules:

[0085] First, if the list of indicator codes passed in is empty, it means that all indicators should be processed.

[0086] The second option is to pass in a list of indicator codes that begin with the positive selection flag "#", which indicates a positive selection and means that the indicators in the indicator lineage table that match this parameter will be processed.

[0087] The third option is to pass in a list of indicator codes that begin with the reverse selection flag "^", which indicates reverse selection and means processing indicators in the indicator lineage table that do not match this parameter.

[0088] Based on the above rules, using Oracle's regular expressions, all the metrics to be processed, as well as the corresponding data processing procedures, can be obtained from the metric lineage table through a simple SQL query.

[0089] S103: Obtain the indicator lineage relationship of the indicator to be processed from the indicator lineage relationship table through the indicator processing interface, and update the data processing step configuration table based on the indicator lineage relationship.

[0090] like Figure 4 As shown, in this embodiment, the indicator lineage relationship of the indicator to be processed can be obtained from the indicator lineage relationship table through the indicator processing interface, and the data processing step configuration table can be updated based on the indicator lineage relationship.

[0091] Specifically, when the indicator to be processed is a full indicator, all valid flag fields in the data processing step configuration table will be set to valid flags.

[0092] When the indicator to be processed is not a full indicator, the valid flag field of the data processing process in the indicator lineage is set as a valid flag.

[0093] When the list of input indicator codes is empty, set the "valid flag" of all data processing steps in the data processing step configuration table of the scheduling tool to "valid".

[0094] When the list of input indicator codes is not empty, the value of the SJJGGC field in the data processing step configuration table of the scheduling tool is set to "valid" for the data processing step obtained in the process of obtaining indicator lineage, and all others are set to "invalid".

[0095] This embodiment can be accomplished using Oracle's regular expressions.

[0096] S104: By calling the processing interface of the scheduling tool through the indicator processing interface, the data processing steps are executed according to the data processing step configuration table to obtain the indicator results of the indicator to be processed.

[0097] After updating the valid flag fields of the data processing step configuration table, this embodiment can call the processing interface of the scheduling tool through the indicator processing interface to execute the data processing steps according to the updated data processing step configuration table to obtain the indicator results of the indicator to be processed.

[0098] The main advantages of this plan are:

[0099] (1) Transparent to the outside world: Data development engineers in each region no longer need to worry about how to specifically implement the user's differentiated indicator processing needs. When the user has differentiated indicator processing needs, they only need to pass in the appropriate parameters to the product's data processing entry point. If the user does not have differentiated indicator processing needs, they can even ignore the input parameters.

[0100] (2) General: By using the indicator processing interface as an intermediary layer, the product and the scheduling tool are isolated, and the two systems are fully decoupled, so that no modifications are needed at the functional level for either system (only indicator-type products need to add input parameters, and other types of products do not need to be modified). In addition, this intermediary layer automatically decides whether to directly call the processing interface of the scheduling tool or modify the effective flag in the data processing steps according to the inherent attributes of the product, so as to ensure that only indicators that match the input parameters are executed. This allows seamless access for any type of product and is a general method for data processing of all data analysis products.

[0101] (3) Reduce processing time and resource consumption: Compared with full processing of indicators, the present invention can significantly reduce processing time and resource consumption by automatically and accurately matching the data processing process to be performed based on the indicator lineage. The present invention also avoids the resource waste caused by repeated processing that may occur when manually calling the data processing process.

[0102] (4) Improve work efficiency: By organizing the input parameters, users can process differentiated indicators, which can greatly improve work efficiency compared to manually calling the data processing process one by one or manually modifying the data processing steps.

[0103] (5) Reduce errors: Avoids errors and omissions in manual operation.

[0104] Based on the above embodiments, the present invention determines the data processing steps to be performed by the indicators to be processed and their lineage, and then processes the indicators to be processed, thereby avoiding the waste of computing and storage resources and improving computing efficiency and accuracy.

[0105] The following combination Figure 5 , Figure 5 This is a structural block diagram of an index processing device provided in an embodiment of the present invention. The device may include:

[0106] The first module 100 is used to organize the data processing flow of indicators into data processing steps in the product code package of indicator-based data analysis products, and configure the data processing steps into the data processing step configuration table of the scheduling tool.

[0107] The second module 200 is used to generate an indicator lineage table based on the product code package, input the indicators to be processed into the data processing main entry point, and call the indicator processing interface through the data processing main entry point.

[0108] The third module 300 is used to obtain the indicator lineage relationship of the indicator to be processed from the indicator lineage relationship table through the indicator processing interface, and update the data processing step configuration table based on the indicator lineage relationship;

[0109] The fourth module 400 is used to call the processing interface of the scheduling tool through the indicator processing interface to execute the data processing steps according to the data processing step configuration table and obtain the indicator results of the indicator to be processed.

[0110] Based on the above embodiments, the present invention determines the data processing steps to be performed by the indicators to be processed and their lineage, and then processes the indicators to be processed, thereby avoiding the waste of computing and storage resources and improving computing efficiency and accuracy.

[0111] Based on the above embodiments, the second module 200 may include:

[0112] The first unit is used to parse the product code package to obtain all developed metrics and the data processing procedures for all metrics;

[0113] The second unit is used to determine the data tables used in the data processing of all indicators, and to determine the dependencies between the data processing processes based on the data tables.

[0114] The third unit is used to retrieve the indicator lineage relationship of each indicator based on the dependency relationship, starting from the data processing process of each indicator, and to construct the indicator lineage relationship table based on the indicator lineage relationship.

[0115] Based on the above embodiments, the second unit may include:

[0116] The first sub-unit is used to determine the operation type of the data table during the data processing process; the operation type includes write operations and read operations.

[0117] The second subunit is used to determine the dependencies between data processing procedures based on the operation type.

[0118] Based on the above embodiments, the data processing step configuration table includes a data processing process field and a valid flag field;

[0119] The data processing procedure field is used to record the name of the data processing procedure in the data processing steps;

[0120] The validity flag field is used to characterize whether the data processing process is valid.

[0121] Based on the above embodiments, the third module 300 may include:

[0122] The fourth unit is used to set all valid flag fields in the data processing step configuration table to valid flags when the indicator to be processed is a full indicator.

[0123] The fifth unit is used to set the valid flag field of the data processing process in the indicator lineage as a valid flag when the indicator to be processed is not a full indicator.

[0124] Based on the above embodiments, the second module 200 may include:

[0125] The sixth unit is used to input the list of indicator codes into the main data processing entry point;

[0126] Unit 7 is used to identify all indicators as indicators to be processed when the indicator code list is empty;

[0127] Unit 8 is used to identify indicators that match the indicator code list as indicators to be processed when the indicator code list is not empty and contains a positive selection flag.

[0128] Unit 9 is used to identify indicators that do not match the indicator code list as indicators to be processed when the indicator code list is not empty and contains a reverse selection flag.

[0129] Based on the above embodiments, the device may further include:

[0130] The fifth module is used to directly call the processing interface of the scheduling tool when the indicator processing interface identifies a non-indicator data analysis product.

[0131] Based on the above embodiments, the present invention also provides an electronic device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various necessary network interfaces, a power supply, and other components.

[0132] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an execution terminal or processor, can implement the method provided in the embodiments of the present invention; the storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0133] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for processing indicators, characterized in that, include: The product code package based on indicator-based data analysis products organizes the data processing flow of indicators into data processing steps, and configures the data processing steps into the data processing step configuration table of the scheduling tool. A metric lineage table is generated based on the product code package. The metric to be processed is input into the data processing entry point, and the metric processing interface is called through the data processing entry point. The indicator lineage relationship of the indicator to be processed is obtained from the indicator lineage relationship table through the indicator processing interface, and the data processing step configuration table is updated based on the indicator lineage relationship; The indicator processing interface calls the processing interface of the scheduling tool to execute the data processing steps according to the data processing step configuration table to obtain the indicator result of the indicator to be processed.

2. The index processing method according to claim 1, characterized in that, Generate an indicator lineage table based on the product code package, including: Parsing the product code package yields all the developed metrics and the data processing procedures for all the metrics; Identify the data tables used in the data processing of all the aforementioned indicators, and determine the dependencies between the data processing processes based on the data tables; Starting with the data processing process of each indicator, the indicator lineage relationship of each indicator is obtained by retrieval based on the dependency relationship, and the indicator lineage relationship table is constructed based on the indicator lineage relationship.

3. The index processing method according to claim 2, characterized in that, Determining the dependencies between the data processing procedures based on the data table includes: Determine the operation type of the data table during the data processing; the operation type includes write operations and read operations. The dependencies between the data processing procedures are determined based on the operation type.

4. The index processing method according to claim 1, characterized in that, The data processing step configuration table includes a data processing procedure field and a valid flag field; The data processing process field is used to record the name of the data processing process in the data processing steps; The validity flag field is used to characterize whether the data processing process is valid.

5. The index processing method according to claim 4, characterized in that, Updating the data processing step configuration table based on the aforementioned kinship indicator includes: When the indicator to be processed is a full indicator, then all the valid flag fields in the data processing step configuration table are set to valid flags; When the indicator to be processed is not a full indicator, the valid flag field of the data processing process in the indicator lineage is set to the valid flag.

6. The index processing method according to claim 1, characterized in that, The indicators to be processed are input into the main data processing entry point, including: Enter the list of indicator codes into the main data processing entry point; If the indicator code list is empty, then all the indicators will be identified as the indicators to be processed. If the indicator code list is not empty and contains a positive selection flag, then the indicator that matches the indicator code list is determined as the indicator to be processed. If the indicator code list is not empty and contains a reverse selection flag, then the indicator that does not match the indicator code list is identified as the indicator to be processed.

7. The index processing method according to claim 1, characterized in that, Also includes: When the indicator processing interface identifies a non-indicator data analysis product, it directly calls the processing interface of the scheduling tool.

8. An index processing device, characterized in that, include: The first module is used to organize the data processing flow of indicators into data processing steps based on the product code package of indicator-based data analysis products, and configure the data processing steps into the data processing step configuration table of the scheduling tool. The second module is used to generate an indicator lineage table based on the product code package, input the indicators to be processed into the data processing main entry point, and call the indicator processing interface through the data processing main entry point. The third module is used to obtain the indicator lineage relationship of the indicator to be processed from the indicator lineage relationship table through the indicator processing interface, and update the data processing step configuration table based on the indicator lineage relationship; The fourth module is used to call the processing interface of the scheduling tool through the indicator processing interface to execute the data processing steps according to the data processing step configuration table to obtain the indicator result of the indicator to be processed.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the index processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the index processing method as described in any one of claims 1 to 7.

Images