Data processing method and device, equipment, storage medium and program product

Abstract

The invention discloses a data processing method and device, equipment, a storage medium and a program product. In response to a first configuration operation input by a user to the target table, configuring an upstream data source associated with the target table, the upstream data source comprising a target client table and a target performance table, and the target client table and the target performance table comprising at least one index; in response to a second configuration operation of the user on each index in the target table, configuring a calculation rule of each index associated with the target table; analyzing a target client table, a target performance table and a calculation rule of each index associated with the target table, and generating a structured query language script; executing the structured query language script, generating a target table, and inserting the target table into a mapping relation sub-table of a physical table and a view of a preset global parameter table to obtain a target view; according to the target view, a database server is connected and accessed to obtain data of the target client table and the target performance table, processing is carried out according to the calculation rules of the indexes, and a target report corresponding to the target table is generated.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a data processing method, apparatus, device, storage medium and program product. Background Technology

[0002] With the deepening of digital operations, financial institutions' headquarters need to conduct comprehensive performance evaluations and analyses of their branches across the country. Comparative analysis reveals a high degree of overlap in the performance evaluation content across branches. However, due to differences in their own operations, each branch's performance evaluation needs to be tailored to its specific circumstances. This results in the performance evaluation analysis work for each branch exhibiting insufficient functional universality, poor scalability, and a high degree of coupling with the branch's own needs, making it difficult to directly share and promote across all institutions. Summary of the Invention

[0003] This application provides a data processing method, apparatus, device, storage medium, and program product that can meet the needs of each branch while also being shared across the entire bank.

[0004] In a first aspect, embodiments of this application provide a data processing method, the method comprising: In response to a user's first configuration operation on a target table, the upstream data source associated with the target table is configured, the upstream data source including a target customer table and a target performance table, the target customer table and the target performance table including at least one indicator; In response to the user's second configuration operation on each of the indicators in the target table, configure the calculation rules for the target table associated with each of the indicators; Parse the target customer table, the target performance table, and the calculation rules of each indicator associated with the target table to generate a structured query language script; The structured query language script is executed to generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. Based on the target view, the database server is accessed to obtain data from the target customer table and the target performance table, and the data is processed according to the calculation rules of each indicator to generate the target report corresponding to the target table.

[0005] Secondly, embodiments of this application provide a data processing apparatus, the apparatus comprising: The first configuration module is used to configure the upstream data source associated with the target table in response to the user's first configuration operation on the target table. The upstream data source includes a target customer table and a target performance table, and the target customer table and the target performance table include at least one indicator. The second configuration module is used to configure the calculation rules for the target table associated with each of the indicators in response to the user's second configuration operation on each of the indicators in the target table; The parsing module is used to parse the target customer table, the target performance table, and the calculation rules of each indicator associated with the target table, and generate a structured query language script; The execution module is used to execute the structured query language script, generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. The generation module is used to connect to the database server to obtain data from the target customer table and the target performance table based on the target view, and process the data according to the calculation rules of each indicator to generate the target report corresponding to the target table.

[0006] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the data processing method described in any of the above.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the data processing method described in any of the above claims.

[0008] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the data processing method described in any of the above claims.

[0009] The data processing method, apparatus, device, storage medium, and program product of this application embodiment are capable of configuring the upstream data source associated with the target table in response to a first configuration operation input by a user on a target table. The upstream data source includes a target customer table and a target performance table, and the target customer table and target performance table include at least one indicator. In response to a second configuration operation by a user on each indicator in the target table, the calculation rules for each indicator associated with the target table are configured. The target customer table, target performance table, and calculation rules for each indicator associated with the target table are parsed to generate a structured query language script. The structured query language script is executed to generate a target table and insert it into a physical table and view mapping relationship sub-table of a preset global parameter table to obtain a target view. The physical table and view mapping relationship sub-table is used to connect to and access a database server. Based on the target view, the database server is connected to and accessed to obtain data from the target customer table and target performance table, and the data is processed according to the calculation rules for each indicator to generate a target report corresponding to the target table. Thus, in this embodiment, the financial institution headquarters only needs to make adaptive configurations from the uniformly maintained customer table, performance table, and calculation rules of each indicator according to the operational needs of each branch. This allows for high coupling with its own needs. Furthermore, since the customer table, performance table, and calculation rules of each indicator are unified, they can be universally shared in the performance evaluation of each branch. Moreover, the calculation rules of the customer table, performance table, and each indicator can also be changed according to actual needs, exhibiting high scalability. Then, by parsing the configuration information, a structured query language script is generated. Even if there are business changes, there is no need to refactor the code; only the configuration needs to be modified, making the calculation logic more flexible. Finally, the target view serves as a unified access point. Downstream database servers do not need to adapt to the physical table access rules of different branches; they only need to obtain standardized data through the view. This not only ensures the security and consistency of data access but also reduces the complexity of cross-system access. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the data processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the configurable computing framework supported by the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0013] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0014] With the deepening of digital operations, financial institutions' headquarters need to conduct comprehensive performance evaluations and analyses of their branches across the country. Comparative analysis reveals a high degree of overlap in the performance evaluation content across branches. However, due to differences in their own operations, each branch's performance evaluation needs to be tailored to its specific circumstances. This results in the performance evaluation analysis work for each branch exhibiting insufficient functional universality, poor scalability, and a high degree of coupling with the branch's own needs, making it difficult to directly share and promote across all institutions.

[0015] The acquisition, storage, use, and processing of data in this application comply with relevant national laws and regulations. It should be noted that certain software, components, models, and other existing industry solutions may be mentioned in the embodiments of this application. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0016] To address the problems of the prior art, embodiments of this application provide a data processing method, apparatus, device, storage medium, and program product. The data processing method provided in this application embodiment will be described first below.

[0017] Figure 1 A flowchart illustrating a data processing method provided in one embodiment of this application is shown. Figure 1 As shown, a data processing method may include the following steps S101 to S105: S101. In response to the user's first configuration operation on the target table, configure the upstream data source associated with the target table. The upstream data source includes the target customer table and the target performance table, and the target customer table and the target performance table include at least one indicator. S102. In response to the user's second configuration operation on each indicator in the target table, configure the calculation rules for each indicator associated with the target table; S103. Parse the target customer table, target performance table and calculation rules of each indicator associated with the target table, and generate a structured query language script; S104. Execute the structured query language script to generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. S105. Based on the target view, connect to the database server to obtain data from the target customer table and the target performance table, process the data according to the calculation rules of each indicator, and generate the target report corresponding to the target table.

[0018] The data processing method of this application embodiment is capable of configuring the upstream data source associated with the target table in response to a first configuration operation input by the user on the target table. The upstream data source includes a target customer table and a target performance table, and the target customer table and target performance table include at least one indicator. In response to a second configuration operation by the user on each indicator in the target table, the method configures the calculation rules for each indicator associated with the target table. The method parses the target customer table, target performance table, and calculation rules for each indicator associated with the target table to generate a structured query language script. The method executes the structured query language script to generate the target table and insert it into a physical table and view mapping relationship sub-table of a preset global parameter table to obtain a target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. Based on the target view, the method connects to and accesses the database server to obtain the data of the target customer table and target performance table, and processes it according to the calculation rules of each indicator to generate a target report corresponding to the target table. Thus, in this embodiment, the financial institution headquarters only needs to make adaptive configurations from the uniformly maintained customer table, performance table, and calculation rules of each indicator according to the operational needs of each branch. This allows for high coupling with its own needs. Furthermore, since the customer table, performance table, and calculation rules of each indicator are unified, they can be universally shared in the performance evaluation of each branch. Moreover, the calculation rules of the customer table, performance table, and each indicator can also be changed according to actual needs, exhibiting high scalability. Then, by parsing the configuration information, a structured query language script is generated. Even if there are business changes, there is no need to refactor the code; only the configuration needs to be modified, making the calculation logic more flexible. Finally, the target view serves as a unified access point. Downstream database servers do not need to adapt to the physical table access rules of different branches; they only need to obtain standardized data through the view. This not only ensures the security and consistency of data access but also reduces the complexity of cross-system access.

[0019] The aforementioned data processing methods are primarily applied to application servers. Since application servers can configure, parse script generation, and execute calculation rules, they require multi-core CPUs (≥8 cores) and large-capacity memory (≥16GB) to support high-concurrency configuration operations and computational tasks. In some embodiments of this application, if the indicator calculation rules are complex or the data volume is large, dedicated computing nodes (such as distributed server clusters) can be added to improve data processing and report generation speed through parallel computing.

[0020] In S101, the aforementioned target table can be a target table corresponding to a financial institution. For example, the target table can be a financial performance table.

[0021] The aforementioned upstream data sources may include a target customer table and a target performance table. For example, the target customer table may be a customer information table, and the target performance table may be a transaction record table.

[0022] The target customer table and the target performance table each include at least one metric. For example, the target customer table may include customer attribution metrics. The target performance table may include performance amount metrics.

[0023] In some embodiments of this application, in response to a user's first configuration operation on a target table, the upstream data source associated with the target table is configured. For example, in response to a user's first configuration operation on a target table, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in a table-level mapping sub-table.

[0024] In S102, the aforementioned calculation rules, for example, can be basic operation rules, conditional judgment rules, or aggregation analysis rules, etc. Specific calculation rules can be set according to the user's actual needs, and are not specifically limited here.

[0025] In some embodiments of this application, in response to a user's second configuration operation on each indicator in the target table, the calculation rules for the target table associated with each indicator are configured. For example, in response to a user's second configuration operation on each indicator in the target table, the calculation rules for the target table associated with each indicator are configured in a field-level mapping sub-table.

[0026] In S103, in some embodiments of this application, the target customer table, target performance table, and calculation rules of each indicator associated with the target table are parsed to generate a structured query language script. For example, the parsing engine can be configured to use the target table as an index to read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the English name of the target table, the upstream table database, the English name of the upstream table, the alias of the upstream table, the upstream table association type, and the association filtering rule information in the table-level mapping sub-table to generate a table-level mapping statement; using the target table as an index, the mapping group sequence number, the field sequence number within the mapping group, the target table database, the English name of the target table, the upstream mapping rules, and the default value of the target field are read in the field-level mapping sub-table to generate a field-level mapping statement; the table-level mapping statement and the field-level mapping statement are concatenated to generate a structured query language script.

[0027] Alternatively, the parsing engine can be configured to use the target table as an index, and read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table alias, the upstream table's association type, and the association filter rule name from the table-level mapping sub-table. It can then query the first rule information of the association filter rule name from the preset high-frequency access configuration information, which is stored in the field-programmable gate array cluster. This high-frequency access configuration information includes multiple rule names and the rule information corresponding to each rule name. Based on the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table's English name, the upstream table database, and the upstream... The table-level mapping statement is generated from the target table's English name, upstream table alias, upstream table association type, and first rule information. Using the target table as an index, the mapping group number, field number within the mapping group, target table database, target table's English name, upstream mapping rule name, and target field default value are read from the field-level mapping sub-table. The second rule information of the upstream mapping rule name is queried from the high-frequency access configuration information. Based on the mapping group number, field number within the mapping group, target table database, target table's English name, second rule information, and target field default value, a field-level mapping statement is generated. The table-level and field-level mapping statements are concatenated to generate a structured query language script.

[0028] In S104, the aforementioned global parameter table can be the core data storage carrier supporting "configuration-driven data processing". Essentially, it is a set of structured database tables designed according to functional layers, used to uniformly store all configuration information related to data processing and efficiency evaluation across all financial institutions. It is a core technical component for realizing the overall general rules of all financial institutions, the configuration of branch-specific needs, and the sharing of configuration results across systems.

[0029] The aforementioned global parameter table may include a sub-table representing the mapping relationship between physical tables and views, which can be used to connect to and access the database server. For example, the sub-table representing the mapping relationship between physical tables and views can be represented by ta99_physm_with_wm_rel_w, and includes: the English name of the view and the English name of the target table.

[0030] The aforementioned database server can deploy relational databases (such as Oracle and MySQL) to store structured data such as target customer tables, target performance tables, and configuration information of target tables; it needs to be equipped with multi-core CPUs, large-capacity memory (≥32GB), and high-IOPS disks (such as SSDs) to ensure data read and write efficiency.

[0031] In some embodiments of this application, a structured query language script is executed to generate a target table and insert it into a preset global parameter table's physical table and view mapping relationship sub-table to obtain a target view. For example, this can be achieved by executing a structured query language script to generate a target table; inserting the target table into the preset global parameter table's physical table and view mapping relationship sub-table to establish an association between the target table and the target view; if a first status feedback message is received from the database server within a preset time period, it is determined that the target view has been generated; if no first status feedback message is received within the preset time period, the target table is inserted into the physical table and view mapping relationship sub-table again; if no first status feedback message is received within the preset time period multiple times consecutively, it is determined that the target table insertion has failed, and an alarm message is generated.

[0032] In S105, the aforementioned target report, exemplarily, can be a profit-generating report. Of course, in this embodiment, different target customer tables and target performance tables correspond to different target reports, which can be set according to the user's actual business needs, and are not specifically limited here.

[0033] In some embodiments of this application, the above-described method of connecting to and accessing a database server to obtain data from the target customer table and the target performance table based on the target view, and processing the data according to the calculation rules of each indicator to generate a target report corresponding to the target table, may specifically involve: connecting to and accessing a database server to obtain data from the target customer table and the target performance table based on the target view; inserting the data from the target customer table and the target performance table into the target table; and processing the data from the target customer table and the target performance table in the target table according to the calculation rules of each indicator in the target table to generate a target report corresponding to the target table.

[0034] In some embodiments, the global parameter table may further include a table-level mapping sub-table, and S101 may specifically include: In response to the user's first configuration operation on the target table, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in the table-level mapping sub-table.

[0035] The aforementioned table-level mapping sub-table, for example, can be represented by ta99_khjy_tablvl_mpng_parm_w, including: mapping group sequence number, table sequence number within the mapping group, target table database, target table English name, upstream table database, upstream table English name, upstream table alias, upstream table association type, and upstream table association filtering rules.

[0036] The aforementioned association types, for example, can be either a left join or an inner join. Of course, other association types are also possible in this application, and can be set according to actual needs; no specific limitations are made here.

[0037] In some embodiments of this application, in response to a user's first configuration operation inputting on a target table, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in a table-level mapping sub-table. For example, this may involve receiving a user's selection operation on a device terminal for multiple preset configuration units, where different configuration units correspond to different upstream data sources or association types; in response to the selection operation, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in the table-level mapping sub-table; the user's input operation on the upstream data source and association type of the target table is received on the device terminal; and in response to the input operation, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in the table-level mapping sub-table.

[0038] In this embodiment, by adding a table-level mapping sub-table to the global parameter table, the association logic between the target table and the upstream data source is separated from the core code and placed in a structured configuration carrier. Users only need to configure the upstream data source name, association type, and association fields corresponding to the target table in this sub-table to complete the first configuration operation. The table-level configuration defines the association relationship and automatically generates JOIN logic, avoiding the need to manually write complex SQL. This decouples the data source association logic from the core system code. When adding / adjusting the upstream data source, only the configuration record needs to be modified, without refactoring the code. At the same time, the table-level mapping sub-table supports the head office's preset general association configuration and the branch's customized configuration, which not only ensures the standardization of association rules across the entire bank but also adapts to the differentiated business needs of branches, greatly reducing the configuration threshold and maintenance costs and improving the system's adaptability to changes in data sources.

[0039] In some embodiments, the global parameter table above also includes a field-level mapping sub-table, and S102 above may specifically include: In response to the user's second configuration operation on each indicator in the target table, the calculation rules for the target table associated with each indicator are configured in the field-level mapping sub-table.

[0040] The above field-level mapping sub-table, for example, can be represented by ta99_khj_fldlvl_mpng_parm_w, including: mapping group sequence number, field sequence number within the mapping group, target table database, target table English name, upstream mapping rule, and target field default value.

[0041] In this embodiment, a field-level mapping sub-table is added to the global parameter table to structure and store the calculation rules of each indicator in the target table. Users only need to fill in "indicator name + calculation rule expression" in the sub-table to complete the second configuration operation. This design achieves deep decoupling between indicator calculation rules and core system logic. Modifying a single indicator rule only requires updating the sub-table configuration, without refactoring the code. At the same time, the field-level mapping sub-table supports the head office to preset general rules and branches to customize special parameters, which not only ensures the consistency of indicator standards across the entire bank, but also adapts to the differentiated business needs of branches, greatly reducing the configuration threshold and promotion cost, and improving the response speed of rule adjustments and the bank-wide reusability.

[0042] In some embodiments, the above-described S103 may specifically include: Using the target table as an index, read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the association filtering rule information from the table-level mapping sub-table, and generate the table-level mapping statement; Using the target table as an index, read the mapping group sequence number, the field sequence number within the mapping group, the target table database, the target table's English name, the upstream mapping rules, and the target field's default value from the field-level mapping sub-table, and generate a field-level mapping statement. The table-level mapping statement and the field-level mapping statement are concatenated to generate a structured query language script.

[0043] In some embodiments of this application, the above-mentioned method uses the target table as an index to read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table alias, the upstream table's association type, and association filtering rule information from the table-level mapping sub-table to generate a table-level mapping statement. For example, the target table's English name can be used as an index to read the mapping group configuration from the table-level parameter table (ta99_khjy_tablvl_mpng_parm_w), and sort it by the mapping group sequence number in ascending order, recording the mapping group sequence number, upstream and downstream table association rules, and other information. Based on the read table-level mapping configuration, an INSERT statement containing multi-table association logic is concatenated to generate an INSERT basic statement containing upstream and downstream table JOIN logic.

[0044] In some embodiments of this application, the target table is used as an index to read the mapping group sequence number, the field sequence number within the mapping group, the target table database, the English name of the target table, the upstream mapping rules, and the default values ​​of the target fields in the field-level mapping sub-table to generate a field-level mapping statement. For example, the target table's English name can be used as an index to read the field calculation rules from the field-level parameter table (ta99_khj_fldlvl_mpng_parm_w) and sort them by field sequence number in ascending order; the field calculation rules (such as SUM operation, CASE WHEN logic) are incorporated into the INSERT statement to supplement the default values ​​of the target fields.

[0045] In this embodiment, the target table is used as the unique index. Standardized configuration information of the table-level / field-level mapping sub-tables is read hierarchically to generate table-level mapping statements (carrying table association logic) and field-level mapping statements (carrying indicator calculation logic), which are then concatenated into a complete SQL script. This ensures the accuracy of the script logic through standardized configuration reading, achieves modular decoupling of the script through hierarchical statement generation, and significantly improves efficiency by replacing manual writing with automated concatenation. Simultaneously, the standardized generated script has a unified format and consistent standards, allowing for rapid reuse across all branches of the bank. This reduces script maintenance and promotion costs while ensuring compatibility with downstream toolchains, completely solving the technical problems of traditional manual script writing: "inefficiency, error-proneness, and lack of universality."

[0046] In some embodiments, the above-mentioned concatenation of table-level mapping statements and field-level mapping statements to generate a structured query language script may specifically include: The table-level mapping statement, field-level mapping statement, and preset delete statement are concatenated to generate a structured query language script.

[0047] The structured query language scripts mentioned above can be complete SQL scripts that include DELETE cleanup, multi-table JOIN, and field calculation logic.

[0048] In some embodiments of this application, the above-mentioned concatenation of table-level mapping statements, field-level mapping statements, and preset deletion statements to generate a structured query language script can be exemplarily achieved by concatenating table-level mapping statements and field-level mapping statements with a "DELETE FROM [target table name];" deletion statement to generate a structured query language script.

[0049] In this embodiment of the application, since the concatenation statement can be used to clear historical data when the script is rerun, the structured query language script generated by concatenating the table-level mapping statement, the field-level mapping statement, and the preset deletion statement can avoid data inconsistency.

[0050] In some embodiments, the above-described S103 may specifically include: Using the target table as an index, read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the association filter rule name from the table-level mapping sub-table; The first rule information associated with the filtering rule name is queried in the preset high-frequency access configuration information. The high-frequency access configuration information is stored in the field programmable gate array cluster. The high-frequency access configuration information includes multiple rule names and the rule information corresponding to each rule name. Based on the mapping group number, the table number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table's alias, the upstream table's association type, and the first rule information, generate a table-level mapping statement; Using the target table as an index, read the mapping group sequence number, the field sequence number within the mapping group, the target table database, the target table's English name, the upstream mapping rule name, and the target field's default value from the field-level mapping sub-table; Query the second rule information of the upstream mapping rule name in the high-frequency access configuration information; Generate field-level mapping statements based on the mapping group number, the field number within the mapping group, the target table database, the target table's English name, the second rule information, and the target field's default value. The table-level mapping statement and the field-level mapping statement are concatenated to generate a structured query language script.

[0051] The aforementioned high-frequency access configuration information may include multiple rule names and the rule information corresponding to each rule name. These multiple rule names and their corresponding rule information may be rule names and rule information whose access frequency exceeds a preset access count.

[0052] High-frequency access configuration information is stored in a Field-Programmable Gate Array (FPGA) cluster. In this way, high-frequency access configuration information that may be associated with upstream data sources is pre-aggregated and stored at the hardware level according to the "target table association dimension" (such as customer ID). Through the direct memory access technology of the FPGA card, data of the same dimension scattered in different database tables is pre-merged into "customer dimension data blocks" and stored in continuous physical blocks of distributed storage. When the query script is executed, the pre-merged data blocks are read directly (instead of random reading of multiple tables), which transforms random IO of multi-table association into continuous IO, improving IO efficiency by more than 10 times.

[0053] In this embodiment, by pre-setting high-frequency access configuration information, including multiple rule names and corresponding rule information, the system can directly query and read the first rule information of the associated filter rule name and the second rule information of the upstream mapping rule name from the high-frequency access configuration information when parsing the target customer table, target performance table, and calculation rules of each indicator associated with the target table to generate a structured query language script. This improves the generation efficiency of the structured query language script. Furthermore, storing the high-frequency access configuration information in a field-programmable gate array (FPGA) cluster provides hardware-level pre-aggregated storage, supporting performance for random access to data from multiple tables and further improving data processing efficiency.

[0054] In some embodiments, prior to S103, the method may further include: Send the target customer table, target performance table, and calculation rules for each indicator associated with the target table to the application-specific integrated circuit (ASIC). Specifically, S103 mentioned above may include: Application-specific integrated circuits (ASICs) can parse the target customer table, target performance table, and calculation rules of each indicator associated with the target table based on the preset high-frequency access configuration information in the field-programmable gate array cluster, and generate structured query language scripts and hardware acceleration instructions. Send hardware acceleration commands to the field-programmable gate array cluster; Specifically, S104 mentioned above may include: The field-programmable gate array (FPGA) cluster responds to hardware acceleration commands, executes a structured query language script, generates a target table, and inserts it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view.

[0055] The aforementioned application-specific integrated circuits, also known as ASICs, are customized integrated circuit chips designed and manufactured specifically for a particular application scenario, task, or user need.

[0056] In some embodiments of this application, the above-mentioned sending of the target customer table, target performance table associated with the target table, and the calculation rules of each indicator to the application-specific integrated circuit (ASIC) can, for example, be done after the user completes the first / second configuration operation, by directly sending the target customer table, target performance table associated with the target table, and the calculation rules of each indicator to the ASIC card, which is then solidified into a hardware instruction set and integrated into the application-specific integrated circuit (ASIC) chip.

[0057] In some embodiments of this application, the application-specific integrated circuit (ASIC) can parse the target customer table, target performance table, and calculation rules of each indicator associated with the target table based on the preset high-frequency access configuration information in the field-programmable gate array (FPGA) cluster, and generate a structured query language script and hardware acceleration instructions. For example, the ASIC card can parse the "data source association relationship" and "indicator calculation logic" in parallel by hardware circuits, and generate an SQL script within 1ms (replacing the millisecond-level serial parsing of the CPU). At the same time, the ASIC card outputs the hardware acceleration instructions for script execution and directly passes them to the FPGA of the computing node, which can skip the script interpretation process of the software layer, thereby realizing a hardware-level pipeline of "configuration → script → execution".

[0058] In this embodiment, the script generation and execution process is migrated to an ASIC+FPGA hardware cluster: First, the ASIC receives the data source and indicator rules associated with the target table, and combines them with the high-frequency access configuration information in the FPGA cluster to parse and generate SQL scripts and hardware acceleration instructions; then, the FPGA cluster responds to the acceleration instructions and executes the script in parallel, generating the target table and establishing view associations. Thus, the ASIC-customized hardware achieves extreme efficiency in rule parsing and script generation, while the parallel computing power of the FPGA cluster achieves high concurrency and low latency in script execution, significantly reducing the CPU load on the core server. Simultaneously, the ASIC's fixed general rules ensure efficiency, the FPGA's adaptation to specific needs ensures flexibility, and the independent deployment of the hardware cluster achieves physical decoupling between business and processing. This solves the technical problems of traditional CPU solutions such as "insufficient computing power, excessive load, and poor adaptability," while ensuring the stability of core business and data security, and adapting to the large-scale, high-concurrency performance evaluation needs of financial institutions.

[0059] In some embodiments, the above-mentioned S104 may specifically include: Execute the structured query language script to generate the target table; Insert the target table into the physical table and view mapping relationship sub-table of the preset global parameter table to establish the association between the target table and the target view; If the database server sends the first status feedback information within a preset time period, the target view is determined to be generated. If no first status feedback information is received within the preset time period, the target table will be inserted again into the physical table and view mapping relationship sub-table; If the first status feedback information is not received within the preset time period for several consecutive times, it is determined that the target table insertion has failed, and an alarm message is generated.

[0060] In some embodiments of this application, the target table is inserted into a physical table-view mapping relationship sub-table of a preset global parameter table. For example, the target table may be inserted into the physical table-view mapping relationship sub-table by relying on the incTable.sh script.

[0061] In some embodiments of this application, when inserting physical table-view mapping records using the incTable.sh script, record insertion may fail due to network interruption, database anomalies, or script execution failure, or duplicate insertion may occur due to the lack of a retry mechanism. This can lead to "missed batch runs" (no records detected) or "duplicate batch runs" (multiple duplicate records detected) in the downstream toolchain. To address this, it can be determined whether to generate the target view by checking whether a first status feedback message is received from the database server within a preset time period.

[0062] The aforementioned first state feedback information, for example, may be feedback information used to represent a state identifier that has been "received".

[0063] The aforementioned preset time period, for example, can be 10 minutes. Of course, the preset time period in this embodiment is not limited to this, and can also be set according to the user's actual needs, which is not specifically limited here.

[0064] In some embodiments of this application, if the database server sends a first status feedback message within a preset time period, it is determined that the target view will be generated. This is because after the downstream indicator tool detects the mapping record, it writes a "received" status flag to the parameter table, indicating that the target table was successfully inserted, and it can also be determined that the target view will be generated.

[0065] In some embodiments of this application, if no first status feedback information is received within a preset time period, the target table is re-inserted into the physical table and view mapping relationship sub-table. For example, if no first status feedback information is received within 10 minutes, a second insertion is automatically triggered.

[0066] In some embodiments of this application, if the first status feedback information is not received within a preset time period multiple times in a row, it is determined that the target table insertion has failed and an alarm message is generated. For example, if the "received" status flag is not received within 10 minutes three times in a row, it is determined that the target table insertion has failed and an alarm message is generated to ensure the closed loop of signal transmission.

[0067] In this embodiment, after the SQL script generates the target table, the target table is inserted into the sub-table of the physical table and view mapping relationship, and a table-view association is established. The insertion result is verified through the status feedback information of the database server: if feedback is received within a preset time period, the view generation is considered successful; if no feedback is received, the insertion is automatically retried; if multiple failures occur consecutively, the insertion is considered a failure and an alarm message is generated. This design achieves real-time and accurate verification of the insertion result through status feedback, avoiding the propagation of hidden errors; the automatic retry mechanism can self-heal occasional faults such as network fluctuations, improving the system's fault tolerance; multiple failure alarms can accurately warn of irreversible faults, shortening the troubleshooting time. The whole system forms a closed-loop fault tolerance mechanism of "verification-retry-alarm", ensuring the accuracy of the target table-view mapping relationship and the continuity of the data processing flow, solving the technical problems of "no awareness of insertion failure, untimely fault handling, and poor business continuity" in traditional solutions, and adapting to the high availability and high reliability requirements of financial systems.

[0068] To facilitate understanding of the data processing method in the embodiments of this application, the actual application process of this data processing method is described as follows: To improve development efficiency, a hierarchical calculation logic rule was established, with the data logic relationships sequentially including customer attribution, performance recognition, indicator calculation, and performance evaluation, minimizing computational complexity. Simultaneously, common, simple rule calculation patterns were refined to form parameterizable and configurable calculation modules, supporting configurable development for most scenarios and improving development efficiency. In the processing from basic detailed data to basic performance indicators, parameterized configuration of the dataset processing was applied, and the indicator tools were informed of the data through customized service interfaces, allowing the indicator tools and performance evaluation tools to enter the batch processing sequence.

[0069] 1. Hierarchical computation model (1) Four-layer logical rules (as shown in the figure above): Customer attribution → Performance recognition → Indicator calculation → Profitability evaluation.

[0070] (2) Modular calculation: Each layer of rules is broken down into configurable units, supporting parameter coverage (e.g., customer attribution rules can be configured as "attributed by organization" or "attributed by customer manager").

[0071] 2. Global Parameter Table Design (1) ta99_khjy_tablvl_mpng_parm_w Parameter table level definition for creating benefits: mapping group sequence number, mapping group table sequence number, target table database, target table English name, upstream table database, upstream table English name, upstream table alias, upstream table association type, upstream table association filtering rules.

[0072] CREATE TABLE ta99_khjy_tablvl_mpng_parm_w ( map_group_id INT, -- Mapping group number table_seq INT, -- Mapping table sequence number within the group target_db VARCHAR(50), -- Target table database target_table VARCHAR(50), -- Target table name source_db VARCHAR(50), -- Upstream table database source_table VARCHAR(50), -- English name of upstream table join_type VARCHAR(20), -- Upstream table join type (INNER / LEFT JOIN) filter_rule TEXT -- Related filtering rules ); (2) ta99_khj_fldlvl_mpng_parm_w Functionality-level field definition: mapping group sequence number, field sequence number within the mapping group, target table database, target table English name, upstream mapping rule, target field default value.

[0073] CREATE TABLE ta99_khj_fldlvl_mpng_parm_w ( map_group_id INT, -- Mapping group number field_seq INT, -- field sequence number within the mapping group target_table VARCHAR(50), -- Target table name mapping_rule TEXT, -- Upstream mapping rule (e.g., SUM(source_field)) default_value VARCHAR(100) -- Default value of the target field ); (3) ta99_physm_with_wm_rel_w Physical table and view mapping relationship table: English name of view, English name of target table.

[0074] CREATE TABLE ta99_physm_with_wm_rel_w ( view_name VARCHAR(50), -- View name (English name) target_table VARCHAR(50) -- Target table name ); 3. Dynamic script generation and execution (1) Read the configuration from the global parameter table of the configuration library and generate the execution command. Parse each target table item by its English name: A. Parse the access control ID. If the access control ID does not exist, exit. If the access control ID is successfully accessed, then the access control ID is successfully accessed.

[0075] B. Read the table-level mapping group sequence number, sorting the sequence numbers from smallest to largest. If the sequence number is 0, report that no table-level or field-level mapping is configured. If the pointer overflows, report an error reading the mapping group sequence number array. Otherwise, read normally and obtain the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table's alias, the upstream table's association type, and the upstream table's association filtering rules to generate the table-level mapping statement.

[0076] C. Read the field-level mapping group sequence number, sorting the sequence numbers from smallest to largest. If the sequence number is 0, report that no field-level mapping is configured. If the pointer overflows, report an error reading the mapping group sequence number array. Otherwise, read normally and obtain the mapping group sequence number, the field sequence number within the mapping group, the target table database, the target table English name, the upstream mapping rule, and the target field default value to generate the field-level mapping statement.

[0077] Configure the parsing engine: def generate_script(target_table): # 1. Read the access control ID and verify permissions. if not validate_access_control(target_table): Raise an exception ("Access control ID does not exist") # 2. Resolve table-level mappings (sorted by map_group_id) table_mappings = query_table_mappings(target_table) if not table_mappings: raise Exception("Table-level mapping not configured") # 3. Generate DELETE statements (supports rerun) script = f"DELETE FROM {target_table};\n" # 4. Concatenate INSERT statements (including JOIN logic) script += build_insert_statement(table_mappings) # 5. Add field-level mappings (sorted by field_seq) field_rules = query_field_mappings(target_table) script = apply_field_rules(script, field_rules) return script The following is a schematic diagram of the logical flow of the above configuration parsing engine generating the execution script: Function: generate_script(target_table_name) Step 1: Read the access control ID corresponding to the target table and perform permission verification. If the access control ID does not exist or the permission is invalid, an "Access control ID does not exist" exception will be thrown, and script generation will be terminated. If the permission verification passes, proceed to the next step.

[0078] Step 2: Resolve table-level mapping configuration Using the target table's English name as an index, read the mapping group configuration from the table-level parameter table (ta99_khjy_tablvl_mpng_parm_w) and sort it by mapping group number in ascending order; If the table-level mapping configuration is not read, an "table-level mapping not configured" exception is thrown, and script generation is terminated. If a valid configuration is read, information such as the mapping group number and the upstream and downstream table association rules are recorded.

[0079] Step 3: Generate historical data cleanup statements Concatenate the "DELETE FROM [target table name];" statement to clear historical data when the script is rerun, thus avoiding data inconsistency.

[0080] Step 4: Concatenate INSERT statements containing multi-table join logic Based on the table-level mapping configuration read in step 2, generate the basic INSERT statement containing the JOIN logic of upstream and downstream tables.

[0081] Step 5: Integrate field-level calculation rules Using the target table's English name as an index, the field calculation rules are read from the field-level parameter table (ta99_khj_fldlvl_mpng_parm_w) and sorted by field number in ascending order; Incorporate field calculation rules (such as SUM operation and CASE WHEN logic) into the INSERT statement and supplement the default value of the target field.

[0082] Step 6: Return to the completed execution script after assembly.

[0083] (2) Command concatenation and execution: Before each target table insert statement, a DELETE statement is concatenated to support script rerun. After the dataset processing statements are executed, the physical table and view mapping relationship table is inserted by calling incTable.sh: the English name of the view and the English name of the target table, to achieve cross-server access, thereby informing the indicator tool that the dataset has been received and batch processing can be initiated.

[0084] 4. Cross-system collaboration: After the script is executed, it calls `incTable.sh` to insert the physical table-view mapping relationship, triggering the downstream toolchain to run batch processes.

[0085] The specific implementation methods described above are provided below: Example 1: Processing of Financial Performance Indicators 1. Configure table-level mapping: - `target_table`: "finance_performance" - `source_table`: ["customer_info", "transaction_records"] - `join_type`: "LEFT JOIN ON customer_id" 2. Configure field-level rules: - `Transaction Amount`: "SUM(transaction_amount) * exchange_rate" - `Client Attribution`: "CASE WHEN region='CN' THEN 'Asia Pacific' ELSE 'Overseas'" 3. Output script: This system generates a complete SQL script containing DELETE cleanup, multi-table JOIN, and field calculation logic.

[0086] Example 2: Cross-system data collaboration 1. Execute the script: After processing the `finance_performance` table, automatically insert records into `ta99physmwith_wmrelw`.

[0087] 2. Trigger downstream: After the indicator tool detects the new view `v_finance_performance`, it starts a batch process and generates an efficiency report.

[0088] This application's embodiments allow for dynamic adjustment of calculation rules through parameter tables, adapting to rapid business iterations and offering flexibility. Furthermore, it automatically generates rerunnable scripts with cleanup logic, reducing manual intervention and improving efficiency. Additionally, it enhances end-to-end execution efficiency through a cross-system notification mechanism based on physical table-view mapping, demonstrating synergy.

[0089] Based on the data processing method provided in the above embodiments, this application also provides specific implementations of a data processing apparatus. Please refer to the following embodiments.

[0090] like Figure 3 As shown, the data processing device 300 provided in this application embodiment may include the following modules: a first configuration module 301, a second configuration module 302, a parsing module 303, an execution module 304, and a generation module 305.

[0091] The first configuration module 301 is used to respond to the user's first configuration operation on the target table, and configure the upstream data source associated with the target table. The upstream data source includes the target customer table and the target performance table, and the target customer table and the target performance table include at least one indicator. The second configuration module 302 is used to respond to the user's second configuration operation on each indicator in the target table and configure the calculation rules for each indicator associated with the target table. Parsing module 303 is used to parse the target customer table, target performance table and calculation rules of each indicator associated with the target table, and generate a structured query language script; The execution module 304 is used to execute the structured query language script, generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. The generation module 305 is used to connect to the database server to obtain data from the target customer table and the target performance table based on the target view, and process the data according to the calculation rules of each indicator to generate the target report corresponding to the target table.

[0092] The data processing apparatus of this application embodiment is capable of configuring the upstream data source associated with the target table in response to a first configuration operation input by the user on the target table. The upstream data source includes a target customer table and a target performance table, and the target customer table and the target performance table include at least one indicator. In response to a second configuration operation by the user on each indicator in the target table, the apparatus configures the calculation rules for each indicator associated with the target table. The apparatus parses the target customer table, the target performance table, and the calculation rules for each indicator associated with the target table to generate a structured query language script. The apparatus executes the structured query language script to generate the target table and inserts it into a physical table and view mapping relationship sub-table of a preset global parameter table to obtain a target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. Based on the target view, the apparatus connects to and accesses the database server to obtain the data of the target customer table and the target performance table, and processes the data according to the calculation rules for each indicator to generate a target report corresponding to the target table. Thus, in this embodiment, the financial institution headquarters only needs to make adaptive configurations from the uniformly maintained customer table, performance table, and calculation rules of each indicator according to the operational needs of each branch. This allows for high coupling with its own needs. Furthermore, since the customer table, performance table, and calculation rules of each indicator are unified, they can be universally shared in the performance evaluation of each branch. Moreover, the calculation rules of the customer table, performance table, and each indicator can also be changed according to actual needs, exhibiting high scalability. Then, by parsing the configuration information, a structured query language script is generated. Even if there are business changes, there is no need to refactor the code; only the configuration needs to be modified, making the calculation logic more flexible. Finally, the target view serves as a unified access point. Downstream database servers do not need to adapt to the physical table access rules of different branches; they only need to obtain standardized data through the view. This not only ensures the security and consistency of data access but also reduces the complexity of cross-system access.

[0093] In some embodiments, the global parameter table further includes a table-level mapping sub-table. The first configuration module 301 is specifically used to configure the upstream data source of the target table and the association type between the target table and the upstream data source in the table-level mapping sub-table in response to a first configuration operation input by the user on the target table.

[0094] In some embodiments, the global parameter table further includes a field-level mapping sub-table. The second configuration module 302 is specifically used to configure the calculation rules of the target table associated with each indicator in the field-level mapping sub-table in response to the user's second configuration operation on each indicator in the target table.

[0095] In some embodiments, the parsing module 303 may specifically include: The first generation unit is used to read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the association filtering rule information from the table-level mapping sub-table using the target table as an index, and generate a table-level mapping statement. The second generation unit is used to read the mapping group number, the field number within the mapping group, the target table database, the English name of the target table, the upstream mapping rules, and the default values ​​of the target fields from the field-level mapping sub-table, using the target table as an index, and to generate field-level mapping statements. The concatenation unit is used to concatenate table-level mapping statements and field-level mapping statements to generate a structured query language script.

[0096] In some embodiments, the above-mentioned splicing unit is specifically used to splice table-level mapping statements, field-level mapping statements, and preset deletion statements to generate a structured query language script.

[0097] In some embodiments, the parsing module 303 may specifically include: The first reading unit is used to read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the association filter rule name from the table-level mapping sub-table, using the target table as an index. The first query unit is used to query the first rule information associated with the filtering rule name in the preset high-frequency access configuration information. The high-frequency access configuration information is stored in the field programmable gate array cluster. The high-frequency access configuration information includes multiple rule names and the rule information corresponding to each rule name. The third generation unit is used to generate table-level mapping statements based on the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the first rule information. The second reading unit is used to read the mapping group number, the field number within the mapping group, the target table database, the English name of the target table, the upstream mapping rule name, and the default value of the target field from the field-level mapping sub-table, using the target table as an index. The second query unit is used to query the second rule information of the upstream mapping rule name in the high-frequency access configuration information; The fourth generation unit is used to generate field-level mapping statements based on the mapping group sequence number, the field sequence number within the mapping group, the target table database, the English name of the target table, the second rule information, and the default value of the target field. The aforementioned concatenation unit is also used to concatenate table-level mapping statements and field-level mapping statements to generate a structured query language script.

[0098] In some embodiments, the execution module 304 may specifically include: Execute the structured query language script to generate the target table; Insert the target table into the physical table and view mapping relationship sub-table of the preset global parameter table to establish the association between the target table and the target view; If the database server sends the first status feedback information within a preset time period, the target view is determined to be generated. If no first status feedback information is received within the preset time period, the target table will be inserted again into the physical table and view mapping relationship sub-table; If the first status feedback information is not received within the preset time period for several consecutive times, it is determined that the target table insertion has failed, and an alarm message is generated.

[0099] Figure 4 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0100] An electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0101] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0102] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0103] In a particular embodiment, memory 402 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0104] The processor 401 implements any of the data processing methods described in the above embodiments by reading and executing computer program instructions stored in the memory 402.

[0105] In one example, the electronic device may also include a communication interface 403 and a bus 410. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0106] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0107] Bus 410 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0108] The electronic device can execute the data processing method described in the embodiments of this application, thereby achieving the combination Figure 1 and Figure 2 The data processing methods and apparatus described.

[0109] Furthermore, in conjunction with the data processing methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the data processing methods in the above embodiments.

[0110] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the data processing methods described in the above embodiments.

[0111] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0112] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0113] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0114] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0115] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data processing method, characterized in that, include: In response to a user's first configuration operation on a target table, the upstream data source associated with the target table is configured, the upstream data source including a target customer table and a target performance table, the target customer table and the target performance table including at least one indicator; In response to the user's second configuration operation on each of the indicators in the target table, configure the calculation rules for the target table associated with each of the indicators; Parse the target customer table, the target performance table, and the calculation rules of each indicator associated with the target table to generate a structured query language script; The structured query language script is executed to generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. Based on the target view, the database server is accessed to obtain data from the target customer table and the target performance table, and the data is processed according to the calculation rules of each indicator to generate the target report corresponding to the target table.

2. The method according to claim 1, characterized in that, The global parameter table also includes a table-level mapping sub-table. The first configuration operation in response to user input on the target table, configuring the upstream data source associated with the target table, includes: In response to the user's first configuration operation on the target table, the upstream data source of the target table and the association type between the target table and the upstream data source are configured in the table-level mapping sub-table.

3. The method according to claim 2, characterized in that, The global parameter table also includes a field-level mapping sub-table. The second configuration operation performed by the user on each of the indicators in the target table, configuring the calculation rules for associating each indicator with the target table, includes: In response to the user's second configuration operation on each of the indicators in the target table, the calculation rules for the target table associated with each of the indicators are configured in the field-level mapping sub-table.

4. The method according to claim 3, characterized in that, The step of parsing the target customer table, the target performance table, and the calculation rules for each indicator associated with the target table, and generating a structured query language script, includes: Using the target table as an index, the mapping group number, the table number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table's alias, the upstream table's association type, and the association filtering rule information are read from the table-level mapping sub-table to generate a table-level mapping statement; Using the target table as an index, the mapping group number, the field number within the mapping group, the target table database, the target table's English name, the upstream mapping rules, and the target field's default value are read from the field-level mapping sub-table to generate a field-level mapping statement; The table-level mapping statement and the field-level mapping statement are concatenated to generate a structured query language script.

5. The method according to claim 4, characterized in that, The step of concatenating the table-level mapping statement and the field-level mapping statement to generate a structured query language script includes: The table-level mapping statement, the field-level mapping statement, and the preset deletion statement are concatenated to generate a structured query language script.

6. The method according to claim 3, characterized in that, The step of parsing the target customer table, the target performance table, and the calculation rules for each indicator associated with the target table, and generating a structured query language script, includes: Using the target table as an index, read the mapping group sequence number, the table sequence number within the mapping group, the target table database, the target table English name, the upstream table database, the upstream table English name, the upstream table alias, the upstream table association type, and the association filter rule name from the table-level mapping sub-table; The first rule information of the associated filtering rule name is queried in the preset high-frequency access configuration information. The high-frequency access configuration information is stored in the field programmable gate array cluster. The high-frequency access configuration information includes multiple rule names and rule information corresponding to each rule name. Based on the mapping group number, the table number within the mapping group, the target table database, the target table's English name, the upstream table database, the upstream table's English name, the upstream table's alias, the upstream table's association type, and the first rule information, generate a table-level mapping statement; Using the target table as an index, read the mapping group number, the field number within the mapping group, the target table database, the target table's English name, the upstream mapping rule name, and the target field's default value from the field-level mapping sub-table; Query the second rule information of the upstream mapping rule name in the high-frequency access configuration information; Based on the mapping group number, the field number within the mapping group, the target table database, the target table English name, the second rule information, and the target field default value, generate a field-level mapping statement; The table-level mapping statement and the field-level mapping statement are concatenated to generate a structured query language script.

7. The method according to claim 1, characterized in that, The execution of the structured query language script generates the target table and inserts it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view, including: Execute the structured query language script to generate the target table; The target table is inserted into the physical table and view mapping relationship sub-table of the preset global parameter table to establish the association between the target table and the target view; If the database server sends a first status feedback message within a preset time period, the target view is determined to be generated. If the first status feedback information is not received within the preset time period, the target table will be inserted again into the physical table and view mapping relationship sub-table; If the first status feedback information is not received within the preset time period multiple times in a row, it is determined that the target table insertion has failed, and an alarm message is generated.

8. A data processing apparatus, characterized in that, The device includes: The first configuration module is used to configure the upstream data source associated with the target table in response to the user's first configuration operation on the target table. The upstream data source includes a target customer table and a target performance table, and the target customer table and the target performance table include at least one indicator. The second configuration module is used to configure the calculation rules for the target table associated with each of the indicators in response to the user's second configuration operation on each of the indicators in the target table; The parsing module is used to parse the target customer table, the target performance table, and the calculation rules of each indicator associated with the target table, and generate a structured query language script; The execution module is used to execute the structured query language script, generate the target table and insert it into the physical table and view mapping relationship sub-table of the preset global parameter table to obtain the target view. The physical table and view mapping relationship sub-table is used to connect to and access the database server. The generation module is used to connect to the database server to obtain data from the target customer table and the target performance table based on the target view, and process the data according to the calculation rules of each indicator to generate the target report corresponding to the target table.

9. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the data processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the data processing method as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the data processing method as described in any one of claims 1-7.

Images