Data processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method is applied to a mobility index summarization engine component, and comprises the following steps: acquiring configuration information, and acquiring original business data from a database according to the configuration information; performing single index calculation on each piece of original business data to obtain a single index result corresponding to each piece of original business data; based on a time step in the configuration information and a preset summarizing mode, constructing an initialization account book of a preset type, and based on an output type in the configuration information, formatting and summarizing each single index result to each initialization account book according to the time step to obtain a result account book; and storing the result account books in a database in batches. By adopting the method, the scene batch running efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of big data processing, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of storage technology, after the basic cash flow logic measurement implemented in the bank, the measured results can be output according to the actual cash flow model of the position and stored in the database for aggregation.

[0003] In traditional technology, when the server runs a scenario batch for a business, it first obtains the position data in the database, and then calculates each position data in the database according to the calculation method predefined by the business requirements to obtain detailed data, for example, the detailed data can be the principal flow and interest flow corresponding to each position data. After completing multiple scenario batches, the server obtains the detailed data corresponding to each position data under each scenario batch in the database, and then stores the detailed data corresponding to each position data in the database.

[0004] However, in the current traditional technology, due to the large amount of position data, the amount of detailed data is also larger, and the detailed data corresponding to a single position data is directly stored in the database. The independent storage of each detailed data occupies the database's IO (Input / Output) time, resulting in frequent database operations and bloated table data, which in turn leads to low efficiency of scenario batch running. Summary of the invention

[0005] Based on this, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0006] In a first aspect, the present application provides a data processing method, which is applied to a liquidity indicator aggregation engine component, comprising:

[0007] Obtaining configuration information, and obtaining original business data from a database according to the configuration information;

[0008] Calculating the single index of each of the original business data to obtain the single index result corresponding to each of the original business data;

[0009] Constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book;

[0010] The result account books are stored in batches in a database.

[0011] In one embodiment, the configuration information further includes multiple observation points and business scenarios, and the single indicator result includes local indicators corresponding to each observation point under the business scenario;

[0012] The single indicator calculation is performed on each of the original business data to obtain a single indicator result corresponding to each of the original business data, including:

[0013] Determining a target product according to the type of the original business data;

[0014] Determine, among a plurality of preset factory modes, a target factory mode corresponding to the target product according to the metrological properties of the target product;

[0015] Based on the target factory model, the observation points and the business scenarios, a single indicator calculation is performed on the original business data to obtain a local indicator corresponding to each observation point under the business scenario.

[0016] In one embodiment, the formatting and aggregating the results of each single indicator into each of the initialization account books according to the time step based on the output type in the configuration information to obtain the result account book includes:

[0017] Determine the target account book corresponding to each of the single indicator results according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book;

[0018] Based on the output type, each of the single indicator results is formatted and aggregated into each of the target account books according to the time step to obtain a result account book.

[0019] In one embodiment, determining the target account book corresponding to each single indicator result according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book includes:

[0020] According to the time step corresponding to each of the single indicator results and the corresponding relationship between the single indicator result and the initialization account book, matching is performed in the result account book corresponding to the initialization account book to obtain a matching result;

[0021] If the matching result is successful, the initialized account book corresponding to the single indicator result is used as the target account book;

[0022] If the matching result is a matching failure, a new target account book is constructed according to the single indicator result and the time step.

[0023] In one of the embodiments, the output type includes a first output type, a second output type, and a third output type;

[0024] Based on the output type, the single indicator results are formatted and aggregated into the target account books according to the time step to obtain a result account book, including:

[0025] If the output type is the first output type, formatting the results of each single indicator according to the time step and storing them in the target account book to obtain a final summary result, and constructing an account result account book according to the final summary result;

[0026] If the output type is the second output type, formatting and aggregating the single indicator results representing the simulated business into the target account book according to the time step to obtain a business simulation summary result, and constructing a result account book according to the business simulation summary result;

[0027] If the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step and the intervals are summarized to obtain the intermediate results of each time step interval, and the result account book is constructed according to the intermediate results.

[0028] In one embodiment, storing the result account books in batches to a database includes:

[0029] According to the preset batch information, the target storage data for each batch of processing is determined in the result account book;

[0030] The target storage data of each batch is inserted into the database in a data insertion form of batch insertion.

[0031] In a second aspect, the present application further provides a data processing device, which is applied to a liquidity indicator aggregation engine component, including:

[0032] An acquisition module, used to acquire configuration information, and acquire original business data from a database according to the configuration information;

[0033] A calculation module, used to calculate a single indicator for each of the original business data to obtain a single indicator result corresponding to each of the original business data;

[0034] A summary module, configured to construct an initialization account book of a preset type based on the time step and the preset summary method in the configuration information, and to format and summarize the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information, so as to obtain a result account book;

[0035] The storage module is used to store the result account books in batches into a database.

[0036] In one embodiment, the configuration information further includes multiple observation points and business scenarios, and the single indicator result includes local indicators corresponding to each observation point under the business scenario;

[0037] The calculation module is specifically used to determine the target product according to the type of the original business data;

[0038] Determine, among a plurality of preset factory modes, a target factory mode corresponding to the target product according to the metrological properties of the target product;

[0039] Based on the target factory model, the observation points and the business scenarios, a single indicator calculation is performed on the original business data to obtain a local indicator corresponding to each observation point under the business scenario.

[0040] In one embodiment, the summary module is specifically used to determine the target account book corresponding to each single indicator result according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book;

[0041] Based on the output type, each of the single indicator results is formatted and aggregated into each of the target account books according to the time step to obtain a result account book.

[0042] In one embodiment, the summary module is specifically used to match the result account book corresponding to the initialization account book according to the time step corresponding to each of the single indicator results and the corresponding relationship between the single indicator result and the initialization account book to obtain a matching result;

[0043] If the matching result is successful, the initialized account book corresponding to the single indicator result is used as the target account book;

[0044] If the matching result is a matching failure, a new target account book is constructed according to the single indicator result and the time step.

[0045] In one embodiment, the output type includes a first output type, a second output type and a third output type; the summary module is specifically used to, if the output type is the first output type, format the summary of each single indicator result according to the time step and put it into the target account book to obtain a final summary result, and construct an account result account book according to the final summary result;

[0046] If the output type is the second output type, formatting and aggregating the single indicator results representing the simulated business into the target account book according to the time step to obtain a business simulation summary result, and constructing a result account book according to the business simulation summary result;

[0047] If the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step and the intervals are summarized to obtain the intermediate results of each time step interval, and the result account book is constructed according to the intermediate results.

[0048] In one of the embodiments, the storage module is specifically used to determine the target storage data for each batch of processing in the result account book according to the preset batch information;

[0049] The target storage data of each batch is inserted into the database in a data insertion form of batch insertion.

[0050] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0051] Obtaining configuration information, and obtaining original business data from a database according to the configuration information;

[0052] Calculating the single index of each of the original business data to obtain the single index result corresponding to each of the original business data;

[0053] Constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book;

[0054] The result account books are stored in batches in a database.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0056] Obtaining configuration information, and obtaining original business data from a database according to the configuration information;

[0057] Calculating the single index of each of the original business data to obtain the single index result corresponding to each of the original business data;

[0058] Constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book;

[0059] The result account books are stored in batches in a database.

[0060] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0061] Obtaining configuration information, and obtaining original business data from a database according to the configuration information;

[0062] Calculating the single index of each of the original business data to obtain the single index result corresponding to each of the original business data;

[0063] Constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book;

[0064] The result account books are stored in batches in a database.

[0065] The above-mentioned data processing method, device, computer equipment, computer-readable storage medium and computer program product reduce the dependence on database performance by setting the process of a single calculation in the liquidity indicator summary engine. The liquidity indicator summary engine independent of the database can make full use of the advantages of distributed processing, improve the efficiency of summarizing single indicator results, and thus improve the efficiency of summarizing original business data; by constructing an initialization account book and formatting and summarizing the single indicator results of the original business data according to the time step and the preset summary method, the single indicator results are summarized in each initialization account book, and the data formatting of the single indicator results is realized, and the data regularity of the single indicator results summarized in the result account book is ensured, so as to improve the efficiency of subsequent data analysis and summary based on the single indicator results summarized in the result account book, and then the result account book after data formatting can realize batch storage of the result account books in the database, thereby improving the database storage efficiency, and improving the data processing performance of the liquidity indicator summary engine, thereby improving the batch running efficiency of the scenario batch running. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 is a flow chart of a data processing method in one embodiment;

[0068] Figure 2 A schematic diagram of a single indicator calculation process in one embodiment;

[0069] Figure 3 A flowchart of formatting and summarizing steps in one embodiment;

[0070] Figure 4 A schematic diagram of a process for aggregating a single indicator result into a target account book in one embodiment;

[0071] Figure 5 It is a schematic diagram of a process of formatting and summarizing based on different output types in one embodiment;

[0072] Figure 6 A schematic diagram of a process for batch storing result account books in one embodiment;

[0073] Figure 7 is a flowchart of an example of a data processing method in an embodiment;

[0074] Figure 8 is a structural block diagram of a data processing device in one embodiment;

[0075] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0077] In one embodiment, Figure 1 As shown, a data processing method is provided. This embodiment takes the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method is applied to a liquidity indicator aggregation engine component, and the method includes the following steps:

[0078] Step 102: Obtain configuration information, and obtain original business data from a database according to the configuration information.

[0079] In an embodiment of the present application, the liquidity indicator aggregation engine component may be set in a head office-level analysis system. The server responds to the user's simulated scenario batch operation or the scenario batch task set according to a preset period. The server creates a new scenario batch task, which has corresponding configuration information, including overall plan information and time steps.

[0080] When creating a new scenario batch task, the server caches the overall solution information of the scenario batch task, including data such as account books, observation points, and interest rate scenarios. Among them, the account book is a tool for managing and recording financial assets and liabilities. The account book is a format set by a bank or financial institution according to business needs, and is a storage form for storing existing business data of the bank or financial institution. For example, the account book can contain data such as account number, account name, account attributes, cash flow information, and interest rate information. The time step can be a time interval set by the user according to business simulation requirements, or a time interval predefined according to periodic batch tasks. In an example, after the server completes the calculation of the cash flow of a single position detail, the interval is summarized according to the time step to form the formatted data of the overall solution for statistics, storage, and processing. Since the time step is used in the measurement process of each original business data, the time step is cached to reduce database access operations.

[0081] When creating a new task, after obtaining the configuration information, the server caches the configuration parameters in memory to avoid multiple queries to the database during each calculation and reduce latency. At the same time, based on the configuration information and the needs of the scenario-based batch task, the server queries the database for the original business data corresponding to the needs of the current scenario-based batch task. The original business data can be position data, which is the business funds in a bank or institution, or other business data, such as contract data, historical transaction records, etc. The server caches the original business data obtained from the query in memory to speed up the subsequent calculation process.

[0082] Step 104, calculate the single index for each original business data to obtain the single index result corresponding to each original business data.

[0083] In an embodiment of the present application, the server divides the original business data into multiple data blocks for processing according to the configured data block size, and instantiates the measurement object corresponding to the measurement attribute of the product through the factory mode according to the measurement attributes of the product, such as contract cash flow indicators, repricing cash flow indicators, etc., and then calculates the corresponding indicators for each original business data according to the calculation rules pre-set in the factory mode to obtain a single indicator result. For example, the single indicator result includes the cash flow and repricing period of the eyebrow pencil contract, and caches each single indicator result in the memory to facilitate direct call of subsequent summary processing. In one example, for a loan business, the server uses the liquidity indicator aggregation engine component to calculate the principal flow and interest flow for each period corresponding to the loan business according to the contract cash flow indicators of the loan business in the distributed resources of the server.

[0084] Step 106, constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating each single indicator result into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book.

[0085] In the embodiment of the present application, the server establishes different initialization account books according to different preset aggregation methods and different time steps. For example, the preset aggregation method is divided into a cash flow account book set and an interest rate risk account book set, and then the account book is initialized in combination with the number of time steps to obtain the initialization account books of different preset types and time intervals containing different time steps. After completing the measurement of the single indicator result and the construction of the initialization account book, the server transmits the single indicator result to the result set aggregation node of the liquidity indicator aggregation engine component, and performs formatting and aggregation processing through the result set aggregation node to form a stream processing.

[0086] The result set aggregation node formats the single indicator result into the corresponding initialization account book according to the output type (for example, default output, planned position, detailed cash flow), and aggregates the data into the initialization account book according to the time step. For example, the preset type of initialization account book can be an initialization account book that divides the original business data according to different classification dimensions, including initialization account books divided according to dimensions such as region, subject, product or corresponding characteristics of products. The initialization account book also includes summary indicators of time intervals under different time steps. For example, for loan business, the time step can be a monthly interval to obtain the principal flow and interest flow of each month, or a quarterly interval to obtain the principal flow and interest flow corresponding to each quarter. After completing the data aggregation, the final result account book is formed for subsequent storage and analysis.

[0087] Step 108, storing the result account books in batches into a database.

[0088] In the embodiment of the present application, since the data has been formatted according to the account book and time step, the data structure before storage has been prepared. Therefore, the server can use the liquidity indicator summary engine component to store the formatted result account book data into the database in the form of batch insertion.

[0089] In the above data processing method, by setting the process of a single calculation in the liquidity indicator aggregation engine, the dependence on database performance is reduced. The liquidity indicator aggregation engine independent of the database can make full use of the advantages of distributed processing and improve the efficiency of aggregating single indicator results; by constructing an initialization account book and formatting and summarizing the single indicator results of the original business data according to the time step and the preset aggregation method, the single indicator results are aggregated into each initialization account book, thereby realizing the data formatting of the single indicator results, ensuring the data regularity of the single indicator results summarized in the result account book, and then the result account book after data formatting can be stored in the database in batches, thereby improving the database storage efficiency, and then improving the batch running efficiency of the scenario batch running.

[0090] In an exemplary embodiment, the configuration information also includes multiple observation points and business scenarios, and a single indicator result includes local indicators corresponding to each observation point under the business scenario; Figure 2 As shown, step 104 includes steps 202 to 206. Among them:

[0091] Step 202: determine the target product according to the type of the original business data.

[0092] In the embodiment of the present application, first, the server identifies the product type corresponding to each original business data, and the server determines the target product corresponding to each original business data according to the classification standard corresponding to the attribute of the original business data and the contract type, product type, and business function, such as a loan contract, a deposit contract, a bond, etc. In one example, the original business data includes different types of loan and deposit contracts, and the server can determine that the target product corresponding to the current original business data is a "fixed rate loan" or a "demand deposit" according to the data type of the original business data.

[0093] Step 204 , determining a target factory mode corresponding to the target product according to the metrological properties of the target product among a plurality of preset factory modes.

[0094] In the embodiment of the present application, the server matches the preset factory mode according to the measurement attributes of the product, such as cash flow characteristics, interest rate changes, repricing cycle, etc., and selects the target factory mode for calculating the single indicator result of the target product. In one example, for a "fixed-rate loan" product, the server matches the preset factory mode for the product and selects a target factory mode that specifically processes fixed-rate cash flows, such as a "fixed-rate cash flow factory", while for a "demand deposit" product, the server selects another target factory mode that processes floating rates.

[0095] Step 206 , based on the target factory model, observation points and business scenarios, a single indicator calculation is performed on the original business data to obtain a local indicator corresponding to each observation point under the business scenario.

[0096] In an embodiment of the present application, the server instantiates the target factory pattern in the liquidity indicator summary engine component, and uses the target factory pattern to calculate the original business data. Among them, the observation point is a preset time point used to evaluate the performance of the original business data at these specific time points. For example, the observation point can be the end of the month, the end of the quarter, the end of the year, etc. During the calculation process, the server divides and processes the original business data according to the observation point, and obtains the state or characteristics of each observation point corresponding to the business data at this time point. The business scenario represents the current market or economic environment, such as interest rate levels, economic growth rates, etc. The business scenario is applied to the original business data to simulate the actual business situation, or the original business data is adjusted according to the business scenario to calculate the data performance under different scenarios. For example, in the scenario of economic recession, the probability of default of the loan is adjusted.

[0097] Finally, based on the target factory model, observation point and business scenario, the server calculates local indicators for each piece of original business data at the observation point. The local indicators include cash flow, interest flow, etc. in different time intervals.

[0098] In this embodiment, the target factory pattern is applied to the original business data, and local indicators are calculated at set observation points. Then, the business scenario is used to adjust and simulate the actual situation to ensure that the single indicator result obtained can accurately reflect the accuracy of the summary under the specific business scenario.

[0099] In an exemplary embodiment, Figure 3 As shown, step 106 includes steps 302 to 304. Among them:

[0100] Step 302, according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book, determine the target account book corresponding to each single indicator result.

[0101] In the embodiment of the present application, the time step corresponding to a single indicator result is a time interval set by the user according to the business simulation requirements or a pre-set time interval, that is, the aggregation conditions for the single indicator result to be aggregated determine how the indicator results are subsequently aggregated into the account book. The server determines how the single indicator result is mapped to the target account book based on the structure and format of the single indicator result and the initialized account book, that is, each single indicator result corresponds to a specific field or data segment in the initialized account book. The server determines which account books are used to store the current single indicator result based on the definition and structure of the time step and the initialized account book. In addition, there may be multiple target account books, each of which reflects a different business perspective (such as cash flow, risk assessment, etc.).

[0102] Step 304, based on the output type, format and aggregate the results of each single indicator into each target account book according to the time step to obtain a result account book.

[0103] In an embodiment of the present application, the server first determines the output type, and formats and summarizes the single indicator results according to the time step according to the preset output type or the specific output type selected by the user, wherein the different output types are used to determine the content of displaying and summarizing the single indicator results. For example, the default output is to display the summary results of the original business data, and the intermediate results of the measurement process are not displayed, while the detailed cash flow output may need to display the detailed information and calculation process of each cash flow one by one to verify the correctness of the calculation logic. Based on the determined time step, the server summarizes the results of each single indicator in a preset format to the corresponding target account books to obtain the result account book. In addition, for the indicator of each observation point, the data at the observation point is summarized in the corresponding target account book.

[0104] In this embodiment, by aggregating the results of a single indicator into the target account book according to the correspondence between the time step and the initialization account book, the target account book can be clearly defined, and the data can be formatted and aggregated according to different output types. By mapping the results of a single indicator to the target account book, structural data based on the result account book is formed, which reduces the amount of data and simplifies the report processing flow, realizes multi-level optimization, realizes data formatting and reduces the amount of data that needs to be stored, reduces the number of database interactions, and thus improves the efficiency of scenario batch running.

[0105] In an exemplary embodiment, Figure 4 As shown, step 302 includes steps 402 to 406. Among them:

[0106] Step 402, according to the time step corresponding to each single indicator result and the corresponding relationship between the single indicator result and the initialization account book, matching is performed in the result account book corresponding to the initialization account book to obtain a matching result.

[0107] In the embodiment of the present application, after the aggregation node receives the results of each single indicator, the server determines whether a result account registration corresponding to the single indicator result already exists based on the batch running scenario, time step and target account book information of the single original business data.

[0108] Step 404: If the matching result is successful, the initialization account book corresponding to the single indicator result is used as the target account book.

[0109] In the embodiment of the present application, if the matching result is a successful match, it means that there is currently a result account book corresponding to the single indicator result, and the server aggregates the single indicator result or the new cash flow into the existing result account book.

[0110] Step 406: If the matching result is a matching failure, a new target account book is constructed according to the single indicator result and the time step.

[0111] In an embodiment of the present application, if the matching result is a matching failure, it means that there is currently no result account book corresponding to the single indicator result. The server will create a new result account book based on the batch running scenario, time step and account book information, and summarize the first single cash flow as the basic cash flow. The summarized single indicator result does not need to be saved again to reduce system resource overhead.

[0112] In this embodiment, the data aggregation and formatting work is completed by the liquidity indicator aggregation engine component to form structured data based on the target account book, which reduces the amount of data and simplifies the report processing process, thereby achieving multi-level optimization.

[0113] In an exemplary embodiment, the output type includes a first output type, a second output type and a third output type, each output type representing an application scenario under different requirements; Figure 5 As shown, step 304 includes steps 502 to 506. Among them:

[0114] Step 502, if the output type is the first output type, format the results of each single indicator according to the time step and put them into the target account book to obtain the final summary result, and construct the result account book according to the final summary result.

[0115] In an embodiment of the present application, the first output type is the default output type. Under the default output type, the server aggregates the results of each single indicator according to the time step based on the target account book corresponding to the single indicator result to obtain the final aggregated result, wherein the intermediate results of the aggregation process and the intermediate calculation process are only cached in the liquidity indicator aggregation engine component and released after the calculation is completed to avoid too much intermediate data occupying storage space, so that the result account book only contains the final aggregated result.

[0116] Step 504, if the output type is the second output type, format and summarize the single indicator results representing the simulated business into the target account book according to the time step, obtain the business simulation summary result, and construct the result account book according to the business simulation summary result.

[0117] In the embodiment of the present application, the user can select the second output type when verifying the business volume model. When the server confirms that the current output type is the second output type, the second output type is the overall forecast involving business simulation, that is, the output plan position. At this time, the time step is usually the future time interval, which is used to calculate and summarize the products to which the original business data belongs in the future time interval. The server processes the single indicator results representing the simulated business, formats and summarizes them according to the time step to ensure that they meet the structural requirements of the target account book, and then obtains the business simulation summary results.

[0118] Step 506, if the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step interval summary, to obtain the intermediate results of each time step interval, and to construct the result account book according to the intermediate results.

[0119] In an embodiment of the present application, the third output type is the output type in a single verification scenario. The user can select the third output type to output detailed cash flow, and can view the step outputs in the calculation process for detailed verification. When the server confirms that the current output type is the third output type, the intervals are formatted and summarized according to the time step based on the single indicator results. During the interval summary process, the intermediate results of each time step interval are obtained. These intermediate results include data generated at each stage during the calculation process, reflecting the detailed calculation method at each time step. In the scenario of auditing, detailed analysis or verification of calculation accuracy, the user can select the third output type to output detailed cash flow, and then the user can view each step in the calculation process in detail for step-by-step verification.

[0120] In this embodiment, the results of a single indicator are formatted and summarized through different output types, and the settlement registration under different dimensions can be output, providing a multi-dimensional result account book to meet data aggregation for different needs, which can adapt to different analysis scenarios and provide flexibility in data analysis.

[0121] In an exemplary embodiment, Figure 6 As shown, step 108 includes steps 602 to 604. Among them:

[0122] Step 602, according to the preset batch information, determine the target storage data for each batch processing in the result account book.

[0123] In an embodiment of the present application, the batch information includes pre-set parameters such as batch size, processing order, and time schedule. The server allocates the summary data in the result account book according to the batch information to obtain the target storage data to ensure that the target storage data of each batch is correct and meets the storage requirements.

[0124] Step 604: insert each batch of target storage data into the database in a batch insert data insertion format.

[0125] In the embodiment of the present application, batch insertion is an efficient data insertion method that can reduce the overhead of the insertion operation and improve the performance of the database operation. Since the target storage data has been formatted according to the result account book and the time step, the target storage data can be inserted in batches. After the server completes the storage allocation of the result account book, it executes the batch insertion operation according to the batch size in the preset batch information, and writes the target storage data in batches into the database.

[0126] In this embodiment, allocating and processing data in batches helps to avoid performance issues caused by processing a large amount of data at one time and improve data processing efficiency. Batch insertion can significantly improve the efficiency of data writing. Compared with inserting one by one, batch insertion reduces the transaction processing overhead of the database, optimizes performance, and thus improves the efficiency of scenario batch running.

[0127] In an exemplary embodiment, Figure 7 As shown, an example of a data processing method is provided, including:

[0128] Step 701, load the output type into the cache;

[0129] Step 702, loading the time step into the cache;

[0130] Step 703, calculating a single index for each original business data to obtain a single index result corresponding to each original business data;

[0131] Step 704, based on the output type in the configuration information, horizontally summarize the results of each single indicator according to the time step to obtain an initial summary result;

[0132] Step 705, according to the time step corresponding to each single indicator result and the corresponding relationship between the single indicator result and the initialization account book, match the result account book corresponding to the initialization account book to obtain the matching result. If the matching result is a successful match, execute step 706; if the matching result is a failed match, execute step 707;

[0133] Step 706, continue to iterate the single indicator result;

[0134] Step 707, for each single indicator result (single cash flow result), all single indicator results are accumulated according to the time step interval corresponding to the single indicator result and summarized into the initialization account book to obtain the result account book;

[0135] Step 708, construct a new target account book (result set by daily time period) based on the single indicator result and time step;

[0136] Step 709, output the result account book.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0139] In an exemplary embodiment, Figure 8 As shown, a data processing device 800 is provided, comprising: an acquisition module 801, a calculation module 802, a summary module 803 and a storage module 804, wherein:

[0140] The acquisition module 801 is used to acquire configuration information and obtain original business data from the database according to the configuration information;

[0141] The calculation module 802 is used to calculate the single index of each original business data to obtain the single index result corresponding to each original business data;

[0142] The aggregation module 803 is used to construct an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and to format and aggregate the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book;

[0143] The storage module 804 is used to store the result account books in batches into the database.

[0144] In one of the embodiments, the configuration information further includes a plurality of observation points and business scenarios, and a single indicator result includes a local indicator corresponding to each observation point under the business scenario;

[0145] The calculation module 802 is specifically used to determine the target product according to the type of the original business data;

[0146] Determine a target factory mode corresponding to the target product according to the metrological properties of the target product among multiple preset factory modes;

[0147] Based on the target factory model, observation points and business scenarios, single indicators are calculated for the original business data to obtain local indicators corresponding to each observation point under the business scenario.

[0148] In one embodiment, the aggregation module 803 is specifically used to determine the target account book corresponding to each single indicator result according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book;

[0149] Based on the output type, the results of each single indicator are formatted and summarized into each target account book according to the time step to obtain the result account book.

[0150] In one embodiment, the summary module 803 is specifically used to match the result account book corresponding to the initialization account book according to the time step corresponding to each single indicator result and the corresponding relationship between the single indicator result and the initialization account book to obtain the matching result;

[0151] If the matching result is successful, the initial account book corresponding to the single indicator result is used as the target account book;

[0152] If the matching result is a matching failure, a new target account book is constructed based on the single indicator result and time step.

[0153] In one embodiment, the output type includes a first output type, a second output type and a third output type; the summary module 803 is specifically used to, if the output type is the first output type, format the results of each single indicator according to the time step and put them into the target account book to obtain the final summary result, and construct the account result account book according to the final summary result;

[0154] If the output type is the second output type, the single indicator results representing the simulated business are formatted and summarized into the target account book according to the time step to obtain the business simulation summary result, and the result account book is constructed according to the business simulation summary result;

[0155] If the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step interval summary to obtain the intermediate results of each time step interval, and the result account book is constructed based on the intermediate results.

[0156] In one embodiment, the storage module 804 is specifically used to determine the target storage data for each batch of processing in the result account book according to the preset batch information;

[0157] Insert the target storage data of each batch into the database according to the data insertion form of batch insertion.

[0158] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0159] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store result accounts. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0160] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0161] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0162] Obtain configuration information, and obtain original business data from the database based on the configuration information;

[0163] Calculate the single index of each original business data to obtain the single index result corresponding to each original business data;

[0164] Based on the time step and preset aggregation method in the configuration information, a preset type of initialization account book is constructed. Based on the output type in the configuration information, the results of each single indicator are formatted and aggregated into each initialization account book according to the time step to obtain the result account book.

[0165] Store the resulting account books in batches into the database.

[0166] In one embodiment, the configuration information further includes multiple observation points and business scenarios, and a single indicator result includes local indicators corresponding to each observation point under the business scenario;

[0167] When the processor executes the computer program, it also performs the following steps:

[0168] Determine the target product based on the type of original business data;

[0169] Determine a target factory mode corresponding to the target product according to the metrological properties of the target product among multiple preset factory modes;

[0170] Based on the target factory model, observation points and business scenarios, single indicators are calculated for the original business data to obtain local indicators corresponding to each observation point under the business scenario.

[0171] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0172] According to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialization account book, determine the target account book corresponding to each single indicator result;

[0173] Based on the output type, the results of each single indicator are formatted and summarized into each target account book according to the time step to obtain the result account book.

[0174] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0175] According to the time step corresponding to each single indicator result and the corresponding relationship between the single indicator result and the initialization account book, matching is performed in the result account book corresponding to the initialization account book to obtain the matching result;

[0176] If the matching result is successful, the initial account book corresponding to the single indicator result is used as the target account book;

[0177] If the matching result is a matching failure, a new target account book is constructed based on the single indicator result and time step.

[0178] In one embodiment, the output type includes a first output type, a second output type, and a third output type; when the processor executes the computer program, the processor further implements the following steps:

[0179] If the output type is the first output type, the results of each single indicator are summarized and formatted according to the time step and put into the target account book to obtain the final summary result, and the account result account book is constructed according to the final summary result;

[0180] If the output type is the second output type, the single indicator results representing the simulated business are formatted and summarized into the target account book according to the time step to obtain the business simulation summary result, and the result account book is constructed according to the business simulation summary result;

[0181] If the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step interval summary to obtain the intermediate results of each time step interval, and the result account book is constructed based on the intermediate results.

[0182] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0183] According to the preset batch information, determine the target storage data for each batch of processing in the result account book;

[0184] Insert the target storage data of each batch into the database according to the data insertion form of batch insertion.

[0185] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0186] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0187] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0188] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0189] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0190] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method is applied to a liquidity indicator aggregation engine component, and the method comprises: Obtaining configuration information, and obtaining original business data from a database according to the configuration information; Calculating the single index of each of the original business data to obtain the single index result corresponding to each of the original business data; Constructing an initialization account book of a preset type based on the time step and the preset aggregation method in the configuration information, and formatting and aggregating the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information to obtain a result account book; The result account books are stored in batches in a database.

2. The method according to claim 1, characterized in that The configuration information also includes multiple observation points and business scenarios, and the single indicator result includes the local indicators corresponding to each observation point under the business scenario; The single indicator calculation is performed on each of the original business data to obtain a single indicator result corresponding to each of the original business data, including: Determining a target product according to the type of the original business data; Determine, among a plurality of preset factory modes, a target factory mode corresponding to the target product according to the metrological properties of the target product; Based on the target factory model, the observation points and the business scenarios, a single indicator calculation is performed on the original business data to obtain a local indicator corresponding to each observation point under the business scenario.

3. The method according to claim 1, characterized in that The step of formatting and aggregating the results of each single indicator into each of the initialization account books according to the time step based on the output type in the configuration information to obtain a result account book includes: Determine the target account book corresponding to each of the single indicator results according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book; Based on the output type, each of the single indicator results is formatted and aggregated into each of the target account books according to the time step to obtain a result account book.

4. The method according to claim 3, characterized in that The determining, according to the time step corresponding to the single indicator result and the corresponding relationship between the single indicator result and the initialized account book, the target account book corresponding to each single indicator result includes: According to the time step corresponding to each of the single indicator results and the corresponding relationship between the single indicator result and the initialization account book, matching is performed in the result account book corresponding to the initialization account book to obtain a matching result; If the matching result is successful, the initialization account book corresponding to the single indicator result is used as the target account book; If the matching result is a matching failure, a new target account book is constructed according to the single indicator result and the time step.

5. The method according to claim 3, characterized in that: The output types include a first output type, a second output type and a third output type; Based on the output type, the single indicator results are formatted and aggregated into the target account books according to the time step to obtain a result account book, including: If the output type is the first output type, formatting the results of each single indicator according to the time step and storing them in the target account book to obtain a final summary result, and constructing an account result account book according to the final summary result; If the output type is the second output type, formatting and aggregating the single indicator results representing the simulated business into the target account book according to the time step to obtain a business simulation summary result, and constructing a result account book according to the business simulation summary result; If the output type is the third output type, the target account book corresponding to the single indicator result is formatted according to the time step and the intervals are summarized to obtain the intermediate results of each time step interval, and the result account book is constructed according to the intermediate results.

6. The method according to claim 1, characterized in that The step of storing the result account books in batches into a database comprises: According to the preset batch information, the target storage data for each batch of processing is determined in the result account book; The target storage data of each batch is inserted into the database in a data insertion form of batch insertion.

7. A data processing device, characterized in that: The device is applied to a liquidity indicator aggregation engine component, and the device includes: An acquisition module, used to acquire configuration information, and acquire original business data from a database according to the configuration information; A calculation module, used to calculate a single indicator for each of the original business data to obtain a single indicator result corresponding to each of the original business data; A summary module, configured to construct an initialization account book of a preset type based on the time step and the preset summary method in the configuration information, and to format and summarize the results of each single indicator into each initialization account book according to the time step based on the output type in the configuration information, so as to obtain a result account book; The storage module is used to store the result account books in batches into a database.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images