Table processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a table processing method and device, computer equipment, a readable storage medium and a program product. The method comprises the steps that a summary sheet is obtained, the summary sheet comprises summary data groups, and the summary data groups are obtained by grouping summary data units in the summary sheet according to a first target dimension; screening out a target summarized data group meeting a target filtering condition from the summarized data groups; aggregating the target summarized data units in the target summarized data groups to obtain target summarized data; processing the target summarized data based on a target processing rule to obtain an incremental data unit; and inserting the incremental data unit into the summary sheet to obtain an updated summary sheet. By adopting the method, the table processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a table processing method, apparatus, computer equipment, readable storage medium, and program product. Background Art

[0002] With the development of computer technology, computers can store large amounts of data, allowing them to process this data using their resources. When the amount of data is large, a summary table can be set up to quickly understand the data distribution. When the data changes, the summary table also needs to be updated accordingly.

[0003] However, in traditional technologies, in the case of continuous data changes, it is necessary to change the master table according to the changed data, and then continue to perform data changes based on the changed master table, which results in low data update efficiency. Summary of the Invention

[0004] Based on this, it is necessary to provide a table processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve table processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a form processing method, comprising:

[0006] Obtaining a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to the first target dimension;

[0007] Filter out target summary data groups that meet target filtering conditions from the summary data groups;

[0008] Aggregating the target summary data units in the target summary data group to obtain target summary data;

[0009] Process the target summary data based on the target processing rules to obtain incremental data units;

[0010] Insert the incremental data unit into the summary table to obtain an updated summary table.

[0011] In one embodiment, aggregating target summary data units in the target summary data group to obtain target summary data includes:

[0012] In the target summary data group, obtaining a dimension value corresponding to a second target dimension in each target summary data unit, where the second target dimension is different from the first target dimension;

[0013] Aggregating the dimension values ​​corresponding to the second target dimension to obtain an aggregate value corresponding to the second target dimension;

[0014] Target summary data is obtained based on the dimension value corresponding to the first target dimension and the aggregation value corresponding to the second target dimension in the target summary data unit.

[0015] In one embodiment, target summary data is processed based on target processing rules to obtain incremental data units, including:

[0016] Based on the target processing rule, the target processing dimension in the target summary data is apportioned to obtain a first apportioned data unit;

[0017] Based on the dimension value corresponding to the target processing dimension in the first allocated data unit, a second allocated data unit corresponding to the target summary data is obtained;

[0018] Based on the first apportioned data unit and the second apportioned data unit, a plurality of incremental data units are obtained.

[0019] In one embodiment, target summary data is processed based on target processing rules to obtain incremental data units, including:

[0020] determining a third target dimension from the first target dimension and the second target dimension based on the target processing rule;

[0021] In the target summary data group, obtaining the dimension value corresponding to the third target dimension in the target summary data;

[0022] Determining, based on the dimension value corresponding to the third target dimension, a dimension value corresponding to the newly added dimension in the target summary data;

[0023] An incremental data unit is obtained based on the dimension value corresponding to the first target dimension, the aggregate value corresponding to the second target dimension, and the dimension value corresponding to the newly added dimension.

[0024] In one embodiment, after processing the target summary data based on the target processing rule to obtain the incremental data unit, the method further includes:

[0025] Insert incremental data cells into the parts list to update the parts list.

[0026] In one embodiment, the method further comprises:

[0027] Get form processing tasks;

[0028] Obtaining a processing rule set based on a table processing task, the processing rule set including multiple association groups, the association groups associating target filtering conditions with matching target processing rules;

[0029] Get the summary table;

[0030] Obtain the first target association group according to the target execution order;

[0031] determining a first target filtering condition and a first target processing rule based on the first target association group;

[0032] Filtering out a target summary data group that meets a target filtering condition from multiple summary data groups;

[0033] Aggregating the target summary data units in the target summary data group to obtain target summary data;

[0034] Process the target summary data based on the target processing rules to obtain incremental data units;

[0035] Insert the incremental data unit into the summary table to obtain an updated summary table;

[0036] A second target association group is obtained according to the target execution order, and the step of obtaining the summary table is continued until the summary table is processed based on each association group in the processing rule set.

[0037] In a second aspect, the present application further provides a form processing device, comprising:

[0038] An acquisition module is used to acquire a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to the first target dimension;

[0039] A filtering module is used to filter out target summary data groups that meet target filtering conditions from the summary data groups;

[0040] an aggregation module, configured to aggregate target summary data units in the target summary data group to obtain target summary data;

[0041] A processing module, configured to process the target summary data based on target processing rules to obtain incremental data units;

[0042] The insert module is used to insert the incremental data unit into the summary table to obtain an updated summary table.

[0043] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0044] Obtaining a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to the first target dimension;

[0045] Filter out target summary data groups that meet target filtering conditions from the summary data groups;

[0046] Aggregating the target summary data units in the target summary data group to obtain target summary data;

[0047] Process the target summary data based on the target processing rules to obtain incremental data units;

[0048] Insert the incremental data unit into the summary table to obtain an updated summary table.

[0049] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0050] Obtaining a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to the first target dimension;

[0051] Filter out target summary data groups that meet target filtering conditions from the summary data groups;

[0052] Aggregating the target summary data units in the target summary data group to obtain target summary data;

[0053] Process the target summary data based on the target processing rules to obtain incremental data units;

[0054] Insert the incremental data unit into the summary table to obtain an updated summary table.

[0055] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0056] Obtaining a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to the first target dimension;

[0057] Filter out target summary data groups that meet target filtering conditions from the summary data groups;

[0058] Aggregating the target summary data units in the target summary data group to obtain target summary data;

[0059] Process the target summary data based on the target processing rules to obtain incremental data units;

[0060] Insert the incremental data unit into the summary table to obtain an updated summary table.

[0061] The above table processing method, device, computer equipment, computer-readable storage medium and computer program product, by obtaining a summary table, since the summary table corresponds to at least one summary data grouping, and the summary data grouping is obtained by grouping the summary data units in the summary table according to the first target dimension, after obtaining the summary table, at least one summary data grouping corresponding to the summary table can be obtained, and then data aggregation is performed based on the summary data grouping, and then processing is performed based on the aggregated data, and the processed data is directly inserted into the summary table, and the operation of obtaining at least one summary data grouping corresponding to the summary table is repeated to complete the processing of the summary table. In this way, the present application does not need to wait for the data in the summary table to be aggregated before performing data processing. After the processed data is directly inserted into the summary table, the subsequent steps can be continued. Under the premise of realizing data processing, the data processing process can be simplified, thereby improving the efficiency of data processing. In addition, when the target summary data is processed based on the target processing rule and the incremental data unit is obtained, the data aggregation corresponding to the summary table can also be asynchronously performed, thereby reducing the time spent on data aggregation in the summary table, thereby improving the efficiency of table processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 A diagram illustrating an application environment of a table processing method in one embodiment;

[0064] Figure 2 1 is a flow chart of a table processing method in one embodiment;

[0065] Figure 3 A schematic diagram of a pre-processing summary table in one embodiment;

[0066] Figure 4 A schematic diagram of a processed summary table in one embodiment;

[0067] Figure 5 Schematic diagram of a process of data aggregation step in one embodiment;

[0068] Figure 6 is a flowchart of a table processing method in another embodiment;

[0069] Figure 7 is a structural block diagram of a table processing device in one embodiment;

[0070] Figure 8 is a diagram of the internal structure of a computer device in one embodiment;

[0071] Figure 9 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0073] The table processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104, or located in the cloud or on another network server. A user can create a new table processing task on the terminal. The terminal then sends the table processing task to the server. The server can obtain a corresponding summary table based on the table processing task. The summary table corresponds to at least one summary data group, which is obtained by grouping the summary data units in the summary table according to a first target dimension. Next, the server selects a target summary data group that meets the target filtering conditions from the multiple summary data groups. The server then aggregates the target summary data units in the target summary data group to obtain target summary data. The target summary data is processed based on the target processing rules to obtain incremental data units. The incremental data units are inserted into the summary table to obtain an updated summary table. The server then sends the updated summary table to the terminal. In this way, when processing data, the server can directly obtain the corresponding data from the summary table, without having to wait for the data in the summary table to be aggregated before processing the data, thereby improving the efficiency of table processing.

[0074] Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0075] It can be understood that the table processing method provided in the embodiment of the present application can also be applied to one of the terminals or servers.

[0076] In an exemplary embodiment, Figure 2 As shown, a table processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate the method, which includes the following steps S202 to S210.

[0077] Step S202: Obtain a summary table.

[0078] Among them, the summary table is a table that collects, organizes and summarizes multiple related data or information, and is used to quickly understand the overall situation and key information of the data. Figure 3 As shown in the figure, a summary table can record information of multiple dimensions. Compared with a detail table, a data in a summary table may be a summary of one or more data in a detail table. For example Figure 4 The row corresponding to "R&D Center" in the summary table is the sum of the two rows corresponding to "R&D Center" in the detail table. The "Current Period Amount" corresponding to "R&D Center" in the summary table (1,000 yuan) is the sum of the two "Current Period Amounts" corresponding to "R&D Center" in the detail table (500 yuan and 500 yuan).

[0079] The summary table includes at least one summary data group. The summary data group is obtained by grouping the summary data units in the summary table according to the first target dimension. A summary data unit refers to a row of data in a table. A dimension corresponds to a column of data in a table. For a database table in a database, a dimension corresponds to a field, and a summary data unit corresponds to a record in the database table.

[0080] The dimension value corresponding to the first target dimension in each summary data unit in the summary data group is consistent. The number of first target dimensions is not limited to one. The first target dimension may be a partial dimension of all dimensions corresponding to the summary table. The first target dimension may be pre-set. Optionally, the server may also determine the first target dimension based on the data type of the dimension value corresponding to each dimension in the summary table. For example, when the data type of the dimension value corresponding to the dimension is a text type, the dimension belongs to the first target dimension. When the data type of the dimension value corresponding to the dimension is a numeric type, the dimension does not belong to the first target dimension.

[0081] It should be noted that the summary table in this embodiment may include multiple summary data units with the same dimension value corresponding to the first target dimension. Figure 4In the summary table, the first target dimension is "organization, period, subject". There are two data units corresponding to "Development Department 1" in the summary table, and the dimension values ​​corresponding to the three first target dimensions of "organization, period, subject" in these two units are consistent, all corresponding to "Development Department 1, January 2025, R&D expenses".

[0082] Exemplarily, after obtaining the summary table, the summary data units in the summary table are grouped according to the first target dimension to obtain summary data groups. For example, the summary data units in the summary table can be grouped by executing a GROUP BY statement on the summary table.

[0083] Optionally, before obtaining the summary table, the summary data units in the summary table may also be grouped according to the first target dimension, so that the summary data units in the summary table obtained in this way are already grouped.

[0084] Step S204: Filter out target summary data groups that meet target filtering conditions from the summary data groups.

[0085] Among them, the target filtering condition can be predetermined. For example, the target filtering condition can be set by the user when determining the table processing task. The target filtering condition can include a filtering dimension and a dimension value corresponding to the filtering dimension. For example, the summary data grouping includes three summary data units, namely "R&D Center, January 2025, R&D Expenses, 1000", "R&D Center, January 2025, R&D Expenses, -500", and "Development Department 1, January 2025, R&D Expenses, 500". Among them, the filtering dimension is organization, and the corresponding dimension value is "R&D Center". The target summary data grouping that meets the target filtering condition is determined to include two summary data units, namely "R&D Center, January 2025, R&D Expenses, 1000" and "R&D Center, January 2025, R&D Expenses, -500".

[0086] Step S206 , aggregating the target summary data units in the target summary data group to obtain target summary data.

[0087] In this embodiment, the target summary data group may include multiple target summary data units, and when performing data processing subsequently, one summary data unit should be processed to ensure the correctness of the data processing.

[0088] For example, in the scenario of allocating the R&D expenses of the R&D center, after obtaining the target summary data grouping ("R&D center, January 2025, R&D expenses, 1000" and "R&D center, January 2025, R&D expenses, -500"), the R&D expenses of the R&D center should be the balance. If the R&D expenses of the R&D center are to be allocated, the remaining R&D expenses (500 yuan) should be allocated. Therefore, the target summary data units in the target summary data grouping need to be aggregated to obtain the target summary data ("R&D center, January 2025, R&D expenses, 500") to ensure the correctness of subsequent processing of the summary data.

[0089] Step S208: Process the target summary data based on the target processing rule to obtain an incremental data unit.

[0090] Target processing rules refer to the way data is processed. For example, target processing rules can be used to allocate, derive, accrue, adjust, and amortize data. Incremental data units are data units that are different from the target summary data after the target summary data is processed.

[0091] For example, the target summary data is "R&D Center, January 2025, R&D Expenses, 500," and the target processing rule is to allocate this target summary data evenly to R&D Departments 1 and 2. After processing the target summary data based on the target processing rule, the resulting incremental data units might include "R&D Department 1, January 2025, R&D Expenses, 250" and "R&D Department 2, January 2025, R&D Expenses, 250."

[0092] Step S210: insert the incremental data unit into the summary table to obtain an updated summary table.

[0093] For example, after obtaining the incremental data unit, the incremental data unit is inserted into the last row of the summary table as a new summary data unit to obtain an updated summary table. In this way, the data stored in the updated summary table can represent the data changes in the original summary table after the data processing.

[0094] It is understandable that after inserting the incremental data unit into the summary table, the server can continue to directly execute the acquisition of the summary table and other subsequent steps without waiting for the data in the summary table to be aggregated and then extracting the target data from the summary table for processing.

[0095] In the above table processing method, by obtaining the summary table, since the summary table corresponds to at least one summary data grouping, and the summary data grouping is obtained by grouping the summary data units in the summary table according to the first target dimension, after obtaining the summary table, at least one summary data grouping corresponding to the summary table can be obtained, and then data aggregation is performed based on the summary data grouping, and then processing is performed based on the aggregated data. The processed data is then directly inserted into the summary table, and the operation of obtaining at least one summary data grouping corresponding to the summary table is repeated to complete the processing of the summary table. In this way, the present application does not need to wait for the data in the summary table to be aggregated before executing data processing. After the processed data is directly inserted into the summary table, the subsequent steps can be continued. Under the premise of realizing data processing, the data processing process can be simplified, thereby improving the efficiency of data processing. In addition, when the present application processes the target summary data based on the target processing rules and obtains the incremental data unit, it can also asynchronously perform the data aggregation corresponding to the summary table, thereby reducing the time consumed by data aggregation in the summary table, thereby improving the efficiency of table processing.

[0096] After the incremental data unit is inserted into the summary table, the summary table can perform data aggregation based on the incremental data unit to update the summary data unit associated with the incremental data unit. However, in some embodiments, for example, when there are many incremental data units, the corresponding data aggregation process may take more time. When the next data processing is performed based on the processing rules, the data units in the summary table may not have completed aggregation. In this case, the filtered target summary data group includes multiple target summary data units. Before the next data processing, in order to ensure the correctness of the data processing, some processing needs to be performed on these target summary data units. The specific processing process can be referred to the following content.

[0097] like Figure 5 As shown, step S206 includes steps S502 to S506.

[0098] Step S502: In the target summary data group, obtain the dimension value corresponding to the second target dimension in each target summary data unit.

[0099] Wherein, the second target dimension is different from the first target dimension.

[0100] It can be understood that the target summary data grouping is obtained by grouping all summary data units in the summary table according to the first target dimension. Therefore, the dimension values ​​corresponding to the first target dimension in the target summary data grouping are consistent. The dimension values ​​corresponding to the second target dimension in each target summary data unit are obtained at different times. Therefore, there may be multiple summary data units in the summary table with the same dimension values ​​corresponding to the first target dimension.

[0101] The data type of the dimension value corresponding to the second target dimension can be either text or numeric, and the number of second target dimensions is not limited to one.

[0102] For example, all dimensions of the summary table can be divided into the first target dimension and the second target dimension. Figure 3 In the summary table, there are four dimensions, of which three dimensions, "Organization", "Period" and "Subject", belong to the first target dimension, and the remaining dimension, "Amount in this Period", belongs to the second target dimension.

[0103] Step S504: Aggregate the dimension values ​​corresponding to the second target dimension to obtain an aggregate value corresponding to the second target dimension.

[0104] Aggregation is the operation of combining multiple dimension values ​​to form a summary result. Aggregation can be of many types and is not limited to aggregation that performs numerical operations on dimension values.

[0105] Illustratively, the step of aggregating the dimension values ​​corresponding to the second target dimension to obtain the aggregate value corresponding to the second target dimension may include performing a numerical operation on each dimension value corresponding to the second target dimension to obtain the aggregate value. For example, the dimension values ​​corresponding to the current period's amount (1,000 yuan, -500 yuan) are added together to obtain an aggregate value of 500.

[0106] It can be understood that digital operations may include sum operations, average operations, maximum operations, minimum operations, etc.

[0107] Step S506 : Obtain target summary data based on the dimension value corresponding to the first target dimension and the aggregation value corresponding to the second target dimension in the target summary data unit.

[0108] Through step S502 and step S504 , the dimension values ​​corresponding to all dimensions corresponding to the target summary data group can be determined, so that the target summary data can be obtained based on the dimension values ​​corresponding to all dimensions.

[0109] In this embodiment, by obtaining the dimension value corresponding to the second target dimension in each target summary data unit of the target summary data group, and then based on the dimension value corresponding to each second target dimension, the aggregation value corresponding to the second target dimension is obtained, so that the summary data corresponding to the target summary data group can be obtained, that is, the target summary data. This ensures that the processing object when the target processing rule is subsequently executed is the correct summary data, rather than partial data, and can ensure the correctness of the table processing. In addition, the data aggregation process is set before data processing and after the summary table is obtained, that is, the summary table can be obtained immediately after data processing, and then data aggregation is performed, and then data processing is performed. All data processing that needs to be performed on the table directly has been performed, so that the data processing steps can be executed in a coherent manner, thereby improving the efficiency of data processing.

[0110] It can be understood that when there is only one target summary data unit in the target summary data group, the step of aggregating the target summary data unit can be omitted, and the target summary data unit can be directly aggregated based on the target processing rule, which can further improve the efficiency of table data.

[0111] In actual applications, the task of processing the summary table may include averaging each data in the summary table, or continuously averaging a certain data in the table. In this case, the summary data unit in the summary table will be divided into multiple incremental data units. The dimension values ​​of certain dimensions in these incremental data units are aggregated into the dimension values ​​corresponding to these dimensions in the target summary data. How to determine these incremental data units can be performed by performing the following steps: Based on the target processing rules, the target processing dimension in the target summary data is apportioned to obtain a first apportioned data unit. Based on the dimension value corresponding to the target processing dimension in the first apportioned data unit, a second apportioned data unit corresponding to the target summary data is obtained. Based on the first apportioned data unit and the second apportioned data unit, multiple incremental data units are obtained.

[0112] Among them, when the target processing rule corresponds to the allocation rule, the target processing dimension can be determined based on the target processing rule. The target processing dimension includes the processing dimension belonging to the first target dimension or the processing dimension belonging to the second target dimension. The first allocation data unit corresponds to the allocation object, and the second allocation data unit corresponds to the allocated object. Optionally, when the dimension value corresponding to the target processing dimension in the first allocation data unit is positive, the dimension value corresponding to the target processing dimension in the second allocation data unit is negative. The dimension value corresponding to the target processing dimension in the second allocation data unit represents the change information of the dimension value corresponding to the target processing dimension in the target summary unit. For example, based on the fact that the sum of the dimension values ​​corresponding to the target processing dimension in the first allocation data unit is 500 yuan, it can be known that the dimension value corresponding to the target processing dimension of the target summary data is 500 yuan less, that is, the dimension value corresponding to the target processing dimension in the second allocation data unit is -500 yuan.

[0113] See also Figure 3 or Figure 4 "Organization," "Period," and "Subject" belong to the first target dimension, while "Current Period Amount" belongs to the second target dimension. The target processing dimensions include "Organization" and "Current Period Amount." For example, based on the allocation rules, 500 yuan of the R&D center's R&D expenses must be divided equally between Development Departments 1 and 2. Therefore, after allocating "Organization" and "Current Period Amount" in the target summary data, the resulting allocated data units are: "Development Department 1, January 2025, R&D Expenses, 250 yuan" and "Development Department 2, January 2025, R&D Expenses, 250 yuan." Correspondingly, the second allocated data unit corresponding to the target summary data is "R&D Center, January 2025, R&D Expenses, -500 yuan." In other words, in this example, there are three incremental data units corresponding to the target summary data.

[0114] In this embodiment, the summary data in the summary table is processed based on the incremental data unit corresponding to the processing rule of the allocation type to obtain newly generated allocation data units and allocation data units corresponding to the changes of the original summary data units, so that the detailed situation of the summary table after being processed by the processing rule can be obtained.

[0115] In other embodiments, the target processing rules can also be used to deduce data in the summary table. For example, for a target field whose field value in the summary table is empty, the field value of the field is determined based on certain field values ​​in the original summary table, then the field value corresponding to the target field can be derived based on these field values. For details, please refer to the following steps: based on the target processing rules, determine the third target dimension from the first target dimension and the second target dimension; obtain the dimension value corresponding to the third target dimension in the target summary data; based on the dimension value corresponding to the third target dimension, determine the dimension value corresponding to the newly added dimension in the target summary data; based on the dimension value corresponding to the first target dimension, the aggregation value corresponding to the second target dimension, and the dimension value corresponding to the newly added dimension, obtain the incremental data unit.

[0116] In this embodiment, based on the target processing rule, the summary data units with null values ​​in some dimension values ​​in the summary table can be supplemented. The reason for these null values ​​in the dimension values ​​may be that they are newly added dimensions or these fields have not been processed.

[0117] It can be understood that the dimension value corresponding to the newly added dimension should be determined by the dimension value corresponding to the existing dimension in the original summary table. For example, a new dimension is created in the original summary table to represent which parent department the summary data unit belongs to. Figure 3 or Figure 4 If a new dimension "Superior Department" is created in the table, the dimension value corresponding to the "Superior Department" can be supplemented based on the dimension "Organization".

[0118] The number of third target dimensions is not limited to one. The step of determining the dimension value corresponding to the newly added dimension in the target summary data based on the dimension value corresponding to the third target dimension may include: determining the dimension value corresponding to the newly added dimension based on the sum of the dimension values ​​corresponding to multiple third target dimensions. The step may also include: querying a target dimension value from a target table that matches the dimension value corresponding to the third target dimension, and determining the dimension value corresponding to the newly added dimension based on the target dimension value.

[0119] In this embodiment, the changes of the obtained incremental data unit relative to the target summary data include the newly added dimension and the dimension value corresponding to the newly added dimension.

[0120] In this embodiment, a third target dimension is determined from the existing dimensions in the target summary data based on the target processing rules. The third target dimension is used to assist in determining the dimension value of the newly added dimension. In this way, the data in the summary table can be supplemented or new data columns can be added based on the target processing rules, thereby speeding up the efficiency of supplementing the table content and improving the efficiency of verifying the table content.

[0121] In some embodiments, the incremental data units processed by the target processing rules can be stored in a detailed table. The detailed table can record the changes in the summary table after each processing by the target processing rules, which is convenient for tracing back the table processing.

[0122] In actual applications, in order to complete a business rule, it is often necessary to execute the target processing rules on the summary table multiple times to obtain an updated summary table that accurately matches the business rule. Therefore, in some embodiments, the server can perform the following steps: obtain a table processing task; obtain a processing rule set based on the table processing task, the processing rule set including multiple association groups, the association groups associating target filtering conditions with matching target processing rules; obtain a summary table; obtain a first target association group according to the target execution order; determine a first target filtering condition and a first target processing rule based on the first target association group; filter out a target summary data group that meets the target filtering condition from multiple summary data groups; aggregate the target summary data units in the target summary data group to obtain target summary data; process the target summary data based on the target processing rule to obtain incremental data units; insert the incremental data units into the summary table to obtain an updated summary table; obtain a second target association group according to the target execution order, and continue to perform the step of obtaining the summary table until the summary table is processed based on each association group in the processing rule set.

[0123] Table processing tasks can be user-created. When created, they correspond to at least one processing rule, which together form a processing rule set. Each element (i.e., an association group) in a processing rule set corresponds to a target filtering condition and a matching target processing rule. Each association group in a processing rule set has a specific execution order, which can be determined by the user when creating the table processing task.

[0124] For example, a user creates a table processing task that requires allocating Dimension A to certain summary data cells in a summary table, and then allocating Dimension B to certain summary data cells in the summary table. Therefore, the processing rule set includes two association groups. The first association group records the summary data cells to be allocated for the first time, along with the specific allocation rules for Dimension A. The second association group records the summary data cells to be allocated for the second time, along with the specific allocation rules for Dimension B.

[0125] Exemplarily, after obtaining the table processing task, the server parses the table processing task to obtain multiple target filtering conditions and respective matching target processing rules corresponding to the table processing task. The target processing rules are then executed sequentially according to the target execution order of the target processing rules corresponding to the table processing task. Specifically, the first first target association group to be executed is first obtained, and based on the first target association group, the first target filtering condition and the first target processing rule corresponding to this table processing are obtained. Through the first target filtering condition, it can be obtained which summary data units in the summary table are to be processed this time. Through the first target processing rule, the target summary data corresponding to these summary data units can be processed to obtain incremental data units that match the first target association group. After that, the obtained incremental data unit is inserted into the summary table, and the second second target association group to be executed is obtained, and the subsequent steps are executed according to the above steps, which will not be repeated here. In this way, a large number of different tasks can be processed on the table.

[0126] It is understandable that the first target filtering condition and the second target filtering condition may be different, and the first target processing rule and the second target processing rule may also be different.

[0127] In this embodiment, by obtaining a table processing task and performing a series of processing on the table based on the table processing task, the efficiency of complex processing of the table can be improved, and the efficiency of obtaining an updated summary table can be accelerated. In addition, in this embodiment, after executing the first processing rule, the process of executing the second processing rule can be quickly entered, that is, after inserting the newly added incremental data unit into the summary data unit, the second processing rule can be directly executed without considering the data sorting process in the summary table after inserting the new data unit. The data aggregation process outside the table reduces the number of times the database is read and written compared to the data sorting process inside the table, which can ensure the speed of table processing.

[0128] In some embodiments, a summary table can be stored based on a target database (such as a ClickHouse database), and the target database can periodically aggregate the data in the summary table. However, when the summary table is stored in the target database, all summary data units whose dimension values ​​corresponding to the second target dimension are zero may be automatically deleted. Therefore, in some embodiments, a redundant field can be created for the summary table, and the redundant field is not zero, and the aggregate value corresponding to the redundant field in each summary data group is also not zero. This can reduce the probability of data deletion, thereby improving the completeness of the data in the table, and further improving the accuracy and reliability of table processing.

[0129] In an exemplary embodiment, a relational database and a ClickHouse database are used in combination, and the ClickHouse database is used as the underlying data storage of the summary table. The summary table and the detail table are migrated to the ClickHouse database, and other rule configuration and other information are still stored in the relational database.

[0130] When storing summary and detail tables in a ClickHouse database, use the MergeTree table engine for the detail table and set all dimensions (fields) in the detail table as primary keys. For summary tables, use the SummingMergeTree table engine, set the first target dimension (first target field) as the primary key, and set the second target dimension (second target field) as an aggregated numeric column.

[0131] First, obtain the summary table. For example, Figure 6 As shown, a database statement of group by the first target dimension field is executed on the summary table, so that multiple summary data groups corresponding to the summary table can be obtained.

[0132] Next, the target summary data grouping that meets the target filtering condition is filtered out from the summary data grouping. And in the target summary data grouping, the dimension value corresponding to the second target dimension in each target summary data unit is obtained, and the second target dimension is different from the first target dimension. The dimension value corresponding to the second target dimension is aggregated to obtain the aggregate value corresponding to the second target dimension. Based on the dimension value corresponding to the first target dimension in the target summary data unit and the aggregate value corresponding to the second target dimension, the target summary data is obtained. That is to say, all the second target dimension fields in the target summary data grouping are aggregated, for example, the dimension values ​​corresponding to each second target dimension are summed up respectively. It can be understood that the aggregation and grouping process can be executed by a database statement, such as executing the database statement of group by all first target dimensions and sum all second target dimensions to aggregate the incompletely aggregated summary data units in the summary table to obtain the summary data corresponding to the summary table.

[0133] Next, based on the target processing rules, the target processing dimension in the target summary data is apportioned to obtain a first apportioned data unit. Based on the dimension value corresponding to the target processing dimension in the first apportioned data unit, a second apportioned data unit corresponding to the target summary data is obtained. Based on the first and second apportioned data units, multiple incremental data units are obtained. In other words, the data derivation rules are applied to the data in the summary table to obtain multiple incremental detailed data units.

[0134] Finally, after obtaining multiple incremental data units, these incremental data units are inserted into the summary table. It is understood that if it is necessary to record the incremental detailed data corresponding to each data derivation to facilitate tracking of each data derivation, the incremental detailed data can be synchronously inserted into the detailed table.

[0135] In addition, for the summary data in the summary table, background asynchronous data aggregation can be performed synchronously during the next data derivation process, so that the summary data in the summary table is the result of aggregation.

[0136] Through the above embodiment, insertion is aggregation, and the next data derivation can be started after the generated new data is inserted into the summary table. There is no need to wait until all the data in the summary table completes the final data aggregation, which can improve the performance of data processing. And based on the ClickHouse database to store the summary table, the advantages of columnar storage can be fully utilized to significantly reduce the space for data storage. In addition, in the process of obtaining the summary table and deriving data from the summary table, a thread can be set to aggregate the data in the summary table, so that the data aggregation step can be omitted in subsequent data derivation, and the data aggregation process and the data derivation process can be executed synchronously, which can further improve the efficiency of table processing and greatly reduce the time spent on complex derivation of the table. It also supports the processing of extremely large amounts of data, and the data processing time increases in a gentle curve as the amount of data increases.

[0137] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0138] Based on the same inventive concept, embodiments of the present application also provide a table processing device for implementing the aforementioned table processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more table processing device embodiments provided below can be found in the above-mentioned limitations of the table processing method and will not be repeated here.

[0139] In an exemplary embodiment, Figure 7As shown, a table processing device 700 is provided, comprising: an acquisition module 701, a filtering module 702, an aggregation module 703, a processing module 704 and an insertion module 705, wherein:

[0140] The acquisition module 701 is used to acquire a summary table, where the summary table includes summary data groups, and the summary data groups are obtained by grouping summary data units in the summary table according to a first target dimension.

[0141] The filtering module 702 is configured to filter out target summary data groups that meet target filtering conditions from the summary data groups.

[0142] The aggregation module 703 is configured to aggregate the target summary data units in the target summary data group to obtain target summary data.

[0143] The processing module 704 is configured to process the target summary data based on the target processing rule to obtain an incremental data unit.

[0144] The inserting module 705 is configured to insert the incremental data unit into the summary table to obtain an updated summary table.

[0145] In some embodiments, the aggregation module 703 is further configured to obtain, in the target summary data group, a dimension value corresponding to a second target dimension in each target summary data unit, where the second target dimension is different from the first target dimension. The dimension values ​​corresponding to the second target dimension are aggregated to obtain an aggregate value corresponding to the second target dimension. The target summary data is obtained based on the dimension values ​​corresponding to the first target dimension in the target summary data unit and the aggregate value corresponding to the second target dimension.

[0146] In some embodiments, the processing module 704 is further configured to perform apportionment processing on the target processing dimension in the target summary data based on the target processing rule to obtain a first apportioned data unit. Based on the dimension value corresponding to the target processing dimension in the first apportioned data unit, a second apportioned data unit corresponding to the target summary data is obtained. Based on the first apportioned data unit and the second apportioned data unit, a plurality of incremental data units are obtained.

[0147] In some embodiments, the processing module 704 is further configured to determine a third target dimension from the first target dimension and the second target dimension based on a target processing rule. In the target summary data group, obtain a dimension value corresponding to the third target dimension in the target summary data. Based on the dimension value corresponding to the third target dimension, determine a dimension value corresponding to the newly added dimension in the target summary data. Based on the dimension value corresponding to the first target dimension, the aggregate value corresponding to the second target dimension, and the dimension value corresponding to the newly added dimension, obtain an incremental data unit.

[0148] In some embodiments, the table processing device 700 further includes an update module, and the update module is configured to insert the incremental data unit into the detail table to update the detail table.

[0149] In some embodiments, the table processing device 700 is also used to: obtain a table processing task. Based on the table processing task, obtain a processing rule set, the processing rule set includes multiple association groups, and the association groups associate target filtering conditions with matching target processing rules. Obtain a summary table. Obtain a first target association group according to the target execution order. Determine the first target filtering condition and the first target processing rule based on the first target association group. Filter out the target summary data group that meets the target filtering condition from multiple summary data groups. Aggregate the target summary data units in the target summary data group to obtain target summary data. Process the target summary data based on the target processing rule to obtain an incremental data unit. Insert the incremental data unit into the summary table to obtain an updated summary table. Obtain a second target association group according to the target execution order, and continue to perform the step of obtaining the summary table until the summary table is processed based on each association group in the processing rule set.

[0150] Each module in the table processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data such as summary tables, detailed tables, and processing rule sets. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a table processing method is implemented.

[0152] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 9 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a table processing method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0153] Those skilled in the art will understand that Figure 8 or Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0154] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0155] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0156] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0158] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0159] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0160] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A table processing method, characterized in that: The method comprises: Obtaining a summary table, the summary table including summary data groups, the summary data groups being obtained by grouping summary data units in the summary table according to a first target dimension; Filtering target summary data groups that meet target filtering conditions from the summary data groups; aggregating target summary data units in the target summary data group to obtain target summary data; Processing the target summary data based on the target processing rules to obtain incremental data units; Insert the incremental data unit into the summary table to obtain an updated summary table.

2. The method according to claim 1, characterized in that The aggregating the target summary data units in the target summary data group to obtain the target summary data includes: In the target summary data group, obtaining a dimension value corresponding to a second target dimension in each of the target summary data units, where the second target dimension is different from the first target dimension; Aggregating the dimension values ​​corresponding to the second target dimension to obtain an aggregate value corresponding to the second target dimension; The target summary data is obtained based on the dimension value corresponding to the first target dimension in the target summary data unit and the aggregation value corresponding to the second target dimension.

3. The method according to claim 2, characterized in that The target summary data is processed based on the target processing rule to obtain the incremental data unit, including: Based on the target processing rule, the target processing dimension in the target summary data is apportioned to obtain a first apportioned data unit; Based on the dimension value corresponding to the target processing dimension in the first allocated data unit, obtaining a second allocated data unit corresponding to the target summary data; Based on the first apportioned data unit and the second apportioned data unit, a plurality of the incremental data units are obtained.

4. The method according to claim 2, characterized in that The target summary data is processed based on the target processing rule to obtain the incremental data unit, including: determining a third target dimension from the first target dimension and the second target dimension based on the target processing rule; In the target summary data group, obtaining a dimension value corresponding to the third target dimension in the target summary data; Determining, based on the dimension value corresponding to the third target dimension, a dimension value corresponding to the newly added dimension in the target summary data; The incremental data unit is obtained based on the dimension value corresponding to the first target dimension, the aggregation value corresponding to the second target dimension, and the dimension value corresponding to the newly added dimension.

5. The method according to any one of claims 1 to 4, characterized in that After the target summary data is processed based on the target processing rule to obtain the incremental data unit, the method further includes: Insert the incremental data unit into the detail table to update the detail table.

6. The method according to claim 1, characterized in that The method further comprises: Get form processing tasks; Acquire a processing rule set based on the table processing task, the processing rule set including a plurality of association groups, the association groups associating the target filtering condition with the matching target processing rule; Obtaining the summary table; Obtain the first target association group according to the target execution order; determining a first target filtering condition and a first target processing rule based on the first target association group; Filtering out a target summary data group that meets a target filtering condition from multiple summary data groups; aggregating target summary data units in the target summary data group to obtain target summary data; Processing the target summary data based on the target processing rules to obtain incremental data units; Inserting the incremental data unit into the summary table to obtain an updated summary table; A second target association group is obtained according to the target execution order, and the step of obtaining the summary table is continued until the summary table is processed based on each of the association groups in the processing rule set.

7. A table processing device, characterized in that: The device comprises: An acquisition module, configured to acquire a summary table, wherein the summary table includes summary data groups, and the summary data groups are obtained by grouping summary data units in the summary table according to a first target dimension; A filtering module, configured to filter out target summary data groups that meet target filtering conditions from the summary data groups; an aggregation module, configured to aggregate the target summary data units in the target summary data group to obtain target summary data; a processing module, configured to process the target summary data based on a target processing rule to obtain an incremental data unit; An inserting module is used to insert the incremental data unit into the summary table to obtain an updated summary table.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images