Design method based on column database data splitting

Abstract

The invention discloses a data splitting design method based on a column database, and relates to the technical field of databases, the method comprises the following steps: obtaining splitting parameters and aggregating data to obtain a summarized value required by a business scene, splitting the data according to a preset splitting rule, and checking whether updating exists or not to obtain a new split summarized value; the summary value is then compared with a new summary value to determine whether data needs to be replenished. If the data need to be supplemented, the missing data are filled according to the supplementing parameters; otherwise, returning the final split object. The abstraction of the splitting rule oriented to business personnel and related design are realized, the high abstraction can better face business of a supply chain scene, the storage characteristics of a bottom layer combined column database are well used, extremely high performance is provided for data splitting, the abstraction of the splitting rule irrelevant to a development language is provided, and the development efficiency is improved. And finally, quickly modeling the business through configuration to meet business requirements.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a design method based on column-based database data splitting. Background Art

[0002] As the amount of data continues to grow, a single database is unable to cope with the problems of high-concurrency query, storage expansion and performance bottlenecks. When large amounts of data are stored in a database, users need to distribute the data to multiple databases or tables according to certain rules through data splitting to reduce the pressure on a single database and ensure efficient query and storage. At the same time, the data can be aggregated and summarized in different dimensions and time. After aggregation and summary, the summarized values ​​can also be modified. The modified values ​​are divided according to certain business rules, such as proportionally to the data before aggregation.

[0003] Prior art, such as the database transaction splitting execution method and device disclosed in the invention patent announcement with announcement number: CN112883045B, belongs to the field of cloud computing technology and can be applied to the financial field and other fields. The database transaction splitting execution method includes: splitting the data transaction according to the type of the data transaction to obtain each sub-transaction; comparing the base version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result; executing each sub-transaction to modify the corresponding database data, and releasing the exclusive lock after each sub-transaction is successfully executed.

[0004] Prior art includes a database table splitting method, system, electronic device and storage medium disclosed in the invention patent publication with publication number: CN112347098B. The method includes: searching for multiple attribute values ​​of each attribute in a basic database table according to a specified dimension; performing a specified operation on the multiple attribute values ​​of each attribute to obtain the attribute value of each attribute after the specified operation; and storing the attribute value of each attribute after the specified operation in a database table of a specified dimension.

[0005] Combining the above solutions, it is found that the current traditional OLTP and OLAP databases, whether relational databases, columnar databases or in-memory databases, do not provide the splitting function after aggregation. They all need to be customized and developed through the SQL syntax provided by the development language of the specific database. They are highly dependent on the development language and database, and are difficult for business personnel to understand. They require in-depth involvement of technical developers. Therefore, the implementation cycle of splitting after aggregation is long and the cost is high. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a design method based on column-based database data splitting, which can effectively solve the problems involved in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a design method based on column-based database data splitting, including S1, obtaining splitting parameters, aggregating the data set in the column-based database to obtain the summary value of business scenario demand data in the column-based database.

[0008] S2. Split the data in the column-based database according to a preset splitting rule, where the result of the preset splitting rule is applied to each cell in the column-based data format.

[0009] S3. Check whether there is an update operation on the column-based database, and obtain the split logic judgment result of the business scenario requirement data through split logic judgment.

[0010] S4. Determine whether to continue splitting based on the splitting logic judgment result of the business scenario demand data. If the splitting is to be continued, execute step S2 again to perform the splitting operation. If not, based on the inspection result of the update operation, synchronously obtain the new summary value after the splitting of the column database.

[0011] S5. Determine whether the split data needs to be completed based on the data summary value of the business scenario requirements in the column-based database and the new summary value after the split of the column-based database. If it needs to be completed, fill in the missing data according to the splitting parameters and return the final split object and update the column-based database. If it does not need to be completed, directly return the final split object and update the column-based database.

[0012] Furthermore, the splitting rule includes: splitting and copying to any cell.

[0013] Split and copy to all cells.

[0014] Split and copy to the cell with the maximum value.

[0015] Split the cells copied to the smallest value.

[0016] Split evenly by the number of cells.

[0017] Split in proportion to the cell values. If all cells are empty, no splitting will be performed.

[0018] Split in proportion according to the cell values. If all cells are empty, split evenly according to the number of cells.

[0019] The cells of the reference indicator are split proportionally. If all cells are empty, no splitting is performed.

[0020] The cells of the reference indicator are split proportionally. If all cells are empty, they are split evenly according to the number of cells.

[0021] Furthermore, the data in the column-based database is split according to a preset splitting rule, and it is checked whether the column-based database has any update operation. The specific process is: obtain a data set in the column-based database, split the data in the column-based database according to the preset splitting rule, execute a query to check whether the column-based database has any update operation, if the column-based database has an update operation, count and prompt the number of rows affected by the update operation, if the column-based database has no update operation, then the number of rows affected by the update operation is recorded as zero and a prompt is given.

[0022] Furthermore, based on the inspection result of the update operation, a new summary value after the column-based database is split is synchronously obtained. The specific process is: traverse and check whether the content of each underlying cell of the data in the column-based database is empty. If at least one underlying cell is not empty, the logical judgment result of the splitting of the business scenario requirement data is defined as continuing to split, and the data in the column-based database is continued to be split according to the preset splitting rules. If all the underlying cells are empty, the logical judgment result of the splitting of the business scenario requirement data is defined as not to split, and the data set after the splitting of the column-based database is directly aggregated to obtain the new summary value after the splitting of the column-based database.

[0023] Furthermore, the method determines whether the split data needs to be filled in based on the summary value of the business scenario requirement data in the column-based database and the new summary value after the split of the column-based database. The specific process is: extract the summary value of the business scenario requirement data in the column-based database and the new summary value after the split of the column-based database and compare them. If the summary value of the business scenario requirement data in the column-based database is equal to the new summary value, filling is not required, and the final split object is returned. If the summary value of the business scenario requirement data in the column-based database is not equal to the new summary value, filling is required, and the summary value of the business scenario requirement data in the column-based database is subtracted from the new summary value, and the split data is adjusted and filled in according to the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database.

[0024] Furthermore, the split data is adjusted and completed by the difference between the new summary value and the summary value of the business scenario requirement data in the column database, the adjusted data is updated back to the column database, and the relevant business parties are notified that the data has been updated and meets the requirements and the number of rows affected by the update operation is displayed.

[0025] Furthermore, the splitting parameters include a preset splitting ratio from large to small and a minimum precision.

[0026] The difference result between the new summary value and the summary value of the business scenario demand data in the column-based database includes: if the difference result between the new summary value and the summary value of the business scenario demand data in the column-based database is positive, then deduct one by one according to the preset split ratio from large to small and the minimum precision until the difference between the new summary value and the summary value of the business scenario demand data in the column-based database is zero;

[0027] If the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is negative, the new summary value is increased one by one according to the preset split ratio from small to large and the minimum precision until the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is zero.

[0028] The second aspect of the present invention also provides a device based on the column-based database data splitting method, including: a data aggregation module, used to obtain splitting parameters, aggregate the data set in the column-based database to obtain the business scenario requirement data summary value in the column-based database.

[0029] The data splitting rule module is used to split the data in the column-based database according to a preset splitting rule, and the result of the preset splitting rule acts on each cell in the column-based data format.

[0030] The data update module is used to check whether there is an update operation on the column database, and obtain the split logic judgment result of the business scenario required data through split logic judgment.

[0031] The data splitting check module is used to decide whether to continue splitting based on the splitting logic judgment results of the business scenario data. If the splitting is to continue, the splitting operation will continue. If not, based on the inspection results of the update operation, the new summary value after the splitting of the column database is synchronously obtained.

[0032] The data completion module is used to determine whether the split data needs to be completed based on the data summary value required by the business scenario in the column database and the new summary value after the column database is split. If completion is required, the missing data is filled according to the splitting parameters and the final split object is returned and the column database is updated. If completion is not required, the final split object is directly returned and the column database is updated.

[0033] The third aspect of the present invention also provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements any of the above-mentioned design methods based on columnar database data splitting.

[0034] A fourth aspect of the present invention also provides a computer-readable storage medium for storing a program, wherein when the program is executed by a processor, the program implements any of the above-mentioned design methods based on column-based database data splitting.

[0035] The present invention has the following beneficial effects:

[0036] (1) The present invention first obtains the splitting parameters and aggregates the data to obtain the summary value required by the business scenario, and splits the data according to the preset splitting rules, and checks whether there is an update operation, and then obtains the splitting result of the data required by the business scenario according to the splitting logic, and decides whether to continue the splitting and obtain the new summary value after the split or directly obtain the new summary value after the split, and decides whether to fill the data based on the comparison between the summary value and the new summary value. If filling is required, the missing data is filled according to the filling parameters; otherwise, the final split object is returned. An abstraction of a splitting rule for business personnel and related designs are implemented. Its high degree of abstraction can better face the business of the supply chain scenario, and makes good use of the storage characteristics of the underlying combined columnar database, which provides extremely high performance for data splitting, and provides a splitting rule abstraction that is independent of the development language. Finally, through configuration, the business is quickly modeled to meet business needs.

[0037] (2) The present invention first extracts a data set and splits it according to different splitting rules, then checks whether there is an update operation, counts the number of affected rows, and finally aggregates the newly split data to obtain a new summary value. In the absence of an update, the aggregation step can be skipped to optimize the computational efficiency. And by defining an abstract class to implement general splitting operations and processing logic, including judging the needs of splitting and completing, the flexibility and efficiency of data splitting are improved, ensuring that business needs can be responded to in real time when processing data, and that the data can be effectively adapted to a dynamic business environment, thereby improving the accuracy and efficiency of data processing.

[0038] (3) The present invention defines a variety of splitting rules in the data splitting design based on the column-based database, which helps to ensure the flexibility and accuracy of the splitting operation and avoid unnecessary calculations in the case of invalid data, thereby improving the processing efficiency. This method effectively combines the characteristics of the column-based database, so that data processing can better meet business needs and further improve the performance and response speed of data splitting.

[0039] (4) The present invention adjusts the split data to meet the business needs by comparing the difference between the new summary value and the summary value of the business scenario demand data, which helps the split data to accurately match the business needs. At the same time, the adjusted data is updated back to the column database and the relevant parties are notified, which helps to improve the accuracy and reliability of the data, enables the system to dynamically respond to changes in business needs, and ensures that the data is always in a valid state, thereby reducing the potential risk of errors.

[0040] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0042] Figure 2 The following is a schematic diagram of an application scenario based on column-based database data splitting.

[0043] Figure 3 This is a structural diagram of the data splitting method based on column-based database. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that the present invention combines the storage characteristics of columnar databases, provides supported splitting rules, provides business personnel with convenient, fast and easy-to-understand rules, helps business personnel to quickly complete the splitting, and can quickly and conveniently serve various business scenarios, and realizes an abstraction of splitting rules and related designs for business personnel. Its high degree of abstraction can better face the business of supply chain scenarios, and makes good use of the storage characteristics of the underlying combined columnar database, which brings extremely high performance for data splitting, and provides a splitting rule abstraction that is independent of the development language. Finally, through configuration, the business is quickly modeled to meet business needs.

[0046] See also Figure 1 As shown, an embodiment of the present invention provides a technical solution for a design method based on column-based database data splitting: a design method based on column-based database data splitting, including S1, obtaining splitting parameters, aggregating the data set in the column-based database to obtain a summary value of business scenario demand data in the column-based database.

[0047] It should be noted that the design of data splitting based on columnar database combines the storage characteristics of columnar database to define an abstract class AbstractSp l itOperator, which is used to perform a general splitting operation, and defines the entire process of executing the splitting, including the processing logic before and after data splitting, judging whether to continue splitting, judging whether to complete the splitted data, and other functions.

[0048] S2. Split the data in the column-based database according to a preset splitting rule, where the result of the preset splitting rule is applied to each cell in the column-based data format.

[0049] In this implementation, the result of the preset splitting rule will distribute the total value to each cell according to the set splitting strategy, thereby changing the specific value of each cell. The final result will be reflected in each cell of the columnar data format, which helps to ensure the integrity and consistency of the data.

[0050] In this implementation, the splitting rules act on cells according to the splitting specifications, which are a series of preset strategies, such as splitting by proportion, copying to the cell with the maximum or minimum value, etc. Further splitting logic judgment is the process of making specific judgments and operations based on the splitting rules when performing the splitting. It will consider the actual situation of the data (for example, whether the underlying cell is empty) to decide whether to continue splitting.

[0051] S3. Check whether there is an update operation on the column-based database, and obtain the split logic judgment result of the business scenario requirement data through split logic judgment.

[0052] In this implementation scheme, splitting the data in the column database according to the preset splitting rules and checking whether the column database has an update operation are in a progressive relationship. Checking whether the column database has an update operation also includes setting a related flag of the split result if a row in the column database is updated, indicating that the split is successful. If no row is updated, the split result is returned, indicating that no modification has been made. The validity of the split operation is ensured.

[0053] S4. Determine whether to continue splitting based on the splitting logic judgment result of the business scenario demand data. If the splitting is to be continued, execute step S2 again to perform the splitting operation. If not, based on the inspection result of the update operation, synchronously obtain the new summary value after the splitting of the column database.

[0054] It should be noted that the result of the splitting logic judgment based on the business scenario demand data is to actually split the data in the column database according to the current splitting parameters and the underlying cell status. If there are cells that are not empty, step S2 is executed again to perform the splitting operation, which means that there is still valid data that can be used for splitting. These non-empty cells provide a basis for allocating their contents to other cells according to the preset splitting rules, thereby realizing data redistribution and optimization. Therefore, the logic of continuing to split is based on these valid data to ensure that the splitting operation is meaningful and can produce results.

[0055] S5. Determine whether the split data needs to be completed based on the data summary value of the business scenario requirements in the column-based database and the new summary value after the split of the column-based database. If it needs to be completed, fill in the missing data according to the splitting parameters and return the final split object and update the column-based database. If it does not need to be completed, directly return the final split object and update the column-based database.

[0056] It should be noted that the data summary value in a column-based database refers to the data summary value before the split, which refers to the sum or statistical value of the data related to the split obtained from the database before the split operation is performed. These data are usually summarized based on a certain indicator or reference object (such as a specific metric, value or data set). For example: If you want to split the sales volume of goods in a sales record system, the summary value before the split may be the total sales of a certain area or store. The new summary value after the split can be compared with the data summary value in the column-based database before the split to compare the data changes before and after the split.

[0057] Specifically, the splitting rules include: Split and copy to any cell, which is used in scenarios where data needs to be flexibly allocated to different cells, such as dynamic reporting or customized analysis.

[0058] Split Copy to All Cells is used when you need to distribute data evenly, such as when a sum needs to be evenly distributed.

[0059] Split cells copied to the maximum value, which is used in scenarios where you want to highlight important data or cells with higher priority, such as when you need to concentrate resources or focus on specific data.

[0060] The cells with the smallest value are split and copied to enhance the balance of the overall analysis while supplementing some low-value data.

[0061] Split evenly by the number of cells, used when all cells are equally important, such as a preliminary allocation strategy.

[0062] Split in proportion to the cell values. If all cells are empty, no splitting is performed. This method is used in scenarios where data has obvious differences and you want to allocate data based on actual conditions.

[0063] Split in proportion to the cell values. If all cells are empty, split evenly according to the number of cells. This rule can be used when data values ​​are missing but you still want to distribute them.

[0064] The cells of the reference indicator are split proportionally, where no splitting is done if all cells are empty. This is used when allocation is based on external or other criteria, usually when comparison and standard evaluation are involved.

[0065] The data is split proportionally according to the value of the cell of the reference indicator. If all cells are empty, the data is split evenly according to the number of cells. This method is used when there are reference indicators but the actual data is missing, but you still want to ensure the rationality and balance of data distribution.

[0066] It should be noted that, as shown in Table 1, based on business analysis, the present invention implements the following splitting rules for different aggregation splits.

[0067] Table 1 Splitting rules for different aggregation splits

[0068]

[0069]

[0070] It should be noted that in Table 1, reference description (1) to reference description (9) correspond to the following aggregation splitting rules:

[0071] Reference instructions (1): Split and copy to any cell.

[0072] Refer to Instructions (2): Split and copy to all cells.

[0073] Refer to Instructions (3): Split and copy cells to the maximum value.

[0074] Refer to Note (4): Split and copy to the cell with the smallest value.

[0075] Refer to Note (5): Split evenly by the number of cells.

[0076] Reference (6): Split in proportion to the cell values. If all cells are empty, no splitting will be performed.

[0077] Reference (7): Split in proportion to the cell values. If all cells are empty, split evenly according to the number of cells.

[0078] Reference Note (8): The cells of the reference indicator are split proportionally. If all cells are empty, no splitting is performed.

[0079] Reference Note (9): The cells of the reference indicator are split proportionally. If all cells are empty, the cells are split evenly according to the number of cells.

[0080] Specifically, the data in the columnar database is split according to the preset splitting rules, and it is checked whether there is any update operation in the columnar database. The specific process is: obtain the data set in the columnar database, split the data in the columnar database according to the preset splitting rules, execute a query to check whether there is any update operation in the columnar database, if the columnar database has an update operation, count and prompt the number of rows affected by the update operation, if the columnar database has no update operation, then the number of rows affected by the update operation is recorded as zero and a prompt is given.

[0081] It should be noted that if the data set of a columnar database is not updated, the aggregation and summary steps can be skipped to avoid unnecessary calculations.

[0082] Specifically, based on the inspection result of the update operation and synchronously obtaining the new summary value after the column-based database is split, the specific process is: according to the splitting parameters, traverse and check whether the content of each underlying cell is empty. If at least one underlying cell is not empty, the splitting logic judgment result of the business scenario requirement data is defined as continuing to split, and continue to split the data in the column-based database according to the preset splitting rules. If all the underlying cells are empty, the splitting logic judgment result of the business scenario requirement data is defined as not to split, and the data set after the column-based database is split is directly aggregated to obtain the new summary value after the column-based database is split.

[0083] It should be noted that, in this embodiment, the disAggStep1 GetValueBeforeUpdate method is called to obtain the data summary value before splitting, that is, the data state before the splitting operation is performed, and the result is stored in the SplitResult object.

[0084] It should be noted that in this embodiment, the ifEmptyToAct ion method is called to determine whether the bottom-level cell to be split is empty and decide whether to continue the split operation. If no further splitting is required, the result is directly returned, indicating that no split operation is performed.

[0085] It should be noted that the bottom-level cell generally refers to the cell at the deepest level in the data structure, which is usually the target object of the split operation, usually a row or a column of a table.

[0086] It should be noted that there are usually several possible situations to determine whether a cell is empty:

[0087] (1)NULL value: a null value in the database.

[0088] (2) Empty string: Even if the data type is not NULL, some fields may be assigned an empty string ("").

[0089] (3) Default value: In some cases, the underlying cell may be initialized to a specific default value, which needs to be determined based on business rules.

[0090] Specifically, whether the split data needs to be filled is determined based on the summary value of the business scenario requirement data in the column-based database and the new summary value after the split of the column-based database. The specific process is: extract the summary value of the business scenario requirement data in the column-based database and the new summary value after the split of the column-based database and compare them. If the summary value of the business scenario requirement data in the column-based database is equal to the new summary value, filling is not required, and the final split object is returned. If the summary value of the business scenario requirement data in the column-based database is not equal to the new summary value, filling is required, and the summary value of the business scenario requirement data in the column-based database is subtracted from the new summary value, and the split data is adjusted and filled according to the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database.

[0091] It should be noted that padding means calculating the difference between the summary value after splitting and the summary value of the business scenario data required. If a difference exists, adjust the value of each row according to the size of the difference. If the sum after splitting is less than the summary value of the business scenario data required, increase the value of each row; if the sum is greater than the target value, reduce the value of each row. In the isMakeUp method, fract ionByScalePerRow indicates the value that needs to be padded for each row, and makeUpRows indicates the number of rows that need to be padded. The specific padding operation may include recalculating and updating the data of these rows.

[0092] If padding is required, the padding parameters are obtained to fill the missing data. If padding is not required, the final split object is returned.

[0093] Specifically, the split data is adjusted and completed by the difference between the new summary value and the summary value of the business scenario requirement data in the column database, the adjusted data is updated back to the column database, and the relevant business parties are notified that the data has been updated and meets the requirements and the number of rows affected by the update operation is displayed.

[0094] Specifically, the splitting parameters include a preset splitting ratio from large to small and a minimum precision.

[0095] The difference between the new summary value and the summary value of the business scenario requirement data in the column-based database includes: if the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is positive, then deduct them one by one according to the preset split ratio from large to small and the minimum precision until the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is zero.

[0096] If the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is negative, the new summary value is increased one by one according to the preset split ratio from small to large and the minimum precision until the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is zero.

[0097] It should be noted that the data in the column-based database is split according to the preset splitting rules, for example, the process of splitting by proportion is as follows:

[0098] A1: Calculates the percentage of each cell in the total value.

[0099] A2: Multiply the modified value by the percentage and round it according to the precision of the indicator.

[0100] A3: Summarize again based on the split values.

[0101] A4: Compare the aggregated value with the split value, including:

[0102] (1) If they are equal, the split is completed.

[0103] (2) If they are not equal, the difference between the aggregated value after splitting and the split value is obtained. The specific process is:

[0104] If the difference is positive, it is deducted one by one according to the preset proportion, from large to small, with the minimum precision, until the difference is 0.

[0105] If the difference is negative, it is increased one by one according to the preset proportion, from small to large, with the minimum precision, until the difference is 0.

[0106] It should be noted that deducting one by one according to the preset split ratio from large to small and the minimum precision is specifically to determine which cell values ​​need to be deducted according to the preset split ratio. Usually, the deduction starts from the cell with the largest proportion. And in each deduction, the cell value is gradually reduced according to the set minimum precision until the difference between the new summary value and the required data summary value is zero.

[0107] It should also be noted that deducting one by one means deducting the cells that meet the conditions in sequence to ensure that the order of deduction complies with the preset rules.

[0108] It should be noted that the column-based database provided in this embodiment is a data structure that is quickly calculated on a column structure, and the splitting rules and methods for different aggregation splitting of the present invention need to be designed based on a column-based storage structure. The results of the splitting rules for different aggregation splitting act on each element (grid) in the column data format for calculation. For example, according to business needs, the splitting rules for different aggregation splitting are divided into multiple situations, and it is necessary to perform an abstract design based on the splitting rules for different aggregation splitting. For the case of splitting by proportion, the splitting results must be consistent with the input value after being summarized again, and the padding operation needs to be considered to ensure the correctness of the results.

[0109] It should be noted that, as shown in Table 2, Table 2 is a commonly used business scenario, which lists the indicators of each commodity in each region in each quarter of 2024, including inventory and expected sales.

[0110] Table 2 Business scenarios

[0111]

[0112]

[0113] It should be noted that the business scenario based on Table 2 is split according to the proportion of data split. For example, business personnel need to summarize the data for the whole country and 2024, as shown in Table 3.

[0114] It should be noted that Table 3 is the summary data for the whole country and 2024 in this scenario, which lists the indicators of various commodities in the country in 2024, including inventory and expected sales.

[0115] Table 3 Summary data for the whole country and 2024 in this scenario

[0116]

[0117] It should be noted that based on Table 3, taking the indicator of "estimated sales volume" of "coffee" in "the whole country" in "2024" as an example, the current summary value is 300. Now according to the sales target requirements, it is adjusted from 300 to 600, and the value of 600 is split into the original locations and quarters according to the original proportions. The results are shown in Table 4.

[0118] It should be noted that Table 4 shows the results of the national and 2024 proportion split in this scenario, which lists the expected sales volume and indicators of coffee in various regions across the country in 2024, including inventory and expected sales.

[0119] Table 4 Results of the national and 2024 ratio split in this scenario

[0120]

[0121]

[0122] It should be noted that in Table 4, because the split values ​​need to be recalculated according to percentage, the final result after aggregation is 601, which is 1 more than 600 (due to rounding), and the extra 1 needs to be deducted to ensure that the total is 600. Therefore, the value of "coffee" in "Beijing" in the "first quarter of 2024" is changed from 40 to 39.

[0123] It should be noted that if Figure 2 As shown, Figure 2 The following is a schematic diagram of an application scenario based on data splitting in a column-based database. Specifically, the total sales of a region are split by product type. The specific process is as follows: Obtaining split parameters is to obtain split parameters from user input or configuration files, such as target region, product type, and split ratio. Data aggregation aggregates the sales data in the column-based database, calculates the total sales of the target region (i.e., the summary value of the business scenario demand data), and splits the sales data according to the preset splitting rules (such as by proportion, by maximum value, etc.). The result of the splitting rule will be applied to the sales of each product type, and then check whether there is an update operation in the column-based database. If there is an update, the number of affected rows is recorded, and the splitting logic judgment result is defined as "continue splitting"; if there is no update, it is defined as "no splitting"; decide whether to continue splitting based on the splitting logic judgment result. If the splitting continues, return to the data splitting step and continue to perform the data splitting operation. If not splitting. Finally, determine whether it needs to be completed. Based on the check result of the update operation, synchronously obtain the new summary value after the column-based database split and return the split object.

[0124] like Figure 3 As shown, the second aspect of the present invention also provides a device based on a column-based database data splitting method, including: a data aggregation module, used to obtain splitting parameters, aggregate the data set in the column-based database to obtain a summary value of business scenario demand data in the column-based database.

[0125] The data splitting rule module is used to split the data in the column-based database according to a preset splitting rule, and the result of the preset splitting rule acts on each cell in the column-based data format.

[0126] The data update module is used to check whether there is an update operation on the column database, and obtain the split logic judgment result of the business scenario required data through split logic judgment.

[0127] The data splitting check module is used to decide whether to continue splitting based on the splitting logic judgment results of the business scenario data. If the splitting is to continue, the splitting operation will continue. If not, based on the inspection results of the update operation, the new summary value after the splitting of the column database is synchronously obtained.

[0128] The data completion module is used to determine whether the split data needs to be completed based on the data summary value required by the business scenario in the column database and the new summary value after the column database is split. If completion is required, the missing data is filled according to the splitting parameters and the final split object is returned and the column database is updated. If completion is not required, the final split object is directly returned and the column database is updated.

[0129] The third aspect of the present invention also provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements any of the above-mentioned design methods based on columnar database data splitting.

[0130] A fourth aspect of the present invention also provides a computer-readable storage medium for storing a program, wherein when the program is executed by a processor, the program implements any of the above-mentioned design methods based on column-based database data splitting.

[0131] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0132] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A design method based on column-based database data splitting, characterized in that: include: S1. Obtain splitting parameters, aggregate the data set in the column-based database to obtain the summary value of the business scenario demand data in the column-based database; S2. Splitting the data in the column-based database according to a preset splitting rule, wherein the result of the preset splitting rule is applied to each cell in the column-based data format; S3. Check whether there is an update operation on the column-based database, and obtain the splitting logic judgment result of the business scenario requirement data through splitting logic judgment; S4. Determine whether to continue splitting based on the splitting logic judgment result of the business scenario demand data. If the splitting is to be continued, execute step S2 again to perform the splitting operation. If the splitting is not to be performed, based on the inspection result of the update operation, synchronously obtain the new summary value after the splitting of the column-based database; S5. Determine whether the split data needs to be completed based on the data summary value of the business scenario requirements in the column-based database and the new summary value after the split of the column-based database. If it needs to be completed, fill in the missing data according to the splitting parameters and return the final split object and update the column-based database. If it does not need to be completed, directly return the final split object and update the column-based database.

2. The method according to claim 1, characterized in that: The splitting rules include: splitting and copying to any cell; Split and copy to all cells; Split and copy the cells with the maximum value; Split the cells copied to the minimum value; Split evenly by the number of cells; Split in proportion to the cell values, and do not split if all cells are empty; Split in proportion according to the cell value. If all cells are empty, split evenly according to the number of cells. The cells of the reference indicator are split proportionally. If all cells are empty, no splitting is performed. The cells of the reference indicator are split proportionally. If all cells are empty, they are split evenly according to the number of cells.

3. The method according to claim 1, characterized in that: The data in the column-based database is split according to the preset splitting rules, and the column-based database is checked for update operations. The specific process is as follows: Get the data set in the columnar database, split the data in the columnar database according to the preset splitting rules, execute the query to check whether there is any update operation in the columnar database, if there is an update operation in the columnar database, count and prompt the number of rows affected by the update operation, if there is no update operation in the columnar database, then record the number of rows affected by the update operation as zero and prompt.

4. The method according to claim 1, characterized in that: Based on the check result of the update operation, the new summary value after the column database is split is synchronously obtained. The specific process is: Traverse and check whether the contents of each underlying cell of the data in the column-based database are empty. If at least one underlying cell is not empty, the splitting logic judgment result of the business scenario requirement data is defined as continuing to split, and the data in the column-based database is continued to be split according to the preset splitting rules. If all the underlying cells are empty, the splitting logic judgment result of the business scenario requirement data is defined as not to split, and the data set after the column-based database is split is directly aggregated to obtain the new summary value after the column-based database is split.

5. The method according to claim 1, characterized in that: The specific process of judging whether the split data needs to be filled based on the summary value of the business scenario demand data in the column-based database and the new summary value after the split in the column-based database is as follows: Extract the summary value of the business scenario requirement data in the column-based database and the new summary value after the column-based database is split and compare them. If the summary value of the business scenario requirement data in the column-based database is equal to the new summary value, there is no need to fill in the gaps, and return the final split object. If the summary value of the business scenario requirement data in the column-based database is not equal to the new summary value, it is necessary to fill in the gaps, and subtract the summary value of the business scenario requirement data in the column-based database from the new summary value. Adjust the filled split data according to the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database. The method of adjusting and completing the split data by using the difference between the new summary value and the summary value of the business scenario requirement data in the column database includes updating the adjusted data back to the column database, notifying the relevant business parties that the data has been updated and meets the requirements, and displaying the number of rows affected by the update operation.

6. The method according to claim 5, characterized in that: The splitting parameters include a preset splitting ratio from large to small and a minimum precision; The difference between the new summary value and the summary value of the business scenario demand data in the column database includes: If the difference between the new summary value and the summary value of the business scenario demand data in the column-based database is positive, the values ​​are deducted one by one according to the preset split ratio from large to small and the minimum precision until the difference between the new summary value and the summary value of the business scenario demand data in the column-based database is zero; If the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is negative, the new summary value is increased one by one according to the preset split ratio from small to large and the minimum precision until the difference between the new summary value and the summary value of the business scenario requirement data in the column-based database is zero.

7. A device for applying the method for splitting data based on a column-based database as claimed in any one of claims 1 to 6, characterized in that: include: The data aggregation module is used to obtain splitting parameters and aggregate the data sets in the column-based database to obtain the data aggregation values ​​required by business scenarios in the column-based database; A data splitting rule module is used to split the data in the column-based database according to a preset splitting rule, wherein the result of the preset splitting rule acts on each cell in the column-based data format; The data update module is used to check whether there is an update operation on the column database, and obtain the split logic judgment result of the business scenario required data through split logic judgment; The data splitting check module is used to determine whether to continue splitting based on the splitting logic judgment results of the business scenario required data. If splitting is to be continued, the splitting operation is continued. If not, the new summary value after splitting of the column-based database is synchronously obtained based on the check results of the update operation; The data completion module is used to determine whether the split data needs to be completed based on the data summary value required by the business scenario in the column database and the new summary value after the column database is split. If completion is required, the missing data is filled according to the splitting parameters and the final split object is returned and the column database is updated. If completion is not required, the final split object is directly returned and the column database is updated.

8. An electronic device, characterized in that: include: A processor, and memory and network interfaces connected to the processor; The network interface is connected to a non-volatile memory in the server; When running, the processor retrieves the computer program from the non-volatile memory through the network interface, and runs the computer program through the memory to execute the design method based on columnar database data splitting as described in any one of claims 1-6 above.

9. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the design method based on column-based database data splitting as described in any one of claims 1 to 6 is implemented.

Images