A method and device for determining database partitioning strategy

Abstract

The present invention provides a method and device for determining a database sub-library and table sub-strategy. The method uses the historical data volume of a database table in a target database in combination with a data prediction model to predict the total data volume in the target database and obtain the total data volume change trend. According to the total data volume change trend, the peak time point when the total data volume reaches a threshold is determined. The historical data in the database table is periodically segmented to obtain multiple groups of historical data corresponding to different segmentation periods. For each segmentation period, each group of historical data corresponding to the segmentation period is verified to obtain corresponding verification results. According to the verification results corresponding to all segmentation periods, it is determined whether the historical data in the database table has changed, and the sub-library and table sub-strategy is determined according to the judgment result. According to the peak time point and the sub-library and table sub-strategy, the sub-library and table sub-strategy of the target database is determined to improve the design rationality of the sub-library and table sub-strategy, thereby ensuring the stable operation of the application system.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for determining a database sub-library and table sub-strategy. Background Art

[0002] In the application and development of computer software technology, the use of databases occupies an important part. With the long-term operation of application systems and the advent of information explosion, databases are becoming more and more bloated and affecting database performance. Therefore, it is necessary to divide the database into different libraries and tables to ensure the stable operation of the application system.

[0003] The current method of sharding a database is to estimate the future data volume during the development and design phase of the application system, and design the database sharding strategy based on the estimated results, completing the database sharding work in advance. However, since many business scenarios cannot estimate the future data volume, the estimated data volume will deviate greatly from the actual situation, which in turn leads to unreasonable design of the sharding strategy, which cannot guarantee the stable operation of the application system, and the rationality of the design of the sharding strategy is poor. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for determining a database partitioning strategy to solve the problem that the current database partitioning method has poor design rationality.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of an embodiment of the present invention discloses a method for determining a database partitioning strategy, the method comprising:

[0007] Using the historical data volume of the database table in the target database collected in advance, combined with the pre-built data prediction model, the total data volume in the target database is predicted to obtain the total data volume change trend of the target database in the future time period;

[0008] Determine, based on the total data volume change trend, a peak time point at which the total data volume of the target database reaches a threshold;

[0009] Periodically dividing the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different segmentation periods, each segmentation period corresponding to multiple groups of historical data;

[0010] For each divided period, verify each set of historical data corresponding to the divided period to obtain a corresponding verification result;

[0011] According to the verification results corresponding to all the divided periods, judging whether the historical data in the database table has changed, and determining the database and table division method of the target database according to the judgment result;

[0012] The library and table sharding strategy of the target database is determined according to the peak time point and the library and table sharding method.

[0013] Preferably, judging whether the historical data in the database table has changed according to the verification results corresponding to all the divided periods, and determining the database and table division method of the target database according to the judgment result, includes:

[0014] In any two adjacent divided periods, if the difference between the verification results corresponding to any pair of historical data with the same group number is greater than or equal to a preset value, it is determined that the historical data in the database table has changed, and the database and table sharding method of the target database is determined to be the first database and table sharding method, which performs database and table sharding based on the hash value of the unique key of the database table;

[0015] If in each pair of adjacent divided periods, the differences between the verification results corresponding to the historical data with the same group number are all less than the preset value, it is determined that the historical data in the database table has not changed, and it is determined that the library and table sharding method of the target database is the second library and table sharding method, which performs library and table sharding based on the range of the unique key of the database table.

[0016] Preferably, for each divided period, verifying each set of historical data corresponding to the divided period to obtain a corresponding verification result includes:

[0017] For each divided period, a cyclic redundancy check (CRC) is performed on each group of historical data corresponding to the divided period to obtain a corresponding CRC check result.

[0018] Preferably, the periodic division of the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different division periods includes:

[0019] According to the unique key of the database table, the historical data in the database table is periodically segmented to obtain multiple groups of historical data corresponding to different segmentation periods.

[0020] Preferably, the method of using the historical data volume of the database table in the target database collected in advance and combining it with the pre-built data prediction model to predict the total data volume in the target database and obtain the total data volume change trend of the target database in the future time period includes:

[0021] By using the historical data volume of the database table in the target database collected in advance, combined with the data prediction model pre-built based on the weighted average algorithm or regression analysis algorithm, the total data volume in the target database is predicted to obtain the total data volume change trend of the target database in the future time period.

[0022] Preferably, after determining the database and table sharding strategy of the target database according to the peak time point and the database and table sharding method, the method further includes:

[0023] Determine the warning time point according to the peak time point and the preset reminder time requirement;

[0024] The database and table sharding strategy is sent to the target object at the warning time point.

[0025] A second aspect of an embodiment of the present invention discloses a device for determining a database partitioning strategy, the device comprising:

[0026] A prediction unit, used to use the historical data volume of the database table in the target database collected in advance, combined with a pre-built data prediction model, to predict the total data volume in the target database, and obtain a change trend of the total data volume of the target database in a future time period;

[0027] A first determining unit, configured to determine a peak time point at which the total data volume of the target database reaches a threshold value according to a change trend of the total data volume;

[0028] A division unit, used for periodically dividing the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different division periods, each of the division periods corresponding to multiple groups of historical data;

[0029] A verification unit, configured to verify each set of historical data corresponding to each divided period, and obtain a corresponding verification result;

[0030] A processing unit, configured to determine whether the historical data in the database table has changed according to the verification results corresponding to all the divided periods, and determine the database and table division method of the target database according to the determination result;

[0031] The second determination unit is used to determine the library and table sharding strategy of the target database according to the peak time point and the library and table sharding method.

[0032] Preferably, the processing unit comprises:

[0033] A first determination module is used to determine that the historical data in the database table has changed if the difference between the verification results corresponding to any pair of historical data with the same group number is greater than or equal to a preset value in any two adjacent divided periods, and to determine that the library and table sharding method of the target database is a first library and table sharding method, which performs library and table sharding based on a hash value of a unique key of the database table;

[0034] The second determination module is used to determine that the historical data in the database table has not changed if the difference between the verification results corresponding to the historical data with the same group number in each pair of adjacent divided periods is less than the preset value, and to determine that the library and table sharding method of the target database is the second library and table sharding method, and the second library and table sharding method performs library and table sharding based on the range of the unique key of the database table.

[0035] Preferably, the verification unit is specifically used for: for each divided period, performing a cyclic redundancy check (CRC) on each group of historical data corresponding to the divided period to obtain a corresponding CRC verification result.

[0036] Preferably, the division unit is specifically used to: periodically divide the historical data in the database table into segments according to the unique key of the database table, so as to obtain multiple groups of historical data corresponding to different division periods.

[0037] Based on the above-mentioned embodiment of the present invention, a method and device for determining a database sub-library and table strategy are provided. The method comprises: using the historical data volume of a database table in a target database collected in advance, combined with a pre-built data prediction model, predicting the total data volume in the target database, and obtaining a total data volume change trend of the target database in a future time period; determining the peak time point when the total data volume of the target database reaches a threshold value according to the total data volume change trend; periodically dividing the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different division periods; for each division period, verifying each group of historical data corresponding to the division period to obtain a corresponding verification result; judging whether the historical data in the database table has changed according to the verification results corresponding to all division periods, and determining the sub-library and table mode of the target database according to the judgment result; determining the sub-library and table strategy of the target database according to the peak time point and the sub-library and table mode, and determining when and how to perform sub-library and table division of the database according to the sub-library and table strategy determined according to the actual situation of the database, thereby improving the design rationality of the sub-library and table strategy, and thus ensuring the stable operation of the application system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flowchart of a method for determining a database partitioning strategy provided by an embodiment of the present invention;

[0040] Figure 2 A line diagram showing the amount of historical data in a database table provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of verification results of each group of historical data corresponding to a divided period provided by an embodiment of the present invention;

[0042] Figure 4 A structural block diagram of a device for determining a database partitioning strategy provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0045] As can be seen from the background technology, when sharding a database, the sharding strategy is usually designed in advance during the development and design phase of the application system. However, since many business scenarios cannot predict the future data volume, the estimated data volume will deviate greatly from the actual situation, which in turn leads to unreasonable design of the sharding strategy, which cannot guarantee the stable operation of the application system and the design rationality of the sharding strategy is poor.

[0046] Therefore, an embodiment of the present invention provides a method and device for determining a database sub-library and table strategy, which uses a data prediction model to process the historical data volume of a database table in a target database, and predicts the change trend of the total data volume of the target database in a future time period. According to the change trend of the total data volume, determine the peak time point when the total data volume of the target database reaches the threshold. Periodically divide the historical data in the database table into segments, and verify each group of historical data corresponding to each segmented period to obtain the corresponding verification results. According to the verification results, determine whether the historical data in the database table has changed, and determine the sub-library and table method of the target database based on the judgment results. Determine the sub-library and table strategy of the target database based on the peak time point and the sub-library and table method. According to the sub-library and table strategy determined according to the actual situation of the database, determine when and how to perform sub-library and table division of the database, so as to improve the design rationality of the sub-library and table strategy, and thus ensure the stable operation of the application system.

[0047] See also Figure 1 , shows a flow chart of a method for determining a database sub-library and table sub-strategy provided by an embodiment of the present invention, the determination method comprising:

[0048] Step S101: using the previously collected historical data volume of the database table in the target database, combined with the pre-built data prediction model, the total data volume in the target database is predicted to obtain the total data volume change trend of the target database in the future time period.

[0049] It should be noted that the database table of the target database refers to: a table used by the business in the target database, and the data volume of the database table refers to: the total number of rows in the database table.

[0050] In the specific implementation of step S101, the historical data volume of the database table in the target database is collected in advance on a regular basis to obtain multiple historical data volumes. For example, the historical data volume of the database table in the past 20 weeks (from the current time) is collected on a regular basis. The change of the historical data volume of the database table in the past 20 weeks is as follows: Figure 2 As shown, through Figure 2 The line diagram of the historical data volume of the database table shown can intuitively understand the changes in the historical data volume in the past 20 weeks.

[0051] After collecting the historical data volume of the database table of the target database, the total data volume in the target database is predicted using the historical data volume and a pre-built data prediction model to obtain the total data volume change trend of the target database in the future time period. Through the total volume change trend, the change trend of the total data volume of the target database in the future time period can be learned.

[0052] In some specific embodiments, the data prediction model can be constructed based on data prediction algorithms such as a weighted average algorithm or a regression algorithm. That is to say, when predicting the changing trend of the total data volume in the target database, the historical data volume of the database table in the target database collected in advance is used, combined with the data prediction model constructed in advance based on the weighted average algorithm or the regression analysis algorithm, to predict the total data volume in the target database and obtain the changing trend of the total data volume of the target database in the future time period.

[0053] Step S102: Determine the peak time point when the total data volume of the target database reaches a threshold value according to the total data volume change trend.

[0054] In the specific implementation of step S102, a threshold of the total number of target databases is preset, and the threshold indicates the maximum value of the total amount of data that can be carried by the target database under the premise of ensuring the stable operation of the application system.

[0055] According to the change trend of the total data volume of the target database, the peak time point when the total data volume of the target database reaches the threshold can be determined, that is, the time point at which the total data volume of the target database reaches the threshold can be determined.

[0056] For example: Assume that the amount of historical data used to predict the total data volume change trend of the target database is the historical data volume of the past 20 weeks, and assume that the threshold is 2 million rows. According to the total data volume change trend, it is determined that the total data volume of the target database reaches the threshold 20 weeks from the current time. At this time, the time corresponding to 20 weeks from the current time is the peak time point.

[0057] In other words, to ensure the stable operation of the application system, the target database needs to be split into different databases and tables before the peak time point arrives.

[0058] It should be noted that the target database must meet a prerequisite for performing database and table sharding operations before the peak time point is reached, which is that the life cycle of the application system has not ended, that is, the application system is still within its life cycle.

[0059] It should be further explained that the application system has a corresponding life cycle, and the application system runs within the life cycle. For example, assuming that the life cycle of the application system is 10 years, the application system will no longer run after running for 10 years.

[0060] Step S103: Periodically divide the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different segmentation periods.

[0061] It should be noted that each division period corresponds to multiple groups of historical data, that is, in each division period, the historical data in the database table is divided into multiple groups of historical data.

[0062] In the specific process of implementing step S103, the historical data in the database table of the target database is periodically segmented, that is, in each segmentation period, the historical data in the database table is segmented into multiple groups of historical data.

[0063] In a specific implementation, the historical data in the database table is periodically segmented according to the unique key of the database table to obtain multiple groups of historical data corresponding to different segmentation periods.

[0064] It can be understood that the unique key of a database table is a primary key or a unique index. A unique key is an ordered and non-repeating value. Each unique key value in a database table represents a row of data.

[0065] For example, a database table containing 5 million rows of data has 5 million different unique key values. When the historical data in the database table is segmented each time, 1 million rows of data are grouped together, so the 5 million rows of data in the database table can be divided into 5 groups of historical data. In other words, each segmentation period corresponds to 5 groups of historical data.

[0066] It should be noted that the value of the unique key may be different depending on the type of data stored in the database table. For example, assuming that the data stored in the database table is account balance data, the value of the unique key is the bank card number. In the embodiment of the present invention, the value of the unique key of the database table is not specifically limited.

[0067] Step S104: for each divided period, verify each set of historical data corresponding to the divided period to obtain a corresponding verification result.

[0068] It can be seen from the content of the above step S103 that each division period divides the historical data in the database table into multiple groups of historical data. In the process of specifically implementing step S104, for each division period, a cyclic redundancy check (CRC) is performed on each group of historical data corresponding to the division period to obtain the corresponding CRC check result.

[0069] That is to say, for each divided period, each group of historical data corresponding to the divided period is subjected to CRC verification to obtain a CRC verification result corresponding to each group of historical data, and one group of historical data corresponds to one CRC verification result.

[0070] To better explain how to perform CRC check on each set of historical data in a divided period, Figure 3 A schematic diagram of the verification result of each group of historical data corresponding to a divided period is shown for example.

[0071] Figure 3In the embodiment, it is assumed that the historical data of the database table of the target database is divided into four groups of historical data in a certain division period, the first group of historical data is the historical data from the first row to the 10000th row in the database table, the second group of historical data is the historical data from the 10000th row to the 20000th row in the database table, the third group of historical data is the historical data from the 20000th row to the 30000th row in the database table, and the fourth group of historical data is the historical data from the 30000th row to the 40000th row in the database table. CRC32 check is performed on the first group of historical data, the second group of historical data, the third group of historical data, and the fourth group of historical data, respectively, to obtain the CRC32 check results corresponding to the first group of historical data, the second group of historical data, the third group of historical data, and the fourth group of historical data, respectively.

[0072] Step S105: Based on the verification results corresponding to all the divided periods, determine whether the historical data in the database table has changed, and determine the library and table division method of the target database based on the judgment result.

[0073] From the above content, it can be seen that in each division period, the historical data in the database table is divided into multiple groups of historical data and CRC verification is performed. Therefore, by comparing the verification results between historical data with the same group number but belonging to different division periods, it can be determined whether the historical data in the database table has changed, and the target database partitioning method can be determined based on the judgment result.

[0074] For example: by comparing the verification results between the Mth groups of historical data in different division periods, it is determined whether the historical data in the database table has changed, where M is an integer greater than or equal to 1, and the maximum value of M is the total number of groups of historical data corresponding to each division period.

[0075] In the process of implementing step S105, in any two adjacent divided periods, if the difference between the verification results corresponding to any pair of historical data with the same group number is greater than or equal to the preset value, it is determined that the historical data in the database table has changed, and the library and table sharding method of the target database is determined to be the first library and table sharding method, which performs library and table sharding based on the hash value of the unique key of the database table.

[0076] For example: in any two adjacent division periods (here taking the first division period and the second division period as an example), if the difference between the verification results corresponding to the Mth group of historical data in the first division period and the Mth group of historical data in the second division period is greater than or equal to the preset value, it is determined that the historical data in the database table has changed.

[0077] If the difference between the verification results corresponding to the historical data with the same group number in each pair of adjacent divided periods is less than the preset value, it is determined that the historical data in the database table has not changed, and the target database is determined to be divided into the second database and table method, which divides the database and table based on the range of the unique key of the database table.

[0078] For example: assuming that there are three division periods (here taking the first division period, the second division period and the third division period as examples), and each division period corresponds to two groups of historical data, the requirements that need to be met for the historical data in the database table to remain unchanged are: the difference between the verification results of the first group of historical data in the first division period and the first group of historical data in the second division period is less than the preset value, and the difference between the verification results of the first group of historical data in the second division period and the first group of historical data in the third division period is less than the preset value, and the difference between the verification results of the second group of historical data in the first division period and the second group of historical data in the second division period is less than the preset value, and the difference between the verification results of the second group of historical data in the second division period and the second group of historical data in the third division period is less than the preset value.

[0079] That is to say, by comparing historical data with the same group number at different time points (ie, different division periods), it is determined whether the historical data in the database table has changed.

[0080] In summary, when it is determined that the historical data in the database table of the target database has changed, the first database and table sharding method is adopted, and when it is determined that the historical data in the database table of the target database has not changed, the second database and table sharding method is adopted.

[0081] For example: when the historical data recorded in the database table is flow data, although the data in the database table is constantly added, the historical data is not updated frequently, that is, after comparing the verification results, it is determined that the historical data in the database table has not changed. At this time, the sequential data segmentation method can be used, that is, the second database and table segmentation method based on the range of the unique key of the database table is used.

[0082] For another example: when the historical data recorded in the database table is account balance data, the historical data in the database table will be updated frequently, that is, the historical data in the database table is determined to have changed through comparison of the verification results. At this time, the database and table can be sharded by taking the modulus of the hash value based on the unique key (such as the bank card number). That is, the first database and table sharding method is to use the hash value based on the unique key of the database table to shard the database and table.

[0083] Step S106: Determine the database and table sharding strategy of the target database according to the peak time point and the database and table sharding method.

[0084] In the process of specifically implementing step S106, the library and table sharding strategy of the target database is determined according to the peak time point and the library and table sharding method (the first library and table sharding method or the second library and table sharding method) determined in the above steps. The library and table sharding strategy indicates at which time point (peak time point) the target database needs to adopt which library and table sharding method (the first library and table sharding method or the second library and table sharding method) to shard the database.

[0085] Preferably, an early warning can be issued based on actual needs, and the early warning target database needs to be warned at what time to perform database and table sharding operations. In the specific implementation, the early warning time point is determined based on the peak time point and the preset reminder time requirements, and the database and table sharding strategy is sent to the target object (such as the database administrator) at the early warning time point.

[0086] For example: Assuming that according to the trend of the total data volume change, it is determined that the total data volume of the target database reaches the threshold 20 weeks from the current time, then the time corresponding to 20 weeks from the current time is the peak time point. And assuming that the reminder time requirement is to issue an early warning 12 weeks in advance, the database and table sharding strategies determined in the above steps are sent to the target object at the early warning time point 8 weeks from the current time, reminding the target object to shard the target database in the determined database and table sharding method before the peak time point.

[0087] It should be noted that the execution order of the above steps S101 to S106 includes but is not limited to the above execution order. "Step S101-Step S102" can be executed first and then "Step S103-Step S106" can be executed, or "Step S101-Step S102" and "Step S103-Step S105" can be executed in parallel before "Step S106". The specific execution order is not limited here.

[0088] In an embodiment of the present invention, a data prediction model is used to process the historical data volume of a database table in a target database, and the total data volume change trend of the target database in a future time period is predicted. According to the total data volume change trend, the peak time point when the total data volume of the target database reaches the threshold is determined. The historical data in the database table is periodically divided into segments, and each group of historical data corresponding to each segmented period is verified to obtain the corresponding verification results. According to the verification results, it is judged whether the historical data in the database table has changed, and the library and table division method of the target database is determined based on the judgment results. According to the peak time point and the library and table division method, the library and table division strategy of the target database is determined. According to the library and table division strategy determined according to the actual situation of the database, it is determined when and how to divide the database into libraries and tables, so as to improve the design rationality of the library and table division strategy, thereby ensuring the stable operation of the application system.

[0089] Corresponding to a method for determining a database partitioning strategy provided by an embodiment of the present invention, see Figure 4 , the embodiment of the present invention also provides a structural block diagram of a device for determining a database partitioning strategy, the device comprising: a prediction unit 401, a first determination unit 402, a division unit 403, a verification unit 404, a processing unit 405 and a second determination unit 406;

[0090] The prediction unit 401 is used to use the historical data volume of the database table in the target database collected in advance and combine it with the pre-built data prediction model to predict the total data volume in the target database and obtain the total data volume change trend of the target database in the future time period.

[0091] In a specific implementation, the prediction unit 401 is specifically used to: use the historical data volume of the database table in the target database collected in advance, combined with the data prediction model pre-constructed based on the weighted average algorithm or regression analysis algorithm, to predict the total data volume in the target database, and obtain the total data volume change trend of the target database in the future time period.

[0092] The first determining unit 402 is used to determine a peak time point when the total data volume of the target database reaches a threshold value according to a change trend of the total data volume.

[0093] The division unit 403 is used to periodically divide the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different division periods, and each division period corresponds to multiple groups of historical data.

[0094] In a specific implementation, the division unit 403 is specifically used to: periodically divide the historical data in the database table into segments according to the unique key of the database table, so as to obtain multiple groups of historical data corresponding to different division periods.

[0095] The verification unit 404 is used to verify each set of historical data corresponding to each divided period to obtain a corresponding verification result.

[0096] In a specific implementation, the verification unit 404 is specifically used to: for each divided period, perform CRC verification on each group of historical data corresponding to the divided period to obtain a corresponding CRC verification result.

[0097] The processing unit 405 is used to determine whether the historical data in the database table has changed according to the verification results corresponding to all the divided periods, and determine the library and table division method of the target database according to the judgment result.

[0098] The second determining unit 406 is used to determine the library and table sharding strategy of the target database according to the peak time point and the library and table sharding method.

[0099] In an embodiment of the present invention, a data prediction model is used to process the historical data volume of a database table in a target database, and the total data volume change trend of the target database in a future time period is predicted. According to the total data volume change trend, the peak time point when the total data volume of the target database reaches the threshold is determined. The historical data in the database table is periodically divided into segments, and each group of historical data corresponding to each segmented period is verified to obtain the corresponding verification results. According to the verification results, it is judged whether the historical data in the database table has changed, and the library and table division method of the target database is determined based on the judgment results. According to the peak time point and the library and table division method, the library and table division strategy of the target database is determined. According to the library and table division strategy determined according to the actual situation of the database, it is determined when and how to divide the database into libraries and tables, so as to improve the design rationality of the library and table division strategy, thereby ensuring the stable operation of the application system.

[0100] Preferably, combined Figure 4 The processing unit 405 includes a first determining module and a second determining module, and the execution principle of each module is as follows:

[0101] The first determination module is used to determine that the historical data in the database table has changed if the difference between the verification results corresponding to any pair of historical data with the same group number in any two adjacent divided periods is greater than or equal to a preset value, and to determine that the target database's database and table partitioning method is the first database and table partitioning method, which performs database and table partitioning based on the hash value of the unique key of the database table.

[0102] The second determination module is used to determine that the historical data in the database table has not changed if the difference between the verification results corresponding to the historical data with the same group number in each pair of adjacent divided periods is less than a preset value, and to determine that the target database's database and table partitioning method is the second database and table partitioning method, and the second database and table partitioning method performs database and table partitioning based on the range of the unique key of the database table.

[0103] Preferably, combined Figure 4 The content shown, the determining device also includes:

[0104] The early warning unit is used to determine the early warning time point according to the peak time point and the preset reminder time requirement, and send the database and table sharding strategy to the target object at the early warning time point.

[0105] In summary, the embodiment of the present invention provides a method and device for determining a database sub-library and table strategy, which uses a data prediction model to process the historical data volume of a database table in a target database, and predicts the change trend of the total data volume of the target database in a future time period. According to the change trend of the total data volume, determine the peak time point when the total data volume of the target database reaches the threshold. Periodically divide the historical data in the database table into segments, and verify each group of historical data corresponding to each segmented period to obtain the corresponding verification results. According to the verification results, determine whether the historical data in the database table has changed, and determine the sub-library and table mode of the target database based on the judgment results. According to the peak time point and the sub-library and table mode, determine the sub-library and table strategy of the target database. According to the sub-library and table strategy determined according to the actual situation of the database, determine when and how to perform the sub-library and table of the database, improve the design rationality of the sub-library and table strategy, and thus ensure the stable operation of the application system.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0107] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0108] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a database partitioning strategy, characterized in that: The method comprises: Using the historical data volume of the database table in the target database collected in advance, combined with the pre-built data prediction model, the total data volume in the target database is predicted to obtain the total data volume change trend of the target database in the future time period; Determine, based on the total data volume change trend, a peak time point at which the total data volume of the target database reaches a threshold; Periodically dividing the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different segmentation periods, each segmentation period corresponding to multiple groups of historical data; For each divided period, verify each set of historical data corresponding to the divided period to obtain a corresponding verification result; According to the verification results corresponding to all the divided periods, judging whether the historical data in the database table has changed, and determining the database and table division method of the target database according to the judgment result; Determine the database and table sharding strategy of the target database according to the peak time point and the database and table sharding method; The step of judging whether the historical data in the database table has changed according to the verification results corresponding to all the divided periods, and determining the database and table division method of the target database according to the judgment result, includes: In any two adjacent divided periods, if the difference between the verification results corresponding to any pair of historical data with the same group number is greater than or equal to a preset value, it is determined that the historical data in the database table has changed, and the database and table sharding method of the target database is determined to be the first database and table sharding method, which performs database and table sharding based on the hash value of the unique key of the database table; If in each pair of adjacent divided periods, the differences between the verification results corresponding to the historical data with the same group number are all less than the preset value, it is determined that the historical data in the database table has not changed, and it is determined that the library and table sharding method of the target database is the second library and table sharding method, which performs library and table sharding based on the range of the unique key of the database table.

2. The method according to claim 1, characterized in that For each divided period, each set of historical data corresponding to the divided period is verified to obtain a corresponding verification result, including: For each divided period, a cyclic redundancy check (CRC) is performed on each group of historical data corresponding to the divided period to obtain a corresponding CRC check result.

3. The method according to claim 1, characterized in that The historical data in the database table is periodically divided into segments to obtain multiple groups of historical data corresponding to different segmentation periods, including: According to the unique key of the database table, the historical data in the database table is periodically segmented to obtain multiple groups of historical data corresponding to different segmentation periods.

4. The method according to claim 1, characterized in that The method uses the historical data volume of the database table in the target database collected in advance, combined with the pre-built data prediction model, to predict the total data volume in the target database, and obtains the total data volume change trend of the target database in the future time period, including: By using the historical data volume of the database table in the target database collected in advance, combined with the data prediction model pre-built based on the weighted average algorithm or regression analysis algorithm, the total data volume in the target database is predicted to obtain the total data volume change trend of the target database in the future time period.

5. The method according to claim 1, characterized in that After determining the database and table sharding strategy of the target database according to the peak time point and the database and table sharding method, the method further includes: Determine the warning time point according to the peak time point and the preset reminder time requirement; The database and table sharding strategy is sent to the target object at the warning time point.

6. A device for determining a database partitioning strategy, characterized in that: The device comprises: A prediction unit, used to use the historical data volume of the database table in the target database collected in advance, combined with a pre-built data prediction model, to predict the total data volume in the target database, and obtain a change trend of the total data volume of the target database in a future time period; A first determining unit, configured to determine a peak time point at which the total data volume of the target database reaches a threshold value according to a change trend of the total data volume; A division unit, used for periodically dividing the historical data in the database table into segments to obtain multiple groups of historical data corresponding to different division periods, each of the division periods corresponding to multiple groups of historical data; A verification unit, configured to verify each set of historical data corresponding to each divided period, and obtain a corresponding verification result; A processing unit, configured to determine whether the historical data in the database table has changed according to the verification results corresponding to all the divided periods, and determine the database and table division method of the target database according to the determination result; A second determination unit is used to determine a library and table sharding strategy of the target database according to the peak time point and the library and table sharding method; The processing unit comprises: A first determination module is used to determine that the historical data in the database table has changed if the difference between the verification results corresponding to any pair of historical data with the same group number is greater than or equal to a preset value in any two adjacent divided periods, and to determine that the library and table sharding method of the target database is a first library and table sharding method, which performs library and table sharding based on a hash value of a unique key of the database table; The second determination module is used to determine that the historical data in the database table has not changed if the difference between the verification results corresponding to the historical data with the same group number in each pair of adjacent divided periods is less than the preset value, and to determine that the library and table sharding method of the target database is the second library and table sharding method, and the second library and table sharding method performs library and table sharding based on the range of the unique key of the database table.

7. The device according to claim 6, characterized in that The verification unit is specifically used for: for each divided period, performing a cyclic redundancy check (CRC) on each group of historical data corresponding to the divided period to obtain a corresponding CRC verification result.

8. The device according to claim 6, characterized in that The division unit is specifically used to periodically divide the historical data in the database table into segments according to the unique key of the database table to obtain multiple groups of historical data corresponding to different division periods.

Images