Method for dynamically partitioning relational cluster database based on spatio-temporal data

Abstract

The invention relates to the technical field of data processing, in particular to a relation cluster database dynamic partitioning method based on spatio-temporal data. The method comprises the steps of 1, performing multi-dimensional decomposition on spatio-temporal data records in a target relation cluster database, and constructing a feature matrix; calculating a load metric of each potential partition based on the feature matrix; 2, calculating a balance index based on the load measurement of each potential partition; calculating a partition recombination trigger threshold based on the balance index; and 3, judging whether partition recombination needs to be carried out or not by combining the partition recombination trigger threshold value and the data migration cost, and if yes, calculating a new partition boundary of each potential partition to complete partition recombination. The dynamic partitioning method not only can realize accurate load balancing, but also can ensure the reasonability of partition adjustment at the lowest migration cost.

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and in particular relates to a dynamic partitioning method for a relational cluster database based on spatiotemporal data. Background Art

[0002] In the field of modern database management, with the rapid growth of data volume and data complexity, how to effectively manage and distribute data load, especially dynamically adjust database partitions to adapt to real-time changing load requirements, has become a key technical problem. Spatiotemporal data is highly dynamic and complex, and traditional static partitioning methods are often difficult to effectively cope with this complexity. Therefore, dynamic partitioning methods based on load adaptation have gradually become a research hotspot in the database field. However, although existing technologies have made certain progress in partition management, existing solutions still face many challenges in practical applications and are difficult to meet the needs of efficient and stable database management.

[0003] In the early days of database partition management, traditional static partitioning technology was widely used. This method partitioned the data set in advance and fixed the partition structure after the partitions were generated, but failed to dynamically adjust to the load changes. The implementation of static partitioning technology mainly relies on predefined rules of data, such as partitioning based on geographic location, timestamp, etc. These methods are simple and easy to implement, and can maintain a certain efficiency in scenarios where the data load is relatively stable. However, for data sets with strong dynamic characteristics and complexity, such as spatiotemporal data, the limitations of static partitioning gradually emerge. When the data load fluctuates or the amount of data increases significantly, the static partitioning scheme is difficult to adapt to the real-time load changes, resulting in some partitions being overloaded while other partitions are underloaded, resulting in resource waste and performance bottlenecks. Therefore, the static partitioning method cannot meet the requirements of modern databases in terms of dynamic load distribution and complex data management. Especially in scenarios with high concurrent access or real-time processing, static partitioning will significantly reduce the system's response speed and load balancing capabilities. Summary of the invention

[0004] The main purpose of the present invention is to provide a dynamic partitioning method for a relational cluster database based on spatiotemporal data. The dynamic partitioning method of the present invention can not only achieve accurate load balancing, but also ensure the rationality of partition adjustment at the lowest migration cost. The method fully considers the load demand, resource status and load change trend of the system, so that the database system can avoid unnecessary resource consumption while maintaining efficient load management, thereby achieving optimal performance in a complex data environment.

[0005] In order to solve the above problems, the technical solution of the present invention is achieved as follows: A dynamic partitioning method for a relational cluster database based on spatiotemporal data, the method comprising: Step 1: Perform multi-dimensional decomposition of spatiotemporal data records in the target relational cluster database to construct a feature matrix; based on the feature matrix, calculate the load metric of each potential partition; Step 2: Based on the load measurement of each potential partition, calculate the balance index; based on the balance index, calculate the partition reorganization trigger threshold; Step 3: Determine whether partition reorganization is needed based on the partition reorganization trigger threshold and data migration cost. If so, calculate the new partition boundary of each potential partition to complete the partition reorganization.

[0006] Furthermore, in step 1, the feature matrix is ​​constructed using the following formula: : ; in, is the number of spatiotemporal data records in the target relational cluster database; is an integer subscript index; Indicates The timestamp of the spatiotemporal data record; For the The rate of change of the spatiotemporal data records; The average rate of change of spatiotemporal data records; is the absolute value operator; is the lower boundary of the time window; is the upper boundary of the time window; is the standard deviation of the rate of change of spatiotemporal data records within the time window; is the time decay function; is the time series entropy function.

[0007] Furthermore, the time decay function is an exponential decay function, a reciprocal decay function or a linear decay function; the time series entropy function is the Shannon entropy function, the autocorrelation-based entropy function, or the spectral entropy function.

[0008] Furthermore, the load metric for each potential partition is calculated using the following formula: ; in, Indicates Potential partitions Load measurement; Represents potential partitions All spatiotemporal data records within Perform summation; Indicates potential partition All spatiotemporal data records within The feature matrix of Represents potential partitions along The border , for spatial position Space vector field Perform an integral; space vector field Indicates the position in space The spatiotemporal data records the flow direction and flow magnitude; is a small vector element on the boundary, representing a small increment of the boundary position during integration, used to partition along the potential Space vector field integrals on boundaries; For potential partition The volume of The extent of the space occupied; For potential partition The number of spatiotemporal data records within; For potential partition Neidi The number of I / O operations recorded in a spatiotemporal data record; For potential partition CPU utilization of spatiotemporal data records.

[0009] Furthermore, the balance index is calculated by the following formula: ; in, time Balance index at ; is the total number of potential partitions; is an integer subscript index; Represents potential partitions With potential partition The shortest distance; For potential partition Volume; is the global average load.

[0010] Furthermore, in step 2, the partition reorganization trigger threshold is calculated based on the balance index using the following formula: ; in, Represents potential partitions Memory usage of Represents potential partitions Disk usage.

[0011] Furthermore, the cost of data migration , calculated by the following formula: ; in, Indicates The spatiotemporal data records and The distance between spatiotemporal data records; For the Potential partitions The gradient of the load metric; is the average number of data records that need to be migrated based on historical data; For time The network bandwidth at that time.

[0012] Furthermore, in step 3, the following formula is used to determine whether partition reorganization is required in combination with the partition reorganization trigger threshold and the data migration cost: ; in, Represents potential partitions Query rate; Represents potential partitions capacity; when When it is 1, it is determined that partition reorganization is required.

[0013] Furthermore, the new partition boundary of each potential partition is calculated by the following formula: : ; in, For potential partition In time The new partition boundaries at that time.

[0014] The dynamic partitioning method of relational cluster database based on spatiotemporal data of the present invention has the following beneficial effects: The beneficial effects of the present invention are reflected in its precise load balancing capabilities. By constructing a feature matrix, the characteristics of the change rate, spatial position and time change of spatiotemporal data records are comprehensively characterized, making the evaluation of partition load more precise and accurate. Specifically, the feature matrix not only retains basic information such as the timestamp, change rate and spatial position of each data record, but also quantifies the dynamic changes of spatiotemporal data in multiple dimensions by introducing time decay functions, time series entropy functions, etc. This method can identify high-frequency data change areas and load distribution areas with higher complexity, providing data support for partition reorganization. Compared with traditional static or single-dimensional dynamic partitioning methods, the load measurement mechanism of the present invention can capture load dynamics more accurately, help to achieve accurate load balance between partitions, and reduce resource waste and performance bottlenecks.

[0015] Secondly, the present invention has significant advantages in controlling the cost of data migration. By calculating the multi-factor cost of data migration, the present invention avoids the high migration cost caused by frequent reorganization and improves the efficiency of resource utilization. When calculating the data migration cost, the present invention introduces factors such as data record distance, load gradient, partition boundary load density and network bandwidth to comprehensively consider the multiple overheads in the migration process. For example, when the load density at the partition boundary is large or the distance between partitions is close, the migration cost is relatively low, and the system can achieve partition reorganization at a lower resource cost. If the network bandwidth is limited or the migration cost between partitions is too high, the system automatically delays the reorganization operation. This dynamic migration cost control strategy enables the system to flexibly adjust the migration cost according to actual conditions, avoiding unnecessary data transmission, thereby improving the overall efficiency and performance stability of the database system.

[0016] In addition, the present invention realizes intelligent control of dynamic load monitoring and partition reorganization triggering through the setting of balance index and trigger threshold. As a measure of load balance, the balance index can evaluate the load balance in real time by calculating the spatial position of the partition load, the partition volume and its change trend. The trigger threshold is dynamically set according to the load change trend and the tolerance of the system, so that the trigger condition of partition reorganization no longer depends on the fixed threshold setting. When the load imbalance is small, the trigger threshold is high, and the system will choose to maintain the existing partition state and reduce the frequency of partition reorganization, thereby saving resources; when the load imbalance is significant, the trigger threshold is lowered, and the system prioritizes partition reorganization to achieve load balance. Such dynamic monitoring and intelligent triggering mechanisms effectively improve the flexibility and adaptability of partition management, enabling the system to operate efficiently in scenarios with frequent load changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a method flow of a dynamic partitioning method for a relational cluster database based on spatiotemporal data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0019] Example 1, reference Figure 1 : A dynamic partitioning method for a relational cluster database based on spatiotemporal data, the method comprising: Step 1: Perform multi-dimensional decomposition of spatiotemporal data records in the target relational cluster database to construct a feature matrix; based on the feature matrix, calculate the load metric of each potential partition; Through the multi-dimensional decomposition of spatiotemporal data, the present invention can capture various attributes and characteristics of spatiotemporal data, including but not limited to the time, geographical location, data density, data access frequency, etc. of data records. These characteristics can provide a comprehensive description of spatiotemporal data so that multiple factors can be comprehensively considered when partitioning the database. Specifically, the spatiotemporal data records in the database often contain complex and multi-dimensional information. Direct dynamic partitioning based on these raw data not only requires large computational complexity, but also makes it difficult to ensure the load balance of the partitions. In order to simplify this process and ensure the efficient operation of the database system after partitioning, it is first necessary to decompose these complex data in multiple dimensions, extract the important characteristic parameters that affect the load, and convert these characteristics into a matrix representation, so as to construct a characteristic matrix suitable for partitioning operations. By representing the multi-dimensional data features in the form of a characteristic matrix, not only can the computational complexity be effectively reduced, but also the core features that have the greatest impact on the load measurement can be extracted from it to achieve more accurate partition calculations.

[0020] The construction method of the feature matrix enables various features of spatiotemporal data to be reflected in the row and column structure of the matrix, so that on the basis of the matrix, the distribution characteristics, access patterns and load impact of spatiotemporal data can be effectively quantified and analyzed. This quantitative method allows each data record in the database to have a unique "fingerprint" in the feature matrix. This fingerprint is obtained by multi-dimensional decomposition of spatiotemporal data, which can not only reflect the spatiotemporal position of the data record, but also reflect its data load attributes and access behavior. For example, the time dimension of a spatiotemporal data record may be reflected as the time point of data generation, the spatial dimension is reflected as the geographic coordinates or the geographic coverage of the data, and the access frequency dimension reflects the frequency of use of the data record. Through multi-dimensional decomposition, each spatiotemporal data record is simplified into a vector, and this vector contains key attributes closely related to load measurement. Next, by arranging these vectors in rows and generating a feature matrix composed of the feature vectors of each data record, the load characteristics of spatiotemporal data in different partitions can be systematically displayed in the form of a matrix. The present invention not only realizes the effective management of spatiotemporal data through such a feature matrix construction method, but also provides a basis for subsequent dynamic partitioning operations. Each element of the feature matrix contains the performance value of a specific data record on a certain feature dimension, and these performance values ​​will play an important role in the subsequent load measurement. Each row of the matrix corresponds to a record in the database, and each column corresponds to a feature dimension. This method allows the spatiotemporal data of the database to be simplified into a structured representation. Based on the feature matrix, the present invention can easily and quickly calculate the load conditions of each potential partition in the database, thereby saving a lot of time in the subsequent dynamic partition calculation process, and can also more accurately grasp the balance of partition loads.

[0021] Step 2: Based on the load measurement of each potential partition, calculate the balance index; based on the balance index, calculate the partition reorganization trigger threshold; Step 2 calculates the load metric of each potential partition based on the feature matrix, and then calculates the balance index reflecting the load balance of each partition through the analysis of the load metric, and determines the trigger threshold of the partition reorganization accordingly. The core idea of ​​this process is to quantify the load situation for each partition of the database to ensure that the system can respond sensitively to the dynamic changes of spatiotemporal data and balance the overall load distribution of the database. The purpose of calculating the load metric is to quantify the load concentration and uniformity according to the usage of each partition in the database, and provide a reliable basis for the dynamic adjustment of the partition. Traditional database partitioning methods are difficult to effectively adapt to high-frequency load changes, and the present invention solves this problem through the refined measurement of spatiotemporal data and the analysis of the balance index based on the load, ensuring that the partition can always maintain a reasonable balance state in the load fluctuation. In this process, the present invention first regards the load metric as the key performance indicator of each partition in the database. The load metric not only covers the storage quantity of data in the partition, but also involves factors such as the access frequency and update rate of the data, thereby fully reflecting the actual usage of each partition. Through the analysis of the feature matrix, the present invention can accurately quantify the load metric of each partition, thereby laying the foundation for the subsequent balance index calculation. The calculated load metric is not only used for load monitoring of the current partition, but also provides a reference for load balancing of the entire system. Through the systematic calculation of load metrics, the system can gain in-depth understanding of the load distribution of each partition in the database, and use this information to determine whether the partition structure of the database needs to be adjusted. The calculation of load metrics ensures that the database can identify the load differences between partitions, so that the reorganization of partitions can be performed in a more accurate manner, thereby effectively avoiding performance problems caused by uneven load in the database system.

[0022] Next, based on the load measurement of each partition, the present invention further calculates the balance index to reflect the load distribution balance of the database in the global scope. The design purpose of the balance index is to comprehensively analyze the current partition load situation so that the system can evaluate the load balance of the database. The smaller the value of the balance index, the more uniform the load distribution, and the larger the value, the greater the load imbalance. Through this numerical analysis, the system can more accurately determine whether the load is concentrated in certain specific partitions, and formulate corresponding load balancing strategies accordingly. Traditional static partition strategies are often difficult to maintain load balance when the load changes significantly, and the present invention solves this problem through dynamic analysis of the balance index. While reflecting the load situation of each partition, the balance index also provides data support for the trigger threshold of partition reorganization. Finally, based on the balance index, the present invention sets the trigger threshold of partition reorganization as an important basis for judging whether to adjust the partition. Through quantitative analysis of the load and dynamic monitoring of the balance index, the system can start partition reorganization in time when the load imbalance reaches a certain level, thereby avoiding performance bottlenecks in the database under high load conditions. The design of the trigger threshold makes partition reorganization no longer rely on fixed time intervals, but automatically judges according to the real-time load situation. This innovation enables the database to respond more flexibly when the load changes. The setting of the trigger threshold combines the analysis results of load measurement and balance index, ensuring that the timing of partition reorganization is more reasonable and achieving the best load balancing effect with the lowest data migration cost.

[0023] Step 3: Determine whether partition reorganization is needed based on the partition reorganization trigger threshold and data migration cost. If so, calculate the new partition boundary of each potential partition to complete the partition reorganization.

[0024] Step 3 combines the partition reorganization trigger threshold of the balance index and the data migration cost to determine whether partition reorganization is needed, and calculates the new partition boundary if the conditions are met, thereby completing the partition reorganization operation. This process plays an important role in the entire dynamic partitioning method because it is directly related to the load balance of the database and the continuous improvement of performance. The partitioning method of traditional databases is often fixed at a predefined time interval or manually adjusted, and lacks real-time performance to cope with dynamic changes in load. The present invention ensures that the database can quickly respond to reorganization when the load is unbalanced by dynamically calculating the trigger threshold of partition reorganization and evaluating the data migration cost, while avoiding unnecessary partition reorganization when the system load is relatively balanced. This mechanism effectively reduces the additional overhead generated by the system during frequent partition reorganization, allowing the database to complete self-adjustment of load distribution in a more efficient manner, thereby ensuring the smooth operation of the overall system. When judging whether a partition needs to be reorganized, the present invention accurately controls the necessity of partition adjustment by combining the balance index and the trigger threshold. The balance index reflects the distribution balance of the current load among the partitions, and the trigger threshold represents the load imbalance range that the system can accept. If the balance index exceeds the trigger threshold, it means that the load between partitions has exceeded the tolerance range of the system. At this time, partition reorganization is required to ensure that the database load remains within a reasonable range. In this step, the present invention innovatively introduces the consideration of data migration cost. Data migration cost refers to the computing and storage overhead caused by data redistribution during partition reorganization. If the degree of load imbalance is not large, but the migration cost of the reorganization process is higher than the current bearing capacity of the system, the system will choose not to reorganize the partitions for the time being, and make corresponding adjustments when the load imbalance is further aggravated. This judgment method that combines the load balance situation with the migration cost enables the present invention to scientifically evaluate and optimize partition reorganization according to the actual load situation, reducing the waste of system resources caused by frequent adjustments in traditional methods.

[0025] When the system determines that partition reorganization needs to be performed, the next key step is to calculate the new partition boundary of each potential partition to achieve the purpose of redistributing the load. In this process, the present invention uses the previously constructed feature matrix to determine the load boundary of each partition, and analyzes and optimizes the distribution of the feature information of each partition to ensure that the load of each partition after reorganization tends to be balanced. The calculation of the new partition boundary takes into account a variety of factors, including the position of each data record in the feature matrix, the access frequency and the update rate. These factors together constitute the redistribution rule of the partition boundary, so that each partition can better cope with future load requirements after data reorganization and reduce the possibility of load imbalance again. This boundary calculation method not only ensures the balanced distribution of the load of each partition, but also minimizes the cost of data migration, making the entire partition reorganization process more efficient. After the partition reorganization is completed, the new partition structure will be updated to the database system, thereby forming a new load distribution pattern. On this basis, the system will continue to monitor the load changes of each partition, and dynamically determine whether further partition adjustment is needed through real-time comparison of the balance index with the trigger threshold. This continuous monitoring and reorganization process enables the database to adaptively distribute the load, ensuring that the system is always in an efficient operating state when facing high-frequency data access or load fluctuations. Compared with traditional methods, the dynamic partitioning method of the present invention not only improves the system's response speed to load changes, but also makes the partition reorganization process more intelligent and cost-effective by optimizing the control of data migration costs. This automated load balancing strategy ensures the stability of system performance while significantly reducing the need for manual intervention.

[0026] Example 2: In step 1, use the following formula to construct the feature matrix : ; in, is the number of spatiotemporal data records in the target relational cluster database; is an integer subscript index; Indicates The timestamp of the spatiotemporal data record; For the The rate of change of the spatiotemporal data records; The average rate of change of spatiotemporal data records; is the absolute value operator; is the lower boundary of the time window; is the upper boundary of the time window; is the standard deviation of the rate of change of spatiotemporal data records within the time window; is the time decay function; is the time series entropy function.

[0027] Specifically, one of the core parameters in the formula is , that is, the rate of change of each spatiotemporal data record. It indicates the change range of data records over time, reflecting the activity or volatility of the data in the current time window. , the average rate of change, provides a global reference that allows us to The difference is used to measure the deviation of the change amplitude of each record from the overall change amplitude. Here, the operation of taking the absolute value is to avoid the positive and negative values ​​of the change amplitude offsetting each other, so as to more intuitively show the actual change amplitude of the data record. By incorporating the deviation of the change rate of each record relative to the average change rate into the calculation, the formula can identify spatiotemporal data records that deviate from the normal in load distribution, which often have a significant impact on database partitioning. Due to the dynamic variability of the load, differential analysis can be used to identify high-rate data that have a key impact on load distribution, thereby providing a basis for partition optimization. Time window boundary and The introduction of adds time constraints to the feature matrix, making the partition analysis more flexible and accurate. Different from the traditional global analysis method, the present invention realizes the local time analysis of spatiotemporal data by introducing the time window. In this way, we can focus more on the changing characteristics of data records in the current time window without being limited by the cumulative impact of past data. This method not only effectively reduces the computational complexity of the feature matrix, but also makes the load analysis closer to the actual situation. Especially in the case of large fluctuations in data distribution, the local analysis provided by the time window can better adapt to the dynamic changes of spatiotemporal data. The upper and lower boundaries of the window and It can also be adjusted dynamically according to actual needs, making the matrix construction more flexible and easier to adapt to different partitioning requirements.

[0028] at the same time, That is, the standard deviation of the rate of change of data within the time window is incorporated into the formula as a load adjustment factor. The existence of this parameter ensures that the formula can dynamically evaluate the degree of fluctuation of data changes within the time window. When the rate of change of data within the window fluctuates greatly, that is, the standard deviation is high, the sensitivity of the formula to data deviation will also increase. This design is intended to balance the system's tolerance for highly volatile data. By adjusting the standard deviation, the present invention can effectively avoid performance bottlenecks caused by excessive concentration or dispersion of loads, thereby achieving more refined load control in dynamic partitions. By adjusting the difference in the rate of change, the feature matrix not only reflects the load level of the data record, but also captures the impact of high volatility data on the overall load structure. In this way, the feature matrix has a natural amplification effect on high rate of change and volatility, which helps to identify more dynamic and influential data records in the load. In addition, the time decay function The introduction of further strengthens the consideration of time factors. The role of the decay function is to give lower weights to older data records and higher weights to data close to the current time. This design allows the feature matrix to focus more on the current state and reduce reliance on historical data. This adjustment of time weights ensures that in dynamic load distribution, the data in the current time window can have a more direct impact on the partitioning strategy, thereby better adapting to dynamically changing needs. Compared with traditional methods, the decay function not only reduces the impact of old data on system load judgment, but also provides a guarantee for the rapid response of the partitioning strategy. Therefore, through the time decay function, the feature matrix It is more flexible in structure and can adaptively adjust the load distribution model according to the weight changes in the time window. Finally, the time series entropy function It measures the time complexity of the data. The purpose of this parameter design is to reflect the uncertainty and nonlinear characteristics of the data in the time dimension. A high entropy value usually means that the time complexity of the data is high, that is, the data shows more complex fluctuations or nonlinear trends in the time dimension. This complexity often brings unpredictable load changes in load distribution. Therefore, by The formula effectively controls high-complexity data, so that its influence on load distribution in the feature matrix is ​​reasonably constrained. This adjustment ensures the accuracy of load calculation, avoids misjudgment due to extreme fluctuations in time complexity, and ensures the stability of the feature matrix and the rationality of the partitioning strategy. In the present invention, the application of time series entropy reflects a comprehensive consideration of the spatiotemporal characteristics of data, so that the partition load not only considers the amplitude of data changes, but also adds the judgment of time complexity, making the partitioning process more scientific and rigorous.

[0029] Example 3: Time decay function is an exponential decay function, a reciprocal decay function or a linear decay function; the time series entropy function is the Shannon entropy function, the autocorrelation-based entropy function, or the spectral entropy function.

[0030] Specifically, the time decay function In the present invention, it is designed as an exponential decay function, a reciprocal decay function or a linear decay function, aiming to give different weights to data in different time periods to reduce the impact of earlier data on the partition strategy. The rapid decay of the exponential function can effectively focus on the spatiotemporal data close to the current time. This method is suitable for situations where the load changes rapidly, because the exponential decay can quickly reduce the impact on the earlier data, making the current load characteristics more clearly visible. In applications with severe load fluctuations, exponential decay helps to improve the response speed of partition adjustment. The reciprocal decay function The weight of past data is reduced in a more moderate way, which is suitable for scenarios that are also sensitive to historical data. Inverse decay makes the weight of data decrease slowly over time, retaining a certain historical influence, thus having a more comprehensive influence on load distribution. Linear decay function The time decay function gives the data a uniformly decreasing weight over time, which is more suitable for situations where the load changes are relatively stable. The uniform weight distribution ensures a smooth transition of the partition strategy. These different forms of time decay functions allow the system to flexibly select according to data characteristics and load requirements, thereby achieving optimal partition load balancing. In addition, the time series entropy function The choice of also plays a key role in load analysis. Time series entropy measures the temporal complexity of data and reflects the uncertainty of data in the temporal dimension. Shannon entropy function is a classic entropy calculation method, which calculates the probability distribution of data distribution and obtains the average uncertainty of data. For irregular load changes, Shannon entropy can reveal the overall complexity of the data series, so that the system can regulate the weight of these data in the partition strategy and reduce the excessive impact of highly complex data on the load. The entropy function based on autocorrelation is different. It calculates entropy based on the temporal autocorrelation of data, so as to focus more on the regularity of data in time. This entropy value is suitable for scenarios where the load has periodic fluctuations. Through the autocorrelation entropy value, the system can identify and utilize the periodic characteristics of the load to help the partition remain stable in the repetitive load pattern. Finally, the spectral entropy function is an entropy calculation method based on the frequency domain, which is suitable for scenarios with high data change frequency. By analyzing the data spectrum, the spectral entropy function can reveal the characteristics of high-frequency load changes, so that the system can make sensitive partition adjustments when the load changes rapidly. The selection of different entropy functions not only enriches the system's characterization of data complexity, but also enables the partitioning strategy to make the best response to different load patterns.

[0031] Embodiment 4: Calculate the load metric of each potential partition by the following formula: ; in, Indicates Potential partitions Load measurement; Represents potential partitions All spatiotemporal data records within Perform summation; Indicates potential partition All spatiotemporal data records within The feature matrix of Represents potential partitions along The border , for spatial position Space vector field Perform an integral; space vector field Indicates the position in space The spatiotemporal data records the flow direction and flow magnitude; is a small vector element on the boundary, representing a small increment of the boundary position during integration, used to partition along the potential Space vector field integrals on boundaries; For potential partition The volume of The extent of the space occupied; For potential partition The number of spatiotemporal data records within; For potential partition Neidi The number of I / O operations recorded in a spatiotemporal data record; For potential partition CPU utilization of spatiotemporal data records.

[0032] Specifically, the first key part in the formula is the rate of change term, namely , which represents the potential partition The rate of change of all spatiotemporal data in the feature matrix over time. For spatiotemporal data recording By taking the absolute value of the change rate of the constructed multidimensional feature quantity and summing up all the data records in the partition, we can obtain the overall dynamic change trend of the partition. The greater the change rate of the feature matrix, the more frequent the spatiotemporal data records in the partition change over time, and the higher the load. This dynamic measurement method enables the load calculation to respond to the changes in the data in the partition in a timely manner, adapt to the environment where spatiotemporal data frequently fluctuates, and provide real-time feedback for dynamic partitioning of the database. Another important part is the vector field integral term along the partition boundary. , which is used to evaluate the mobility of spatiotemporal data at the partition boundary. Here, the vector field Represents the spatial location at the partition boundary The integral is the sum of the contributions of the vector fields at all boundary locations. This term provides dynamic information about the spatial distribution of data for the load metric, especially the flow at the boundary reflects the trend of data flowing into or out of the partition. If the integral value is high, it means that the data flows frequently at the boundary, indicating that the frequency of data exchange within and outside the partition is high and the load is heavy. By integrating the spatial flow on the boundary, the formula effectively captures the spatial flow characteristics of the data, so that the load metric can take into account the dynamic changes in space and time. In order to obtain the density of the load metric, the formula introduces the volume term , that is, partition The spatial range occupied by the partition. By normalizing the rate of change of the feature matrix and the boundary fluidity to the partition volume, the formula calculates the load intensity per unit space. In this way, partitions of different sizes can be compared by load density to evaluate their load distribution. This spatial normalization operation makes the load measurement universal. Regardless of the size of the partition, a load density index can be obtained, which helps to make subsequent partition adjustments scientific. The product part of the formula It further integrates the ratio of the number of I / O operations and CPU utilization of each spatiotemporal data record. Indicates The number of I / O operations of a spatiotemporal data record, that is, the access and storage frequency of the record, reflects the pressure of data load on storage; is the CPU utilization of the record, reflecting the consumption of computing resources when processing the data. By calculating the ratio of I / O operations to CPU utilization, the formula can identify those I / O-intensive or CPU-intensive spatiotemporal data records, thereby providing a more refined basis for partition load measurement. By comparing the number of I / O operations and CPU utilization, the partition load can take into account the dual needs of storage and computing resources. Finally, this ratio is multiplied to ensure that the impact of the I / O and CPU states of all spatiotemporal data records in the formula on the load measurement is fully considered, especially when there is a large amount of I / O-intensive or CPU-intensive data in a partition, the multiplication term can amplify its impact on the overall load, so that the system can give priority to these high-load partitions during dynamic partitioning.

[0033] Example 5: The balance index is calculated by the following formula: ; in, time Balance index at ; is the total number of potential partitions; is an integer subscript index; Represents potential partitions With potential partition The shortest distance; For potential partition Volume; is the global average load.

[0034] Specifically, the key part of the formula is , that is, the load of each partition With the global load average This ratio provides a relative reference for the load status of each partition, allowing the system to identify areas with large load deviations. When the load of a partition is significantly higher than the average, this ratio will be relatively large, making a greater contribution to the overall value of the balance index. The introduction of this relative load ratio ensures that the balance index can reflect the uneven distribution of loads among different partitions, providing a basis for judgment when the system is unbalanced. At the same time, by taking the global average load as a benchmark, the load ratio in the formula can sensitively track the overall load distribution, especially when the load deviation gradually accumulates. The system can promptly perceive the gradual accumulation of deviations and avoid further deterioration of the load imbalance. In addition, the formula The term is the second-order partial derivative of the partition load, which represents the load in the spatial position and time The purpose of this design is to identify partitions with more drastic load fluctuations by capturing the load change trend over time and space. When the load of a partition changes significantly in time and space, the value of the second-order partial derivative will be higher, indicating that the partition may have an unstable load situation. The introduction of this item enables the balance index to dynamically reflect the instantaneous change characteristics of the load, especially when the load of a partition increases or decreases at an accelerated rate, the value of the second-order derivative will amplify this change trend, thereby occupying a more prominent position in the balance index. The introduction of this load change rate ensures the dynamic nature of load monitoring, allowing the system to adjust the partition structure in a timely manner according to the degree of load fluctuation to adapt to real-time changing load requirements.

[0035] It is worth noting that the formula also includes an exponential decay factor: , This is done by taking into account the spatial distance and volume factors between partitions to accurately control the load correlation between partitions. represents the shortest distance between two adjacent partitions, and and Represents the volume of the partition respectively. Through this attenuation factor, the formula effectively limits the impact of partitions that are far apart on each other's load calculation, and gives higher association weights to adjacent partitions with larger volumes. This design ensures that the load analysis is more targeted, that is, it focuses on partitions that are close in location and have relatively concentrated loads. When two partitions are close in distance and large in volume, the load connection between them will also be stronger, thereby enhancing their importance in the balance index calculation, while the load connection between partitions that are far away or small in volume will be weakened. This spatial attenuation mechanism enables the load distribution analysis to more reasonably reflect the actual relationship between partitions, ensuring the effectiveness and accuracy of the balance index in partition layout. Through these multi-level designs, the balance index The calculation results can provide a global perspective on load distribution, which not only covers the load status of each partition, but also takes into account the relative position between partitions and the load change trend. Compared with traditional load balancing evaluation methods, the balance index can more accurately reflect the complexity of dynamic load changes by comprehensively considering time and space factors. Especially in the case of uneven load distribution, it can provide timely early warning signals so that the system can trigger partition reorganization to achieve balanced load distribution. In this way, the calculation of the balance index becomes an important decision-making basis in database dynamic partition management. It not only ensures the balance of system load, but also improves the system's response speed to load changes, ensuring that the database can still operate efficiently under high load environments.

[0036] Embodiment 6: In step 2, the partition reorganization trigger threshold is calculated based on the balance index by the following formula: ; in, Represents potential partitions Memory usage of Represents potential partitions Disk usage.

[0037] Specifically, the main part of the formula is , that is, the balance index for all partitions The time rate of change of the balance index is summed up. This term reflects the rate of change of the load distribution between partitions. When the time rate of change of the balance index is high, it means that the load imbalance between partitions is increasing, and the necessity of triggering partition reorganization is also increasing. The time derivative of the balance index helps the system capture the changing trend of the load distribution at different times, so that the trigger threshold can more sensitively respond to load fluctuations. Through this term, the formula can identify the moments when the load changes greatly, thereby providing a response mechanism based on real-time load fluctuations for partition reorganization. This design based on time derivatives provides a basis for dynamic adjustment of the trigger threshold, allowing the system to quickly make partition adjustment decisions when the load accelerates imbalance. The boundary integral term in the formula The load density of each partition on the boundary is further considered. The ratio of the rate of change of the characteristic matrix The load intensity of the partition per unit time and space is quantified, reflecting the distribution density of the load on the boundary. The significance of boundary integral is that the boundary is often the key location for data exchange and flow, and the measurement of boundary load density can effectively reveal the distribution characteristics of the partition load. When the load density at the boundary of a partition is high, the boundary load of this partition is more concentrated, indicating that the data flow between it and the adjacent partition is frequent, and the need to trigger partition reorganization is more urgent. Through the analysis of boundary load density, the formula can accurately identify the partitions that may cause partition reorganization in spatial position, providing a more scientific basis for the trigger threshold. Especially in the case of high load density, the boundary integral term of the partition can capture the key load change location, so that the system is more targeted when the partition is reorganized.

[0038] The logarithmic terms included in the formula It provides a storage resource balance measure for the trigger threshold, ensuring that partition reorganization takes into account the reasonable allocation of resources while taking into account the load distribution. Disk usage The ratio can reflect the partition's demand for storage resources. If the ratio of memory to disk usage is large, it means that the partition's memory demand is significantly higher than its disk demand. This may mean that the partition is accessed frequently and needs to read and write memory frequently, thereby increasing the partition's load pressure. By taking the logarithm of this ratio, the formula can smooth the imbalance between memory and disk usage within a reasonable range and effectively reflect the configuration status of storage resources in the trigger threshold. When the memory and disk usage ratio deviates significantly from the balance, the contribution of the logarithmic term in the formula will increase, thereby increasing the trigger threshold and accelerating the triggering of partition reorganization. This resource balance measure ensures that the trigger threshold can ensure the reasonable configuration of storage resources while monitoring the load, providing double protection for the partition optimization of the system. Finally, by summing all the above factors and calculating the partition number, Taking the average value, the formula obtains an overall trigger threshold . This averaging method ensures that the trigger threshold can reflect the global load distribution, not just the load status of individual partitions. Through this global calculation, the trigger threshold has good universality and stability, and can balance the load distribution characteristics and resource allocation of each partition. When the system's load distribution reaches or exceeds the threshold, it means that the load imbalance between partitions has exceeded the system's tolerance range, triggering partition reorganization to rebalance the load distribution. This formula is not only flexible in the design of the trigger threshold, but also improves the accuracy of load balancing through multi-level considerations, making partition adjustment more intelligent and dynamic.

[0039] Example 7: Data Migration Cost , calculated by the following formula: ; in, Indicates The spatiotemporal data records and The distance between spatiotemporal data records; For the Potential partitions The gradient of the load metric; is the average number of data records that need to be migrated based on historical data; For time The network bandwidth at that time.

[0040] Specifically, the first core of the formula is This one, Represents the partition The load, is the global average load. The ratio of each partition load to the global average load can clearly reflect the degree of load deviation of each partition. The introduction of this deviation value helps to identify partitions that are relatively heavy or light in the load distribution, and thus provides a basis for the overall calculation of the balance index. This item in the formula assumes the function of introducing the microscopic characteristics of the load distribution within the partition into the global evaluation system. In this way, the system can identify partitions whose loads deviate significantly from the average value, that is, those partitions that have a greater impact on the system balance, so that these areas can be given priority in dynamic partition management. On the other hand, the formula also introduces the rate of change of partition load over time and space. , this item captures the acceleration of load changes in time and space through the second-order partial derivative of the partition load. Unlike the simple rate of change of the load, the design of the second-order derivative not only reflects the rate of change of the partition load, but also reveals the trend of load changes in time and space. When the acceleration of the load change is large, the value of the balance index will increase significantly, indicating that the load balance state between partitions is relatively unstable. When the system calculates the balance index, it can judge the load fluctuation trend through the change of this item, so as to identify the partitions with unbalanced load earlier. This mechanism makes the system more sensitive to load changes, especially in the case of accelerated load fluctuations. The design of the second-order derivative ensures that the system can capture unstable load states in a short time. In this way, the trigger mechanism of partition reorganization can respond to the dynamic changes of the load more flexibly, make effective adjustments before the load is unbalanced, and improve the stability and response speed of the database system.

[0041] In addition, the spatial attenuation term in the formula The balance index can be used to further improve the accuracy of load distribution by combining spatial relationships and partition volumes. Indicates adjacent partitions and The shortest distance between and are the volumes of the partitions, respectively. By calculating the combined effect of spatial distance and volume between partitions, the balance index can more accurately identify those high-load partitions that are closely connected in space. The design of this spatial attenuation term enables the system to give priority to the load correlation of adjacent partitions when calculating the balance index, thereby avoiding the situation where isolated load deviations in space are mistaken for overall system load imbalance. When two partitions are close and large in volume, the value of this term is close to 1, indicating that the two partitions are closely connected in space and have a greater impact on each other's load. Therefore, the balance index will be further adjusted according to the adjacent load distribution, so that the system can reasonably distribute the load in space. This spatial attenuation mechanism not only helps the system identify areas with severe load deviations, but also makes the balance index more accurate in actual load distribution analysis. For example, in some application scenarios where the distribution of spatiotemporal data is highly concentrated, there may be obvious load concentration in a specific area. Through the spatial attenuation term, the system can express this local load concentration as a high value in the balance index, thereby triggering the system's partition reorganization decision and alleviating the pressure of high-load partitions. This design not only ensures that the system can accurately identify key load areas when adjusting partitions, but also makes the load balancing calculation consistent with the actual distribution characteristics of spatiotemporal data, thus improving the load management efficiency of the system. By integrating multiple factors such as partition load deviation, load change rate, and spatial attenuation of adjacent partitions, a comprehensive and dynamic quantitative standard is provided for the load balancing of the database. Unlike traditional load balancing strategies, the balance index of the present invention can accurately reflect the load imbalance of the system before the load deviation accumulates to a certain extent through real-time monitoring and analysis of the load status. Especially in an environment where spatiotemporal data changes dynamically, the system can judge whether partition reorganization is needed through the instant feedback of the balance index to ensure the uniform distribution of the load. This real-time balance judgment mechanism enables the database system to quickly adapt to frequent load changes, avoiding performance bottlenecks caused by load concentration or uneven distribution. Compared with the static load balancing method, the balance index has significant advantages in dynamics, flexibility and accuracy, and can meet the high requirements for load management in large-scale and complex database applications. By introducing multi-level considerations such as load deviation, load acceleration and spatial relationship, the balance index formula of the present invention realizes multi-dimensional analysis from data distribution, change trend to spatial association in load monitoring, ensuring that the system can flexibly adjust the partition strategy to adapt to the dynamic fluctuations of the load.

[0042] Embodiment 8: In step 3, the following formula is used to determine whether partition reorganization is required in combination with the partition reorganization trigger threshold and the data migration cost: ; in, Represents potential partitions Query rate; Represents potential partitions capacity; when When it is 1, it is determined that partition reorganization is required.

[0043] Specifically, in the formula, the data migration cost and balance index Ratio It is the primary basis for judging partition reorganization. The data migration cost reflects the cost required to perform partition reorganization, while the balance index measures the degree of balance of system load. A smaller value means that the cost of performing partition reorganization is lower under the current load imbalance state, and the system can adjust the load with less resource overhead to achieve balance. When the ratio is higher, the cost of performing partition reorganization is relatively high, and the system may not be suitable for reorganization operations temporarily. The introduction of this ratio ensures resource optimization of partition reorganization operations, avoids frequent partition adjustments under high migration costs, and improves the efficiency of system resource utilization. The right side of the formula The item introduces the trigger threshold and the number of partitions. , the ratio of query rate to capacity, as the condition for partition reorganization. First, the trigger threshold As a global load balancing criterion, it is related to the number of partitions. Ratio This ensures that the system can trigger partition reorganization only when the global load exceeds the acceptable range. By normalizing the threshold to the number of partitions, this design allows the trigger standard to be applicable to situations with different numbers of partitions, and to maintain reasonable load balancing judgment regardless of the size of the database. In addition, through this normalization operation, the system can respond sensitively when the load imbalance reaches a certain level and adjust the partition strategy in time to avoid wasting resources. The product term on the right side of the formula In the partition query rate With capacity The ratio provides a relative indicator of the current access demand and capacity of the partition. The query rate reflects the access frequency of the partition, that is, the amount of queries that the partition needs to process; and the capacity indicates the maximum load that the partition can support. By multiplying and accumulating the ratio of query rate to capacity, the formula can intuitively reflect the relative access pressure of each partition under the current load conditions. If the query rate of some partitions is significantly higher than its capacity, the value of the product term will be large, indicating that the system's load demand has approached or exceeded its capacity limit, and this situation is also more likely to trigger a partition reorganization operation. When Less than When , the formula returns , indicating that the system needs to reorganize the partition. This conditional judgment mechanism ensures that the partition reorganization operation is triggered only when the migration cost is low and the system load is obviously unbalanced, so as to achieve load balancing at the lowest resource cost. On the contrary, when the condition is not met, the formula returns , indicating that partition reorganization will not be performed for the time being, thus avoiding unnecessary migration operations when the load is relatively balanced. This design ensures the resource efficiency of partition management, allowing the system to dynamically adapt to load requirements without frequently triggering partition adjustments.

[0044] Embodiment 9: Calculate the new partition boundary of each potential partition by the following formula: : ; in, For potential partition In time The new partition boundaries at that time.

[0045] Specifically, the starting term of the formula Represents a partition At the initial moment The boundary position of the partition is the initial reference point of the partition boundary. By making adjustments based on this, this formula allows the system to flexibly update the boundary while maintaining the original partition structure. This adjustment method based on the initial boundary makes the partition change more stable, ensuring that the boundary update will not change too much in a short period of time, which is beneficial to the long-term performance and consistency maintenance of the partition. The exponential decay term in the formula The balance index is used With trigger threshold The difference between the balance index and the trigger threshold, as well as the size of the partition volume, dynamically adjusts the intensity of the boundary update. When the difference between the balance index and the trigger threshold is large, it means that the current partition load deviates greatly from the ideal load balance state, and the system has a greater need to adjust the partition. In this case, the result of the exponential decay term is large, and the amplitude of the boundary update increases accordingly to quickly respond to the load imbalance. On the contrary, when the balance index is close to the trigger threshold, that is, the partition load is close to the balanced state, the exponential term tends to decrease, thereby slowing down the intensity of the boundary adjustment and ensuring the stability of the partition structure. In addition, the partition volume The introduction of makes this boundary adjustment process adaptable to partitions of different sizes. For larger partitions, the volume term will play an amplifying role in the denominator, suppressing the amplitude of boundary changes; while for smaller partitions, the boundary adjustment is more sensitive, ensuring that the system can respond quickly when the load difference is large. The boundary integral term in the formula It is for partition The cumulative component of the load change rate on the boundary of . Here, and Represent the rate of change of the feature matrix and load metric over time, respectively. The superposition of these two items reflects the dynamic change of load density on the partition boundary. By integrating the load change rate on the partition boundary, the formula can capture the difference in load density of partitions in spatial locations, especially the data mobility at the boundary. If the load density at a partition boundary changes dramatically, the boundary integral value will be large, indicating that the area needs more adjustment space, thereby promoting the expansion or contraction of the boundary. On the other hand, if the load change at the boundary is small, the integral result is also low, and the boundary update amplitude is small to ensure the stability of the partition structure. This measurement of boundary load changes ensures that the partition boundary can keep pace with the load dynamics and avoids data flow bottlenecks within and outside the partition caused by load fluctuations.

[0046] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic partitioning method for relational cluster database based on spatiotemporal data, characterized in that: The method comprises: Step 1: Perform multi-dimensional decomposition of spatiotemporal data records in the target relational cluster database to construct a feature matrix; based on the feature matrix, calculate the load metric of each potential partition; Step 2: Based on the load measurement of each potential partition, calculate the balance index; based on the balance index, calculate the partition reorganization trigger threshold; Step 3: Determine whether partition reorganization is needed based on the partition reorganization trigger threshold and data migration cost. If so, calculate the new partition boundary of each potential partition to complete the partition reorganization.

2. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 1, characterized in that: In step 1, use the following formula to construct the feature matrix : ； in, is the number of spatiotemporal data records in the target relational cluster database; is an integer subscript index; Indicates The timestamp of the spatiotemporal data record; For the The rate of change of the spatiotemporal data records; The average rate of change of spatiotemporal data records; is the absolute value operator; is the lower boundary of the time window; is the upper boundary of the time window; is the standard deviation of the rate of change of spatiotemporal data records within the time window; is the time decay function; is the time series entropy function.

3. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 2, characterized in that: Time decay function is an exponential decay function, a reciprocal decay function or a linear decay function; the time series entropy function is the Shannon entropy function, the autocorrelation-based entropy function, or the spectral entropy function.

4. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 3, characterized in that: The load metric for each potential partition is calculated using the following formula: ； in, Indicates Potential partitions Load measurement; Represents potential partitions All spatiotemporal data records within Perform summation; Indicates potential partition All spatiotemporal data records within The feature matrix of Represents potential partitions along The border , for spatial position Space vector field Perform an integral; space vector field Indicates the position in space The spatiotemporal data records the flow direction and flow magnitude; is a small vector element on the boundary, representing a small increment of the boundary position during integration, used to partition along the potential Space vector field integrals on boundaries; For potential partition The volume of The extent of the space occupied; For potential partition The number of spatiotemporal data records within; For potential partition Neidi The number of I / O operations recorded in a spatiotemporal data record; For the potential partition CPU utilization of spatiotemporal data records.

5. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 4, characterized in that: The balance index is calculated using the following formula: ； in, time Balance index at ; is the total number of potential partitions; is an integer subscript index; Represents potential partitions With potential partition The shortest distance; For potential partition Volume; is the global average load.

6. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 5, characterized in that: In step 2, the partition reorganization trigger threshold is calculated based on the balance index using the following formula: ； in, Represents potential partitions Memory usage of Represents potential partitions Disk usage.

7. The method for dynamic partitioning of a relational cluster database based on spatiotemporal data according to claim 6, characterized in that: Data migration cost , calculated by the following formula: ； in, Indicates The spatiotemporal data records and The distance between spatiotemporal data records; For the Potential Partitions The gradient of the load metric; is the average number of data records that need to be migrated based on historical data; For time The network bandwidth at that time.

8. The method for dynamic partitioning of a relational cluster database based on spatiotemporal data as claimed in claim 7, characterized in that: In step 3, the following formula is used to determine whether partition reorganization is needed, combined with the partition reorganization trigger threshold and the data migration cost: ； in, Represents potential partitions Query rate; Represents potential partitions capacity; when When it is 1, it is determined that partition reorganization is required.

9. The dynamic partitioning method of relational cluster database based on spatiotemporal data according to claim 8, characterized in that: The new partition boundaries of each potential partition are calculated using the following formula: : ； in, For potential partition In time The new partition boundaries at that time.

Images