Method and device for dynamically adjusting fragments

Abstract

The invention relates to a fragment dynamic adjustment method and device, and the method comprises the steps: obtaining a configuration data fragment set corresponding to a region node, wherein the configuration data fragment set comprises at least one configuration data fragment; for any configuration data fragment in the configuration data fragment set, configuration data stored in the configuration data fragment is determined, and the service priority of the configuration data is searched; and adjusting the configuration data fragments according to the query rate per second and the service priority of the configuration data fragments. By acquiring the configuration data fragment set of the regional node and dynamically adjusting the fragment granularity in combination with the per-second query rate of each configuration data fragment and the service priority of the corresponding configuration data, the dynamic change of the data access heat can be accurately adapted, the problem of fragment node overload caused by centralized access of hotspot data is effectively avoided, and the data access efficiency is improved. The hotspot data processing delay is obviously reduced, and the system response efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer processing technology, and in particular to a method and apparatus for dynamic adjustment of slices. Background Technology

[0002] With the popularization of distributed technology, large-scale distributed systems involving multi-regional deployments have been widely used in key fields such as finance and energy. Such systems rely heavily on the real-time synchronization, security, controllability, and continuous availability of configuration data. Efficient sharding management of configuration data has become a core element in ensuring the overall performance of the system.

[0003] Currently, the mainstream data sharding solutions in the industry mainly adopt static hash sharding or sharding by region. Both solutions have the characteristic of fixed sharding granularity, that is, the sharding rules are fixed after the system is deployed and cannot be dynamically adjusted according to the actual running status.

[0004] The aforementioned fixed sharding mechanism has significant limitations: it cannot adapt to the dynamic changes in data access popularity. When hot data is accessed in a concentrated manner, it can easily lead to overload of the corresponding sharding nodes, resulting in problems such as hot data processing delay and reduced system response efficiency. It is difficult to meet the high real-time and high availability requirements of large-scale distributed systems for configuration data. Summary of the Invention

[0005] To address the significant limitations of the aforementioned fixed sharding mechanism—its inability to adapt to dynamic changes in data access frequency—leading to overload of corresponding shard nodes during concentrated access to hot data, resulting in processing delays and decreased system response efficiency, and failing to meet the high real-time and high availability requirements of large-scale distributed systems for configuration data, this application provides a method, apparatus, electronic device, and storage medium for dynamic sharding adjustment. The specific technical solution is as follows: In a first aspect of this application, a method for dynamic sharding adjustment is provided, applicable to any regional node in a distributed system, the method comprising: Obtain the configuration data shard set corresponding to the region node, wherein the configuration data shard set contains at least one configuration data shard; For any configuration data shard in the configuration data shard set, determine the configuration data stored in the configuration data shard and find the service priority of the configuration data; The configuration data shards are adjusted based on the query rate per second of the configuration data shards and the service priority.

[0006] In an optional implementation, adjusting the configuration data shards based on the query rate per second of the configuration data shards and the service priority includes: Based on the query rate per second of the configured data shards and the service priority, the target shard granularity is determined, and the configured data shards are adjusted according to the target shard granularity.

[0007] In an optional implementation, determining the target shard granularity based on the query rate per second of the configured data shards and the service priority includes: Obtain the basic sharding granularity and the resource adaptation coefficient of the distributed system; The dynamic demand weighting factor is determined based on the query rate per second of the configured data shards and the business priority. The target sharding granularity is determined based on the basic sharding granularity, the dynamic demand weighting factor, and the distributed system resource adaptation coefficient.

[0008] In an optional implementation, determining the dynamic demand weighting factor based on the query rate per second of the configured data shards and the business priority includes: Determine the preset business priority coefficient corresponding to the business priority, and determine the logarithm of the query rate per second for the configured data shards; The dynamic demand weighting factor is obtained by adding the preset business priority coefficient to the logarithm of the query rate per second of the configured data shard.

[0009] In an optional implementation, determining the target sharding granularity based on the basic sharding granularity, the dynamic demand weighting factor, and the distributed system resource adaptation coefficient includes: Obtain the product between the basic sharding granularity and the dynamic demand weight factor, and divide the product by the distributed system resource adaptation coefficient to obtain the target sharding granularity.

[0010] In an optional implementation, for any configuration data fragment in the configuration data fragment set, the following processing is also performed: Determine whether the query rate per second is greater than a preset hotspot threshold, and whether the duration for which the query rate per second is greater than the preset hotspot threshold reaches a preset first duration; If the query rate per second is greater than a preset hotspot threshold and the duration of the query rate per second being greater than the preset hotspot threshold reaches a preset first duration, the configuration data shard is horizontally split to obtain multiple sub-configuration data shards. Establish a mapping relationship between the configuration data shard and the multiple sub-configuration data shards.

[0011] In an optional implementation, for any configuration data fragment in the configuration data fragment set, the following processing is also performed: Determine whether the query rate per second is less than a preset cold data threshold, and whether the duration for which the query rate per second is less than the preset cold data threshold reaches a preset second duration; If the query rate per second is less than a preset cold data threshold and the duration of the query rate per second being less than the preset cold data threshold reaches a preset second duration, the configuration data fragments are vertically merged to obtain merged configuration data fragments. Establish a mapping relationship between the configuration data shards and the merged configuration data shards.

[0012] In an optional implementation, the method further includes: Once a change in the configuration data is detected, the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data are determined. A holographic fingerprint of the modified configuration data is generated based on the data content hash, the operation trajectory hash, and the spatiotemporal hash.

[0013] In an optional implementation, determining the data content hash, operation trajectory hash, and spatiotemporal hash of the modified configuration data includes: The modified configuration data is hashed to obtain the data content hash. Determine the data change operation sequence corresponding to the modified configuration data, and perform hash processing on the data change operation sequence to obtain the operation trajectory hash; Obtain the fragment identifier of the configuration data fragment, the region identifier of the region node, and the current timestamp; The fragment identifier, the region identifier, and the current timestamp are hashed to obtain a spatiotemporal hash.

[0014] In an optional implementation, the method further includes: Receive a request from the client for the modified configuration data, and in response to the request, obtain the holographic fingerprint and the most recent N historical holographic fingerprints, where N is a positive integer; The holographic fingerprint and the most recent N historical holographic fingerprints are returned to the client so that the client can perform the following steps: Verify the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints; If the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints are verified, the modified configuration data is loaded.

[0015] In an optional implementation, the method further includes: Upon detecting a change in the configuration data, a configuration data change message is sent to the central area node, causing the central area node to perform the following steps: Receive the configuration data change message and broadcast it to the remaining regional nodes in the distributed system; Obtain the voting results of the remaining regional nodes, wherein the remaining regional nodes vote in response to the configuration data change message to generate the voting results; Based on the voting results, the configuration data change message will be synchronized to the remaining regional nodes.

[0016] In an optional implementation, synchronizing the configuration data change message to other regional nodes based on the voting results includes: If the voting result indicates agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight. If the agreed weight exceeds the preset weight, the configuration data change message will be synchronized to the remaining regional nodes.

[0017] In an optional implementation, the method further includes: Obtain special event markers, business cycle tags, and the historical access sequence of the configuration data; Based on the special event marker, the business cycle label, and the historical access sequence, predict the access probability and estimated query rate per second of the configuration data within a future preset third time period; Based on the access probability and the estimated query rate per second, a preheating strategy for the configuration data is determined, and the configuration data is preheated according to the preheating strategy.

[0018] In an optional implementation, determining the preheating strategy for the configuration data based on the access probability and the estimated query rate per second includes: When the access probability is within the first probability range and the estimated query rate per second is within the first query rate range, preheating to the local cache is determined as the preheating strategy for the configuration data. When the access probability is in the second probability range and the estimated query rate per second is in the second query rate range, preheating to the regional shared cache is determined as the preheating strategy for the configuration data. When the access probability is in the third probability interval and the estimated query rate per second is in the third query rate interval, the preheating metadata is determined as the preheating strategy for the configuration data.

[0019] In a second aspect of this application, a sharding dynamic adjustment device is also provided, applied to any regional node in a distributed system, the device comprising: The shard set acquisition module is used to acquire the configuration data shard set corresponding to the region node, wherein the configuration data shard set contains at least one configuration data shard. The priority determination module is used to determine the configuration data stored in any configuration data fragment in the configuration data fragment set, and to find the service priority of the configuration data. The sharding dynamic adjustment module is used to adjust the configuration data shards according to the query rate per second of the configuration data shards and the business priority.

[0020] In an optional implementation, the segmentation dynamic adjustment module specifically includes: The sharding granularity determination submodule is used to determine the target sharding granularity based on the query rate per second of the configured data sharding and the business priority. The sharding dynamic adjustment submodule is used to adjust the configuration data shards according to the target sharding granularity.

[0021] In an optional implementation, the granularity determination submodule specifically includes: Granularity and coefficient acquisition unit, used to obtain basic sharding granularity and distributed system resource adaptation coefficient; The factor determination unit is used to determine the dynamic demand weight factor based on the query rate per second of the configured data shards and the business priority. The sharding granularity determination unit is used to determine the target sharding granularity based on the basic sharding granularity, the dynamic demand weighting factor, and the distributed system resource adaptation coefficient.

[0022] In an optional implementation, the factor determination unit is specifically used for: Determine the preset business priority coefficient corresponding to the business priority, and determine the logarithm of the query rate per second for the configured data shards; The dynamic demand weighting factor is obtained by adding the preset business priority coefficient to the logarithm of the query rate per second of the configured data shard.

[0023] In an optional implementation, the granularity determination unit is specifically used for: Obtain the product between the basic sharding granularity and the dynamic demand weight factor, and divide the product by the distributed system resource adaptation coefficient to obtain the target sharding granularity.

[0024] In an optional implementation, the apparatus further includes a sharding module, configured to perform the following processing on any configuration data shard in the configuration data shard set: determining whether the query rate per second is greater than a preset hotspot threshold and whether the duration of the query rate per second being greater than the preset hotspot threshold reaches a preset first duration; if the query rate per second is greater than the preset hotspot threshold and the duration of the query rate per second being greater than the preset hotspot threshold reaches the preset first duration, horizontally splitting the configuration data shard to obtain multiple sub-configuration data shards; and establishing a mapping relationship between the configuration data shard and the multiple sub-configuration data shards.

[0025] In an optional implementation, the apparatus further includes a shard merging module, configured to perform the following processing on any configuration data shard in the configuration data shard set: determining whether the query rate per second is less than a preset cold data threshold and whether the duration of the query rate per second being less than the preset cold data threshold reaches a preset second duration; if the query rate per second is less than the preset cold data threshold and the duration of the query rate per second being less than the preset cold data threshold reaches the preset second duration, vertically merging the configuration data shard to obtain a merged configuration data shard; and establishing a mapping relationship between the configuration data shard and the merged configuration data shard.

[0026] In an optional implementation, the apparatus further includes: The hash determination module is used to determine the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data after detecting a change in the configuration data. The fingerprint generation module is used to generate a holographic fingerprint of the modified configuration data based on the data content hash, the operation trajectory hash, and the spatiotemporal hash.

[0027] In an optional implementation, the hash determination module is specifically used for: The modified configuration data is hashed to obtain the data content hash. Determine the data change operation sequence corresponding to the modified configuration data, and perform hash processing on the data change operation sequence to obtain the operation trajectory hash; Obtain the fragment identifier of the configuration data fragment, the region identifier of the region node, and the current timestamp; The fragment identifier, the region identifier, and the current timestamp are hashed to obtain a spatiotemporal hash.

[0028] In an optional implementation, the apparatus further includes: The fingerprint acquisition module is used to receive a request from the client for the changed configuration data, and in response to the request, acquire the holographic fingerprint and the most recent N historical holographic fingerprints, where N is a positive integer; The fingerprint return module is used to return the holographic fingerprint and the most recent N historical holographic fingerprints to the client, so that the client can perform the following steps: verify the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints; if the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints pass the verification, load the changed configuration data.

[0029] In an optional implementation, the apparatus further includes: The message sending module is used to send a configuration data change message to the central regional node after detecting a change in the configuration data, so that the central regional node performs the following steps: receiving the configuration data change message and broadcasting it to the other regional nodes in the distributed system; obtaining the voting results of the other regional nodes, wherein the other regional nodes respond to the configuration data change message by voting and generating the voting results; and synchronizing the configuration data change message to the other regional nodes according to the voting results.

[0030] In an optional implementation, the message sending module is specifically used for: If the voting result indicates agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight. If the agreed weight exceeds the preset weight, the configuration data change message will be synchronized to the remaining regional nodes.

[0031] In an optional implementation, the apparatus further includes: The sequence acquisition module is used to acquire special event markers, business cycle tags, and historical access sequences of the configuration data. The prediction module is used to predict the access probability and estimated query rate per second of the configuration data within a preset third time period in the future, based on the special event marker, the business cycle label, and the historical access sequence. The strategy determination module is used to determine the preheating strategy for the configuration data based on the access probability and the estimated query rate per second. The data preheating module is used to preheat the configuration data according to the preheating strategy.

[0032] In an optional implementation, the strategy determination module is specifically used for: When the access probability is within the first probability range and the estimated query rate per second is within the first query rate range, preheating to the local cache is determined as the preheating strategy for the configuration data. When the access probability is in the second probability range and the estimated query rate per second is in the second query rate range, preheating to the regional shared cache is determined as the preheating strategy for the configuration data. When the access probability is in the third probability interval and the estimated query rate per second is in the third query rate interval, the preheating metadata is determined as the preheating strategy for the configuration data.

[0033] In a third aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the slice dynamic adjustment methods described in the first aspect above.

[0034] In a fourth aspect of the embodiments of this application, a storage medium is also provided, wherein the storage medium stores instructions that, when run on a computer, cause the computer to execute any of the segmentation dynamic adjustment methods described in the first aspect above.

[0035] In a fifth aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above-described slice dynamic adjustment methods.

[0036] The technical solution provided in this application embodiment obtains a set of configuration data shards corresponding to a regional node. The set of configuration data shards contains at least one configuration data shard. For any configuration data shard in the set of configuration data shards, the configuration data stored in the configuration data shard is determined, and the service priority of the configuration data is found. Based on the query rate per second and the service priority of the configuration data shard, the configuration data shard is adjusted.

[0037] This technical solution breaks away from the rigid rules of traditional static hash sharding or regional sharding. By obtaining the configuration data shard set of regional nodes and dynamically adjusting the sharding granularity based on the query rate per second of each configuration data shard and the business priority of the corresponding configuration data, it can accurately adapt to the dynamic changes in data access popularity. This effectively avoids the problem of shard node overload caused by centralized access to hot data, significantly reduces the processing latency of hot data, improves system response efficiency, ensures real-time synchronization and continuous availability of configuration data, and ultimately meets the core requirements of high real-time performance and high availability of configuration data management in large-scale distributed systems, thereby optimizing the overall system performance and service stability. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0041] Figure 1 This is a schematic diagram illustrating the implementation process of a segmentation dynamic adjustment method in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the implementation process of another segmentation dynamic adjustment method shown in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the implementation process of a holographic fingerprint generation method in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the implementation process of a data synchronization method in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the implementation process of a data preheating method in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a segmented dynamic adjustment device shown in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0044] In this embodiment, the distributed system includes at least one regional node, and each regional node contains at least one configuration data shard. For example, the distributed system includes N (N is a positive integer) regional nodes (e.g., regional node in city A, regional node in city B), designated as regional node 1, 2, 3, ..., N. Each regional node contains several configuration data shards, which store corresponding server-side configuration data. For any configuration data shard of any regional node, the shard granularity (e.g., 1KB to 100MB) can be dynamically adjusted based on business importance and query rate per second.

[0045] Based on this, such as Figure 1 The diagram shown illustrates the implementation flow of a dynamic sharding adjustment method provided in this application embodiment. This method is applied to any regional node in the aforementioned distributed system and may specifically include the following steps: S101, obtain the set of configuration data shards corresponding to the regional node. The set of configuration data shards contains at least one configuration data shard.

[0046] In this embodiment, the regional node is first located, and then a complete list of all configuration data shards under that regional node is retrieved and obtained to form a corresponding configuration data shard set (for example, the configuration data shard set of a certain regional node includes two shards, "parameter configuration shard A" and "routing rule shard B"). This step is a prerequisite for subsequent dynamic adjustment of shard granularity. Only by clearly identifying which configuration data shards a certain regional node specifically manages can the query rate per second and business priority be analyzed for each configuration data shard, thereby enabling dynamic optimization of shard granularity.

[0047] In this context, a regional node refers to an independent service node in a distributed system, divided according to its geographical deployment location or business cluster. Each regional node is responsible for storing configuration data and processing business requests in its corresponding region. A configuration data shard is the smallest data unit after splitting the overall server-side configuration dataset (such as full permission configuration and routing rules) according to preset rules to achieve efficient management of configuration data. Each shard carries a portion of the configuration data. A configuration data shard set is the overall collection of all configuration data shards managed and associated by a certain regional node, and the set must contain at least one shard (small regional nodes may only manage one configuration data shard, while large nodes can manage multiple shards of different types).

[0048] S102, for any configuration data fragment in the configuration data fragment set, determine the configuration data stored in the configuration data fragment and find the service priority of the configuration data.

[0049] In this embodiment of the application, "any configuration data shard in the configuration data shard set" means that each configuration data shard in the acquired configuration data shard set of a certain region node is analyzed one by one, rather than some configuration data shards. This is to ensure the comprehensiveness of subsequent shard granularity adjustments and avoid local load imbalance or core business data management vulnerabilities caused by missing shards.

[0050] The core step in determining the configuration data stored in each configuration data shard is to accurately identify the specific configuration data content and type carried within each shard. Since configuration data shards are the units that divide the overall server-side configuration dataset, different shards correspond to configurations for different business scenarios (for example, "parameter configuration shard A" might store core parameters for a region's financial transaction business: payment timeout thresholds, risk control interface call frequency limits; "routing rule shard B" might store routing rules for user registration requests in a region: node allocation strategies for mobile / PC requests). The purpose of this step is to clarify the "data attributes" of the shards, avoiding a disconnect between subsequent priority judgments, load assessments, and business requirements due to ambiguous data types (for example, financial transaction parameters should not be confused with ordinary interface parameters).

[0051] The business priority of configuration data lookup refers to matching the priority level of configuration data within the current shard according to the system's predefined business priority rules (usually based on the importance, scope of impact, and Service Level Agreement (SLA) of the business supported by the configuration data, such as core priority, normal priority, and low priority). For example, configuration data supporting fund transactions and permission verification (such as payment permission configuration and risk control parameters) belongs to core priority; any anomalies will directly affect business continuity and fund security. Configuration data supporting personalized settings for ordinary users (such as interface theme parameters) belongs to normal priority. Configuration data supporting log archiving rules belongs to low priority. The system needs to extract the priority information of the current shard data from the preset priority mapping table or configuration data metadata tags. The key value of this step is to bind the technical "data sharding" with the business-level "importance," avoiding subsequent adjustments to sharding granularity based solely on load while neglecting the stability requirements of core businesses (for example, core priority configuration data should not be arbitrarily split to distribute load).

[0052] S103, adjust the configuration data shards according to the query rate per second and business priority of the configuration data shards.

[0053] In this embodiment, the query per second (QPS) of the configuration data shard is the core quantitative indicator for measuring the real-time load pressure of the shard, reflecting the number of access requests made by the business system to the configuration data stored in the shard per unit time (e.g., if the shard carrying payment permission configuration is called 500 times per second, the QPS is 500). The system collects the QPS data of each shard in real time through built-in monitoring modules (such as distributed tracing and node load statistics components). The higher the QPS, the greater the access pressure on the shard, and the more likely it is to become a "hot shard," making the computing and storage resources of the corresponding node more prone to overload; the lower the QPS, the lighter the shard load, and the lower the resource utilization may be.

[0054] The system adjusts the configured data shards by configuring the queries per second (QPS) and the corresponding business priority of the configured data. First, the target shard granularity G (e.g., 86.7MB, 112.5MB) is calculated based on the QPS and business priority of the configured data shards. The core adjustment action is then performed to adjust the configured data shards according to the target shard granularity. Specifically, the existing shard granularity (current size) of the configured data shard is directly adjusted to the target shard granularity G. That is, the value G is used as the final standard to ensure that the shard size precisely matches the target shard granularity, thereby achieving a dynamic balance between resource utilization and access efficiency.

[0055] Based on the above description of the technical solutions provided in the embodiments of this application, a set of configuration data shards corresponding to a regional node is obtained. The set of configuration data shards contains at least one configuration data shard. For any configuration data shard in the set of configuration data shards, the configuration data stored in the configuration data shard is determined, and the service priority of the configuration data is found. The configuration data shard is adjusted according to the query rate per second and the service priority of the configuration data shard.

[0056] This technical solution breaks away from the rigid rules of traditional static hash sharding or regional sharding. By obtaining the configuration data shard set of regional nodes and dynamically adjusting the sharding granularity based on the query rate per second of each configuration data shard and the business priority of the corresponding configuration data, it can accurately adapt to the dynamic changes in data access popularity. This effectively avoids the problem of shard node overload caused by centralized access to hot data, significantly reduces the processing latency of hot data, improves system response efficiency, ensures real-time synchronization and continuous availability of configuration data, and ultimately meets the core requirements of high real-time performance and high availability of configuration data management in large-scale distributed systems, thereby optimizing the overall system performance and service stability.

[0057] like Figure 2 The diagram illustrates the implementation flow of another dynamic sharding adjustment method provided in this application embodiment. This method is applied to any regional node in the aforementioned distributed system and may specifically include the following steps: S201, obtain the set of configuration data shards corresponding to the regional node. The set of configuration data shards contains at least one configuration data shard.

[0058] In this embodiment of the application, this step is similar to step S101 above, and will not be described in detail here.

[0059] S202, for any configuration data fragment in the configuration data fragment set, determine the configuration data stored in the configuration data fragment and find the service priority of the configuration data.

[0060] In this embodiment of the application, this step is similar to step S102 above, and will not be described in detail here.

[0061] S203, obtain the basic sharding granularity and the distributed system resource adaptation coefficient.

[0062] In this embodiment, the basic sharding granularity and the distributed system resource adaptation coefficient are obtained. The basic sharding granularity is the system-preset basic shard size, determined based on storage capacity, network bandwidth, etc., for example, a basic sharding granularity of 1MB. The distributed system resource adaptation coefficient is the system-preset cluster resource coefficient.

[0063] S204 determines the dynamic demand weighting factor based on the query rate per second of the configured data shards and the business priority.

[0064] In this embodiment of the application, the dynamic demand weighting factor can be determined based on the query rate per second of the configuration data shard and the service priority of the configuration data stored in the configuration data shard.

[0065] Specifically, the preset business priority coefficient corresponding to the business priority is determined, and the logarithm of the query rate per second of the configured data shard is determined. The preset business priority coefficient and the logarithm of the query rate per second of the configured data shard are added together to obtain the dynamic demand weight factor.

[0066] For example, business priorities are divided into four levels: P0, P1, P2, and P3, with the priority decreasing progressively, i.e., P0 has the highest priority and P3 has the lowest priority. The corresponding priorities are positively correlated with the preset business priority coefficients. For example, the preset business priority coefficients are P0=4, P1=3, P2=2, and P3=1. Based on this, the preset business priority coefficients corresponding to the business priorities are determined, and the logarithm of the query rate per second for the configured data shards is determined. The preset business priority coefficients are added to the logarithm of the query rate per second for the configured data shards to obtain the dynamic demand weight factor. The calculation formula is P+log(QPS), where P is the preset business priority coefficient and QPS is the query rate per second.

[0067] S205 determines the target sharding granularity based on the basic sharding granularity, dynamic demand weighting factor, and distributed system resource adaptation coefficient, and adjusts the configured data sharding according to the target sharding granularity.

[0068] In this embodiment of the application, after the dynamic demand weight factor is calculated through the above steps, the target sharding granularity can be determined based on the basic sharding granularity, the dynamic demand weight factor, and the distributed system resource adaptation coefficient, and the configuration data sharding can be adjusted according to the target sharding granularity.

[0069] The process involves obtaining the product between the base sharding granularity and the dynamic demand weighting factor, dividing the product by the distributed system resource adaptation coefficient, and then calculating the target sharding granularity. The formula is: target sharding granularity G = base_size × (P + log(QPS)) / K, where base_size is the base sharding granularity, P is the preset business priority coefficient, K is the distributed system resource adaptation coefficient, and QPS is the query rate per second.

[0070] Furthermore, for any configuration data shard in the configuration data shard set, its horizontal splitting, vertical merging, or remaining unchanged can be determined based on its query rate per second. Therefore, for any configuration data shard in the configuration data shard set, it is determined whether its query rate per second is greater than a preset hotspot threshold (e.g., 1000) and whether the duration of the query rate per second exceeding the preset hotspot threshold reaches a preset first duration (e.g., 5 seconds). If the query rate per second exceeds the preset hotspot threshold and the duration of the query rate per second exceeding the preset hotspot threshold reaches the preset first duration, the configuration data shard is horizontally split to obtain multiple sub-configuration data shards. A mapping relationship is established between the configuration data shard and the multiple sub-configuration data shards. This mapping relationship allows the system to know the shard structure after the horizontal splitting of the configuration data shards.

[0071] It should be noted that horizontally splitting the configuration data shards essentially means splitting them into 2^M (M is a positive integer) sub-configuration data shards according to the primary key hash. When synchronizing data between configuration data shards and sub-configuration data shards, the data version is marked by a spacetime stamp vector (containing region ID, physical clock, and logical counter) to avoid the clock drift problem of traditional single timestamps.

[0072] In addition, it can be determined whether the query rate per second of the configuration data shard is less than a preset cold data threshold (e.g., 10) and whether the duration of the query rate per second being less than the preset cold data threshold reaches a preset second duration (e.g., 24 hours). If the query rate per second is less than the preset cold data threshold and the duration of the query rate per second being less than the preset cold data threshold reaches the preset second duration, the configuration data shard is vertically merged to obtain a merged configuration data shard. A mapping relationship is established between the configuration data shard and the merged configuration data shard. This mapping relationship allows the system to know the shard structure after the vertical merging of the configuration data shards.

[0073] It's important to note that configuration data shards with consistently low QPS over a period of time are defined as cold shards. Vertical merging combines multiple cold shards into a larger cold shard, freeing up redundant storage and resources. For example, two low-priority, low-QPS shards can be merged into one, reducing the number of shards.

[0074] If the query rate per second for configured data shards does not meet the conditions for horizontal splitting and vertical merging mentioned above, the configured data shards can remain unchanged.

[0075] Furthermore, in this embodiment, configuration data has a corresponding fingerprint. Traditional fingerprints are generated only based on data content and cannot trace version change trajectories, making them vulnerable to replay attacks or tampering attacks. Therefore, this embodiment expands fingerprint generation from a single "content" dimension to three dimensions: "content, trajectory, and spatiotemporal." The "trajectory" dimension can trace version change trajectories, and the "spatiotemporal" dimension can prevent replay attacks, thereby improving anti-tampering capabilities.

[0076] Based on this, such as Figure 3 The diagram shown illustrates the implementation flow of a holographic fingerprint generation method provided in this application embodiment. This method is applied to any regional node in a distributed system and includes the following steps: S301, after detecting a change in configuration data, determines the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data.

[0077] In this embodiment of the application, for any configuration data fragment in the configuration data fragment set, the configuration data stored therein may change. After the change, it is necessary to generate a corresponding holographic fingerprint.

[0078] Therefore, when a change in configuration data is detected, the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data are determined. Subsequently, a holographic fingerprint of the changed configuration data is generated based on the data content hash, operation trajectory hash, and spatiotemporal hash.

[0079] This involves hashing the changed configuration data to obtain a data content hash. For example, the SM3 algorithm can be used to hash the changed configuration data to obtain a data content hash.

[0080] Determine the data change operation sequence corresponding to the changed configuration data, and perform hash processing on the data change operation sequence to obtain the operation trajectory hash. For example, if the data change operation sequence is "Create" → "Modify" → "Delete", convert the data change operation sequence into the string create_modify_delete, and then use the SM3 algorithm to perform hash processing to generate the operation trajectory hash.

[0081] Obtain the shard identifier of the configuration data shard, the region identifier of the region node, and the current timestamp. Perform hash processing on the shard identifier, region identifier, and current timestamp to obtain a spatiotemporal hash. For example, the region ID of the region node (AP-SH01), the shard ID of the configuration data shard (node-001), and the current timestamp (2025-12-03T10:00:00.123456789) are concatenated into the string AP-SH-01_20251203T100000.123456789_node-001, and then hashed using the SM3 algorithm to generate a spatiotemporal hash.

[0082] S302 generates a holographic fingerprint of the changed configuration data based on the data content hash, operation trajectory hash, and spatiotemporal hash.

[0083] In this application embodiment, the three dimensions of data content hash, operation trajectory hash and spatiotemporal hash obtained in the above steps can be used to generate a holographic fingerprint of the changed configuration data.

[0084] The three dimensions of data content hash, operation trajectory hash, and spatiotemporal hash obtained from the above steps can be used to generate a holographic fingerprint (e.g., a 256-bit holographic fingerprint) through a cascaded compression algorithm and stored in the blockchain evidence storage module.

[0085] The configuration data stored in the configuration data fragment may change, and the client may access it after the change. To address this, the system receives requests from the client for the changed configuration data and returns the current holographic fingerprint and the N most recent historical holographic fingerprints. The client verifies the "continuity" and "spatiotemporal rationality" of the fingerprint chain composed of the current holographic fingerprint and the N most recent historical holographic fingerprints to determine whether the changed configuration data is complete and has not been tampered with, and thus decides whether to load the changed configuration data.

[0086] Specifically, the system receives a request from the client for the modified configuration data, and in response to the request, obtains the holographic fingerprint and the most recent N historical holographic fingerprints, where N is a positive integer. The system then returns the holographic fingerprint and the most recent N historical holographic fingerprints to the client, enabling the client to perform the following steps: verifying the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints; and if the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints pass the verification, loading the modified configuration data.

[0087] It should be noted that regarding spatiotemporal rationality, for example, if a client is in region A and requests configuration data from a node in region B, then the requested region ID must match the ID of the node in region B, and the requested timestamp must be within a reasonable range, such as not including future timestamps. Regarding the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints, this means that the hash of the preceding holographic fingerprint (e.g., the operation trajectory hash) is contained within the hash of the following holographic fingerprint. For example, if the current holographic fingerprint is F4, and the three most recent historical holographic fingerprints are F1, F2, and F3 in chronological order, then F4 must contain the operation trajectory hash of F3, F3 must contain the operation trajectory hash of F2, and F2 must contain the operation trajectory hash of F1.

[0088] Furthermore, once a change in configuration data is detected, the configuration data change message needs to be synchronized to various regional nodes through a consensus protocol. Mainstream consensus protocols require multiple rounds of network interaction, resulting in latency of up to hundreds of milliseconds in cross-regional scenarios, which cannot meet the real-time synchronization requirements of configuration data. To address this, this application provides a spatiotemporal consensus protocol: based on "regional spatiotemporal stamps + priority weights" to achieve distributed consensus, regional nodes dynamically adjust their voting weights according to physical distance, reducing cross-regional data synchronization latency.

[0089] Based on this, such as Figure 4 The diagram shown illustrates the implementation flow of a data synchronization method provided in this application embodiment. This method is applied to any regional node in a distributed system and includes the following steps: S401, after detecting a change in configuration data, a configuration data change message is sent to the central area node so that the central area node can perform the following steps.

[0090] In this embodiment of the application, for any configuration data fragment in the configuration data fragment set, the configuration data stored therein may change. After the change occurs, a configuration data change message is generated and can be sent to the central area node so that the central area node can perform the following steps S402~S404.

[0091] The configuration data change message carries the data change content, a spatiotemporal stamp vector (including the region ID, nanosecond-level physical timestamp, and logical counter (incremented by 1 for each change) to mark the spatiotemporal uniqueness of the data version), the data version, and weight proof (the service carrying capacity certificate of the region node, proving its voting qualification), etc. This application embodiment does not limit these.

[0092] S402 receives configuration data change messages and broadcasts them to other regional nodes in the distributed system.

[0093] The central regional node can receive configuration data change messages and broadcast them to the other regional nodes in the distributed system. For example, the central regional node broadcasts configuration data change messages to regional node 2, regional node 3, and regional node 4, while the configuration data change messages are provided by regional node 1.

[0094] S403, obtain the voting results of the remaining regional nodes, where the remaining regional nodes respond to the configuration data change message to vote and generate voting results.

[0095] In this embodiment of the application, after receiving the configuration data change message broadcast by the central regional node, the other regional nodes respond to the configuration data change message by voting and generating a voting result. The voting result can be either agreeing to the change or disagreeing with the change.

[0096] It should be noted that for nodes in other regions, they need to respond to the configuration data change message and vote within a certain time (e.g., 50ms) to generate voting results. Before voting, it is possible to verify that the timestamp in the configuration data change message is not forged (e.g., not earlier than the last change time and not later than the current time), and to verify that the structure of the configuration data change message conforms to the protocol requirements (e.g., containing the correct region ID and weight proof fields).

[0097] Other regional nodes can return the voting results to the central regional node, which can then retrieve the voting results from the other regional nodes. For example, the central regional node retrieves the voting results from regional nodes 2, 3, and 4.

[0098] S404, based on the voting results, synchronizes the configuration data change message to the other regional nodes.

[0099] In this embodiment, the configuration data change message can be synchronized to the remaining regional nodes based on the voting results. The voting results can be either approval or disapproval of the change. The approval votes can be aggregated, and an approval weight can be calculated. If the approval weight exceeds a preset weight, the configuration data change message is synchronized to the remaining regional nodes.

[0100] Specifically, for the voting results of the remaining regional nodes, if the voting results indicate agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight. If the agreement weight exceeds a preset weight (e.g., 0.5), the configuration data change message is synchronized to the remaining regional nodes. The remaining regional nodes themselves have a basic voting weight. The physical distance between the remaining regional nodes and the regional node providing the configuration data change message is calculated, and the basic voting weight is attenuated according to the physical distance. For example, for every 1000 kilometers increase in physical distance, the basic voting weight decreases by 10% (to avoid consensus blocking caused by network latency of long-distance nodes). Then, if the voting results indicate agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight.

[0101] For example, the configuration data change message is provided by region node 1. The voting results of region nodes 2, 3, and 4 all represent the agreement result. The basic voting weight of region node 2 is 0.4, the basic voting weight of region node 3 is 0.3, and the basic voting weight of region node 4 is 0.3. The physical distances between region node 1 and region nodes 2, 3, and 4 are shown in Table 1 below, and the corresponding attenuated voting weights are also shown in Table 1 below. The attenuated voting weights of region nodes 2, 3, and 4 are added together to obtain the agreement weight. If the agreement weight exceeds 0.5, then the configuration data change message will be synchronized to region nodes 2, 3, and 4.

[0102]

[0103] Table 1 Furthermore, in this embodiment, existing systems often use fixed rules for preheating configuration data (such as full loading at startup), without considering business fluctuations, resulting in low resource utilization. Therefore, this embodiment provides a pre-test cache preheating method to improve cache hit rate.

[0104] Based on this, such as Figure 5 The diagram shown is a schematic representation of the implementation process of a data preheating method provided in this application embodiment. This method is applied to any regional node in a distributed system and includes the following steps: S501 retrieves historical access sequences of special event markers, business cycle tags, and configuration data.

[0105] In this embodiment, for any configuration data shard in the configuration data shard set, a special event marker, a business cycle label, and a historical access sequence of the configuration data therein are obtained. The historical access sequence of the configuration data refers to the access volume at a certain time each day for the last N days, for example, the access volume at 10:00 AM each day for the last 7 days (e.g., [100, 150, 200, 300, 250, 180, 120]). The business cycle label refers to whether the day is a weekday or a weekend (e.g., Monday = 1, Sunday = 0). The special event marker refers to whether the day is a special day (e.g., certain activity = 2, certain activity = 1, weekday = 0).

[0106] S502 predicts the probability of accessing configuration data and the estimated query rate per second within a preset third time period in the future, based on special event markers, business cycle tags, and historical access sequences.

[0107] In this embodiment, based on special event markers, business cycle tags, and historical access sequences, the probability of accessing configuration data and the estimated query rate per second within a preset third time period are predicted. For example, the probability of accessing configuration data and the estimated query rate per second are predicted within the next hour.

[0108] One approach is to provide a predictive model, such as an LSTM neural network, that learns the daily / weekly / monthly access patterns of configuration data. This allows for the pre-warming of high-probability access data closer to the expected time, improving cache hit rates. To this end, special event markers, business cycle labels, and historical access sequences can be input into the pre-trained predictive model to obtain the access probability and estimated query rate per second for configuration data within a predetermined third time period in the future.

[0109] S503 determines the preheating strategy for configuration data based on the access probability and the estimated query rate per second, and preheats the configuration data according to the preheating strategy.

[0110] In this embodiment of the application, the access probability and estimated query rate per second of the configuration data within a preset third time period can be used to determine the preheating strategy of the configuration data, and the configuration data can be preheated according to the preheating strategy.

[0111] If the access probability is in the first probability range (e.g., >80%) and the estimated query rate per second is in the first query rate range, it indicates that the configuration data is high-probability data. Therefore, preheating to the local cache is determined as the preheating strategy for the configuration data. The configuration data is preheated according to the preheating strategy. Specifically, the configuration data can be preheated to the local cache, where the local cache needs to reserve a certain amount of space (e.g., 30%) to cope with sudden access.

[0112] If the access probability is in the second probability range (e.g., 40%-80%) and the estimated query rate per second is in the second query rate range, it indicates that the configuration data is medium probability data. Therefore, preheating to the regional shared cache is determined as the preheating strategy for the configuration data. The configuration data is then preheated according to the preheating strategy, which may specifically be preheating the configuration data to the regional shared memory.

[0113] If the access probability is in the third probability range (e.g., <40%) and the estimated query rate per second is in the third query rate range, it indicates that the configuration data is low probability data. Therefore, the preheating metadata is determined as the preheating strategy for the configuration data. The configuration data is then preheated according to the preheating strategy. Specifically, it can be that only the metadata of the configuration data is preheated, and the data is loaded immediately upon access.

[0114] Thus, this application embodiment dynamically adjusts the granularity of sharding by QPS and business priority to improve the processing efficiency of hot data; it expands the fingerprint from a single dimension of "content" to three dimensions of "content, spatiotemporal, and trajectory" to enhance anti-tampering capabilities by using holographic fingerprinting; it reduces cross-regional consensus latency by using a spatiotemporal consensus protocol; and it improves cache hit rate by learning the "daily / weekly / monthly" access patterns of configuration data through predictive cache preheating to preheat high-probability access data in advance.

[0115] Corresponding to the above method embodiments, this application also provides a segmented dynamic adjustment device, such as... Figure 6 As shown, the device may include: a fragment set acquisition module 610, a priority determination module 620, and a fragment dynamic adjustment module 630.

[0116] The shard set acquisition module 610 is used to acquire the configuration data shard set corresponding to the region node, wherein the configuration data shard set contains at least one configuration data shard. The priority determination module 620 is used to determine the configuration data stored in any configuration data fragment in the configuration data fragment set, and to find the service priority of the configuration data. The sharding dynamic adjustment module 630 is used to adjust the configuration data shards according to the query rate per second of the configuration data shards and the service priority.

[0117] In an optional implementation, the segmentation dynamic adjustment module specifically includes: The sharding granularity determination submodule is used to determine the target sharding granularity based on the query rate per second of the configured data sharding and the business priority. The sharding dynamic adjustment submodule is used to adjust the configuration data shards according to the target sharding granularity.

[0118] In an optional implementation, the granularity determination submodule specifically includes: Granularity and coefficient acquisition unit, used to obtain basic sharding granularity and distributed system resource adaptation coefficient; The factor determination unit is used to determine the dynamic demand weight factor based on the query rate per second of the configured data shards and the business priority. The sharding granularity determination unit is used to determine the target sharding granularity based on the basic sharding granularity, the dynamic demand weighting factor, and the distributed system resource adaptation coefficient.

[0119] In an optional implementation, the factor determination unit is specifically used for: Determine the preset business priority coefficient corresponding to the business priority, and determine the logarithm of the query rate per second for the configured data shards; The dynamic demand weighting factor is obtained by adding the preset business priority coefficient to the logarithm of the query rate per second of the configured data shard.

[0120] In an optional implementation, the granularity determination unit is specifically used for: Obtain the product between the basic sharding granularity and the dynamic demand weight factor, and divide the product by the distributed system resource adaptation coefficient to obtain the target sharding granularity.

[0121] In an optional implementation, the apparatus further includes a sharding module, configured to perform the following processing on any configuration data shard in the configuration data shard set: determining whether the query rate per second is greater than a preset hotspot threshold and whether the duration of the query rate per second being greater than the preset hotspot threshold reaches a preset first duration; if the query rate per second is greater than the preset hotspot threshold and the duration of the query rate per second being greater than the preset hotspot threshold reaches the preset first duration, horizontally splitting the configuration data shard to obtain multiple sub-configuration data shards; and establishing a mapping relationship between the configuration data shard and the multiple sub-configuration data shards.

[0122] In an optional implementation, the apparatus further includes a shard merging module, configured to perform the following processing on any configuration data shard in the configuration data shard set: determining whether the query rate per second is less than a preset cold data threshold and whether the duration of the query rate per second being less than the preset cold data threshold reaches a preset second duration; if the query rate per second is less than the preset cold data threshold and the duration of the query rate per second being less than the preset cold data threshold reaches the preset second duration, vertically merging the configuration data shard to obtain a merged configuration data shard; and establishing a mapping relationship between the configuration data shard and the merged configuration data shard.

[0123] In an optional implementation, the apparatus further includes: The hash determination module is used to determine the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data after detecting a change in the configuration data. The fingerprint generation module is used to generate a holographic fingerprint of the modified configuration data based on the data content hash, the operation trajectory hash, and the spatiotemporal hash.

[0124] In an optional implementation, the hash determination module is specifically used for: The modified configuration data is hashed to obtain the data content hash. Determine the data change operation sequence corresponding to the modified configuration data, and perform hash processing on the data change operation sequence to obtain the operation trajectory hash; Obtain the fragment identifier of the configuration data fragment, the region identifier of the region node, and the current timestamp; The fragment identifier, the region identifier, and the current timestamp are hashed to obtain a spatiotemporal hash.

[0125] In an optional implementation, the apparatus further includes: The fingerprint acquisition module is used to receive a request from the client for the changed configuration data, and in response to the request, acquire the holographic fingerprint and the most recent N historical holographic fingerprints, where N is a positive integer; The fingerprint return module is used to return the holographic fingerprint and the most recent N historical holographic fingerprints to the client, so that the client can perform the following steps: verify the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints; if the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints pass the verification, load the changed configuration data.

[0126] In an optional implementation, the apparatus further includes: The message sending module is used to send a configuration data change message to the central regional node after detecting a change in the configuration data, so that the central regional node performs the following steps: receiving the configuration data change message and broadcasting it to the other regional nodes in the distributed system; obtaining the voting results of the other regional nodes, wherein the other regional nodes respond to the configuration data change message by voting and generating the voting results; and synchronizing the configuration data change message to the other regional nodes according to the voting results.

[0127] In an optional implementation, the message sending module is specifically used for: If the voting result indicates agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight. If the agreed weight exceeds the preset weight, the configuration data change message will be synchronized to the remaining regional nodes.

[0128] In an optional implementation, the apparatus further includes: The sequence acquisition module is used to acquire special event markers, business cycle tags, and historical access sequences of the configuration data. The prediction module is used to predict the access probability and estimated query rate per second of the configuration data within a preset third time period in the future, based on the special event marker, the business cycle label, and the historical access sequence. The strategy determination module is used to determine the preheating strategy for the configuration data based on the access probability and the estimated query rate per second. The data preheating module is used to preheat the configuration data according to the preheating strategy.

[0129] In an optional implementation, the strategy determination module is specifically used for: When the access probability is within the first probability range and the estimated query rate per second is within the first query rate range, preheating to the local cache is determined as the preheating strategy for the configuration data. When the access probability is in the second probability range and the estimated query rate per second is in the second query rate range, preheating to the regional shared cache is determined as the preheating strategy for the configuration data. When the access probability is in the third probability interval and the estimated query rate per second is in the third query rate interval, the preheating metadata is determined as the preheating strategy for the configuration data.

[0130] This application also provides an electronic device, such as... Figure 7 As shown, it includes a processor 71, a communication interface 72, a memory 73, and a communication bus 74. The processor 71, the communication interface 72, and the memory 73 communicate with each other through the communication bus 74. Memory 73 is used to store computer programs; When processor 71 executes the program stored in memory 73, it performs the following steps: Obtain the configuration data shard set corresponding to the region node, the configuration data shard set containing at least one configuration data shard; for any configuration data shard in the configuration data shard set, determine the configuration data stored in the configuration data shard, and find the service priority of the configuration data; adjust the configuration data shard according to the query rate per second of the configuration data shard and the service priority.

[0131] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0132] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0133] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0134] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0135] In another embodiment provided in this application, a storage medium is also provided, which stores instructions that, when run on a computer, cause the computer to execute any of the segmentation dynamic adjustment methods described in the above embodiments.

[0136] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the segmentation dynamic adjustment methods described in the above embodiments.

[0137] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0138] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0140] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for dynamic adjustment of segments, characterized in that, The method, applicable to any regional node in a distributed system, includes: Obtain the configuration data shard set corresponding to the region node, wherein the configuration data shard set contains at least one configuration data shard; For any configuration data shard in the configuration data shard set, determine the configuration data stored in the configuration data shard and find the service priority of the configuration data; The configuration data shards are adjusted based on the query rate per second of the configuration data shards and the service priority.

2. The method according to claim 1, characterized in that, The step of adjusting the configuration data shards based on the query rate per second of the configuration data shards and the service priority includes: Based on the query rate per second of the configured data shards and the service priority, the target shard granularity is determined, and the configured data shards are adjusted according to the target shard granularity.

3. The method according to claim 2, characterized in that, The step of determining the target shard granularity based on the query rate per second of the configured data shards and the service priority includes: Obtain the basic sharding granularity and the resource adaptation coefficient of the distributed system; The dynamic demand weighting factor is determined based on the query rate per second of the configured data shards and the business priority. The target sharding granularity is determined based on the basic sharding granularity, the dynamic demand weighting factor, and the distributed system resource adaptation coefficient.

4. The method according to claim 3, characterized in that, The step of determining the dynamic demand weighting factor based on the query rate per second of the configured data shards and the business priority includes: Determine the preset business priority coefficient corresponding to the business priority, and determine the logarithm of the query rate per second for the configured data shards; The dynamic demand weighting factor is obtained by adding the preset business priority coefficient to the logarithm of the query rate per second of the configured data shard.

5. The method according to claim 1, characterized in that, For any configuration data shard in the aforementioned configuration data shard set, the following processing is also performed: Determine whether the query rate per second is greater than a preset hotspot threshold, and whether the duration for which the query rate per second is greater than the preset hotspot threshold reaches a preset first duration; If the query rate per second is greater than a preset hotspot threshold and the duration of the query rate per second being greater than the preset hotspot threshold reaches a preset first duration, the configuration data shard is horizontally split to obtain multiple sub-configuration data shards. Establish a mapping relationship between the configuration data shard and the multiple sub-configuration data shards.

6. The method according to claim 1, characterized in that, For any configuration data shard in the aforementioned configuration data shard set, the following processing is also performed: Determine whether the query rate per second is less than a preset cold data threshold, and whether the duration for which the query rate per second is less than the preset cold data threshold reaches a preset second duration; If the query rate per second is less than a preset cold data threshold and the duration of the query rate per second being less than the preset cold data threshold reaches a preset second duration, the configuration data fragments are vertically merged to obtain merged configuration data fragments. Establish a mapping relationship between the configuration data shards and the merged configuration data shards.

7. The method according to claim 1, characterized in that, The method further includes: Once a change in the configuration data is detected, the data content hash, operation trajectory hash, and spatiotemporal hash of the changed configuration data are determined. A holographic fingerprint of the modified configuration data is generated based on the data content hash, the operation trajectory hash, and the spatiotemporal hash.

8. The method according to claim 7, characterized in that, The determination of the data content hash, operation trajectory hash, and spatiotemporal hash of the modified configuration data includes: The modified configuration data is hashed to obtain the data content hash. Determine the data change operation sequence corresponding to the modified configuration data, and perform hash processing on the data change operation sequence to obtain the operation trajectory hash; Obtain the fragment identifier of the configuration data fragment, the region identifier of the region node, and the current timestamp; The fragment identifier, the region identifier, and the current timestamp are hashed to obtain a spatiotemporal hash.

9. The method according to claim 7 or 8, characterized in that, The method further includes: Receive a request from the client for the modified configuration data, and in response to the request, obtain the holographic fingerprint and the most recent N historical holographic fingerprints, where N is a positive integer; The holographic fingerprint and the most recent N historical holographic fingerprints are returned to the client so that the client can perform the following steps: Verify the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints; If the spatiotemporal rationality and the continuity between the holographic fingerprint and the most recent N historical holographic fingerprints are verified, the modified configuration data is loaded.

10. The method according to claim 1, characterized in that, The method further includes: Upon detecting a change in the configuration data, a configuration data change message is sent to the central area node, causing the central area node to perform the following steps: Receive the configuration data change message and broadcast it to the remaining regional nodes in the distributed system; Obtain the voting results of the remaining regional nodes, wherein the remaining regional nodes vote in response to the configuration data change message to generate the voting results; Based on the voting results, the configuration data change message will be synchronized to the remaining regional nodes.

11. The method according to claim 10, characterized in that, The step of synchronizing the configuration data change message to other regional nodes based on the voting results includes: If the voting result indicates agreement, the voting weights of the remaining regional nodes are summed to obtain the agreement weight. If the agreed weight exceeds the preset weight, the configuration data change message will be synchronized to the remaining regional nodes.

12. The method according to claim 1, characterized in that, The method further includes: Obtain special event markers, business cycle tags, and the historical access sequence of the configuration data; Based on the special event marker, the business cycle label, and the historical access sequence, predict the access probability and estimated query rate per second of the configuration data within a future preset third time period; Based on the access probability and the estimated query rate per second, a preheating strategy for the configuration data is determined, and the configuration data is preheated according to the preheating strategy.

13. The method according to claim 12, characterized in that, The step of determining the preheating strategy for the configuration data based on the access probability and the estimated query rate per second includes: When the access probability is within the first probability range and the estimated query rate per second is within the first query rate range, preheating to the local cache is determined as the preheating strategy for the configuration data. When the access probability is in the second probability range and the estimated query rate per second is in the second query rate range, preheating to the regional shared cache is determined as the preheating strategy for the configuration data. When the access probability is in the third probability interval and the estimated query rate per second is in the third query rate interval, the preheating metadata is determined as the preheating strategy for the configuration data.

14. A segmented dynamic adjustment device, characterized in that, The device is applied to any regional node in a distributed system and includes: The shard set acquisition module is used to acquire the configuration data shard set corresponding to the region node, wherein the configuration data shard set contains at least one configuration data shard. The priority determination module is used to determine the configuration data stored in any configuration data fragment in the configuration data fragment set, and to find the service priority of the configuration data. The sharding dynamic adjustment module is used to adjust the configuration data shards according to the query rate per second of the configuration data shards and the business priority.

Images