A method and device for scaling a distributed database and a storage medium

Abstract

A method and device for scaling a distributed database and a storage medium, comprising: in the case of meeting a scaling condition, scaling the distributed database using a consistent hashing algorithm; during the scaling, suspending a write request from a client for a target physical node; obtaining feature information of the suspended write request according to a preset strategy; in the case of the obtained feature information reaching a feature threshold, suspending the scaling, and processing a new write request from the client for the target physical node and the suspended write request whose feature information reaches the feature threshold, until the scaling is resumed in the case of re-meeting a scaling preparation condition. The embodiments of the present disclosure enable the distributed database to still provide normal services during scaling, thereby avoiding service interruption of the distributed database and improving service availability of the business distributed database.

Description

Technical Field

[0001] This article relates to data processing technology for distributed databases, and in particular to a method, apparatus, and storage medium for scaling up and down a distributed database. Background Technology

[0002] As the scale of business expands or contracts, distributed databases need to be scaled up or down accordingly to adapt to business needs.

[0003] In related technologies, consistent hashing algorithms are often used to expand or shrink distributed databases. When adding or deleting nodes, consistent hashing algorithms can ensure that most records should still be assigned to the same node as before, thus minimizing the amount of data migration.

[0004] However, using consistent hashing to expand or shrink a distributed database can still cause some service interruptions, thus affecting the service availability of the business distributed database. Summary of the Invention

[0005] This application provides a method, apparatus, and storage medium for scaling up and down a distributed database, which can maintain the service of the distributed database without interruption, thereby avoiding impacting the service availability of the business distributed database.

[0006] On the one hand, this application provides a method for scaling up and down a distributed database, including:

[0007] Under the condition that the expansion and contraction conditions are met, the consistent hashing algorithm is used to expand or shrink the distributed database.

[0008] During the scaling up or scaling down process, write requests from clients to target physical nodes are suspended; wherein, the target physical node is the physical node involved in the scaling up or scaling down process of the distributed database; and, the characteristic information of the suspended write requests is obtained according to a preset strategy.

[0009] If the acquired feature information reaches the feature threshold, the expansion or reduction is paused, and new write requests from the client for the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the expansion or reduction is resumed when the expansion or reduction preparation conditions are met again.

[0010] On the other hand, this application provides a scaling device for a distributed database, including: a memory and a processor, wherein the memory is used to store an executable program;

[0011] The processor is used to read and execute the executable program to implement the above-described method for scaling up and down a distributed database.

[0012] Furthermore, this application also provides a scaling device for a distributed database, comprising:

[0013] The scaling unit is used to scale up or down the distributed database using a consistent hashing algorithm when the scaling conditions are met.

[0014] A suspension unit is used to suspend write requests from clients to a target physical node during the expansion or contraction of the distributed database; wherein the target physical node is a physical node involved in the expansion or contraction of the distributed database; and to obtain characteristic information of the suspended write requests according to a preset strategy.

[0015] The request processing unit is used to pause the expansion or contraction when the acquired feature information reaches the feature threshold, and to process new write requests from the client for the target physical node, as well as suspended write requests when the feature information reaches the feature threshold, until the expansion or contraction is resumed when the expansion or contraction preparation conditions are met again.

[0016] In another aspect, this application also provides a storage medium containing a computer program, which, when executed by a processor, implements the above-described method for scaling up and down a distributed database.

[0017] Compared with related technologies, this application includes scaling up or down a distributed database using a consistent hashing algorithm when scaling up or down conditions are met; during the scaling up or down process, suspending write requests from clients to target physical nodes; wherein the target physical node is the physical node involved in the scaling up or down process of the distributed database; and obtaining feature information of the suspended write requests using a preset strategy; when the obtained feature information reaches a feature threshold, pausing the scaling up or down process, and processing new write requests from clients to target physical nodes, as well as suspended write requests whose feature information reaches the feature threshold, until the scaling up or down process is resumed when the scaling up or down preparation conditions are met again. Therefore, the distributed database can still provide services normally during scaling up or down, thereby avoiding service interruption of the distributed database and improving the service availability of the business distributed database.

[0018] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0020] Figure 1 A flowchart illustrating a method for scaling up or down a distributed database, as provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a method for scaling up distributed database nodes using a consistent hashing algorithm in the prior art;

[0022] Figure 3 A schematic diagram of the structure of a distributed database scaling device provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the structure of a distributed database scaling device provided in an embodiment of this application. Detailed Implementation

[0024] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0025] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0026] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0027] This application provides a method for scaling up and down a distributed database, such as... Figure 1 As shown, it includes:

[0028] Step 101: Under the condition that the expansion and contraction conditions are met, the distributed database is expanded or contracted using the consistent hashing algorithm;

[0029] Step 102: During the expansion or contraction process, suspend write requests from clients to the target physical node; wherein the target physical node is the physical node involved in the expansion or contraction process of the distributed database; and, obtain the characteristic information of the suspended write requests using a preset strategy.

[0030] Step 103: If the acquired feature information reaches the feature threshold, pause the expansion or reduction and process new write requests from the client for the target physical node, as well as suspended write requests when the feature information reaches the feature threshold, until the expansion or reduction is resumed when the expansion or reduction preparation conditions are met again.

[0031] For example, the feature threshold can be a threshold for the features of pending write requests that need to be processed, set according to the actual situation. Whenever the feature information of one or more pending write requests reaches the feature threshold, it means that the one or more pending write requests need to be processed; otherwise, it will affect the availability of the services provided by the client to the distributed database.

[0032] For example, obtaining the feature information of suspended write requests using a preset strategy may include: periodically obtaining the feature information of suspended write requests. Specifically, when periodically obtaining the feature information of suspended write requests, the feature information of suspended write requests may be obtained at preset time intervals. The preset time interval can be flexibly set according to specific circumstances. If the preset time interval is short, the feature information can be obtained in time to reach the feature threshold, that is, the suspended write request that needs to be processed. If the resource consumption is relatively large, and the preset time interval is long, the suspended write request whose feature information reaches the feature threshold may not be obtained in time compared to the case of a short preset time interval, but the resource consumption is relatively small.

[0033] In related technologies, distributed databases need to be scaled up or down to adapt to business needs as the business scale expands or contracts. During the scaling process, the database becomes unavailable and requires shutdown or service interruption, resulting in the failure of all or some SQL requests. Distributed databases are widely used in large systems and are usually implemented in two main ways: pure native and proxy modes.

[0034] Taking the proxy pattern as an example, the diagram illustrating the use of consistent hashing algorithm to scale up distributed database nodes is as follows: Figure 2 As shown, consistent hashing is used to manage distributed database storage nodes (such as MySQL) and dynamically scale them up and down. N1, N2, and N3 are the hash nodes of the three existing MySQL instances, and are real nodes.

[0035] V1, V2, V3...Vn are virtual nodes added to adjust the system balance, and are N1, N2, N3...Nn virtual nodes respectively. The actual number of virtual nodes can be adjusted according to the balance situation. The three sharding key values ​​are hashed to correspond to three nodes, which are then assigned to the nearest real node in sequence.

[0036] Taking the addition of MySQL 4 as an example, before the expansion, data between N1 and N3 (clockwise) resided in MySQL 3; after the expansion, data between N1 and N4 (clockwise) resided in MySQL 4, while data between N4 and N3 remained in MySQL 3. The detailed expansion process is as follows:

[0037] Step 1: Create a new MySQL physical instance node and create the corresponding database tables according to the expansion plan (MySQL4).

[0038] Step 2: Select a specific time point that is relatively close to the current time as the base point, and start the full data migration from N3 to N4. After completion, the data before the base point will be backed up in N3 and N4.

[0039] Step 3: After completing the full migration, begin incremental data catching up based on the base point time, ensuring the incremental data catches up to be almost identical on both sides. Because N3 continuously receives new data, N4's data will always lag slightly behind N3. Scaling / scaling preparation conditions can include the unbacked-up index of the target physical node not exceeding a first preset value. In the example above, this could specifically mean that N3's data has been largely backed up on N4. Unbacked-up indicators can include the amount of unbacked-up data, the unbacked-up data rate, etc. For example, if the unbacked-up data amount does not exceed 'a', or the unbacked-up data rate does not exceed 'b', then the scaling / scaling preparation conditions are met.

[0040] Step 4: Observe the duration of recent normal transaction operations. Under normal circumstances, each transaction should be completed within milliseconds (this duration is called T1). If long transactions occur (such as batch processing tasks), postpone the scaling up, or communicate with the business beforehand to confirm that the corresponding operation should not be performed during the scaling up process. This step aims to minimize the operation time of subsequent steps. Scaling up / down preparation conditions may include that the longest transaction processing time of the distributed database does not exceed a second preset value, such as T1 in the example above, or a value calculated based on T1.

[0041] In some embodiments, scaling up or down includes a preparation phase, such as steps 1 to 4 described above, where data backup is performed on the physical nodes involved. Once the scaling up or down preparation conditions are met, the system can enter scaling up or scaling down mode, as described below.

[0042] Step 5: The distributed database enters expansion mode via parameter control (this parameter can be controlled by operations personnel or automatically by the system), and all read requests are normal. Write requests to N3 (including explicitly starting new transactions) will be suspended unless they are within an already opened transaction.

[0043] Step 6: The distributed database detects all transaction operations. Theoretically, it can end all enabled transactions after waiting for the duration T1 of one transaction in Step 4.

[0044] Step 7: Check the incremental catch-up status. In theory, the catch-up of a small amount of data can be completed within milliseconds (this time is called T2, which is the time it takes for this new data to be synchronized to node N4 after node N3 suspends a new request). All data of N3 and N4 will eventually be completely consistent.

[0045] Step 8: The distributed database performs a sharding logic switch. That is, the system recognizes the existence of N4, and new requests in N1-N4 will land in the expanded MySQL4. After the switch, the distributed database is de-expanded.

[0046] Step 9: The distributed database sequentially executes the suspended write requests, which may fall on N3 or N4.

[0047] Step 10: The distributed database checks and cleans up redundant data in N3 and N4, and the expansion is complete.

[0048] The process of scaling down a distributed database is similar to steps 1-10 above.

[0049] The distributed database scaling method provided in this application embodiment expands or shrinks the distributed database using a consistent hashing algorithm when the scaling conditions are met. During the scaling or shrinking process, write requests from clients to target physical nodes are suspended. The target physical node is the physical node involved in the scaling or shrinking process of the distributed database. Furthermore, feature information of the suspended write requests is obtained using a preset strategy. When the obtained feature information reaches a feature threshold, the scaling or shrinking is paused, and new write requests from the client to the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the scaling or shrinking preparation conditions are met again, thus resuming the scaling or shrinking. Therefore, the distributed database can still provide normal service during scaling or shrinking, thereby avoiding service interruption of the distributed database and improving the service availability of the business distributed database.

[0050] In one exemplary instance, the feature information includes at least one of the following: the duration of the suspended write request, and the number of suspended write requests.

[0051] In one exemplary instance, when the feature information includes: the suspension duration of the suspended write request, the feature threshold includes: the product of the client request timeout duration and a preset timeout threshold.

[0052] For example, assuming the client request timeout duration is T3 and the timeout threshold is K, the feature threshold is T3*K. When the suspension duration of a suspended write request exceeds T3*K, new write requests from the client targeting the target physical node, as well as suspended write requests with a suspension duration exceeding T3*K, are processed. K can be adjusted according to the actual situation, for example, set to 80%. Then, when the suspension duration of a suspended write request exceeds T3*80%, these suspended write requests need to be processed.

[0053] In one exemplary instance, when the distributed database is expanded using a consistent hashing algorithm, the target physical node includes: the physical node that is closest to the new physical node in a clockwise direction.

[0054] In one exemplary instance, when the distributed database is downsizing using a consistent hashing algorithm, the target physical node includes: the physical node to be deleted.

[0055] For example, such as Figure 2As shown, when the distributed database is expanded, the target physical node is N4, and when the distributed database is shrunk, the target physical node is N3.

[0056] In one exemplary instance, processing new write requests from the client targeting a target physical node, and suspended write requests where feature information reaches a feature threshold, includes:

[0057] First, allocate a first number of threads for concurrent processing to the new write request according to the pre-allocation rules, and allocate a second number of threads for concurrent processing to the suspended write request;

[0058] Secondly, new write requests from the client are processed concurrently using the first number of concurrent processing threads, and suspended write requests in the suspended request queue are processed concurrently using the second number of concurrent processing threads.

[0059] For example, in order to speed up data processing, new write requests added during the pause and write requests that were suspended before the pause can be processed separately. During the processing, a first number of threads with a concurrent processing capacity can be allocated to the new write requests and a second number of threads with a concurrent processing capacity can be allocated to the suspended write requests. The allocated threads can then be used to process the corresponding write requests.

[0060] For example, write requests can be stored using queues. New write requests and pending write requests can be stored using different queues, called the pending request queue and the new request queue, respectively.

[0061] For example, after scaling is paused, if there are any unprocessed write requests in the new request queue, they are placed in the pending request queue.

[0062] In one exemplary instance, suspending the write request from the client for the target physical node includes:

[0063] First, whenever a suspended write request for a target physical node is received from a client, the received suspended write request is placed at the tail of the suspended request queue.

[0064] Secondly, during the pause of the expansion or contraction, whenever a new write request for the target physical node is received from a client, the new write request is inserted at the tail of the request processing queue.

[0065] Queues are a first-in, first-out (FIFO) data processing method. Using queues to store suspended write requests ensures that the first suspended write request is processed first, thus guaranteeing the orderly processing of suspended write requests.

[0066] In one exemplary instance, restoring the expanded or shrunk capacity includes:

[0067] First, determine whether all write requests in the request processing queue have been processed.

[0068] Secondly, if there are unprocessed write requests in the request processing queue, the unprocessed write requests are inserted into the tail of the suspended request queue as the suspended write requests.

[0069] In one exemplary instance, the scaling conditions include:

[0070] The unbacked-up index of the target physical node does not exceed a first preset value, and / or the longest transaction processing time of the distributed database within a predetermined time does not exceed a second preset value.

[0071] For example, after the full migration is completed, the target physical node begins to catch up with incremental data based on the base point time (the base point time is a specific time point that is close to the current time), so that the incremental data catches up with the data of the physical node that backs up the target physical node. However, since the target physical node is constantly receiving new data, the data of the target physical node will always lag slightly behind the backup physical node. Therefore, a second preset value can be set to scale up or down when the unbacked-up index does not exceed the second preset value.

[0072] For example, under normal circumstances, each transaction should be completed within milliseconds. If a long transaction occurs (such as batch processing of tasks), the scaling will be postponed. Therefore, the scaling will be performed when the longest processing time of a transaction in the distributed database does not exceed the second preset value within the predetermined time.

[0073] The step of using the first number of concurrent processing threads to concurrently process new write requests from the client includes:

[0074] Write requests in the request processing queue are processed concurrently using the first number of concurrent processing threads.

[0075] For example, in a single concurrent processing operation, the number of write requests in the request processing queue is equal to the number of threads in the first concurrent processing operation, and these obtained write requests are processed in parallel using the first number of threads.

[0076] In one exemplary instance, when suspending a new write request from the client for the target physical node again, it further includes:

[0077] First, determine whether the write requests in the request processing queue have been processed.

[0078] Secondly, when the write requests in the request processing queue are not completely processed, insert the write requests in the request processing queue into the end of the pending request queue as the pending write requests.

[0079] Exemplarily, when a new write request from a client for a target physical node is suspended again, the normal received write requests in the request processing queue may not be completely processed yet. Then, it is necessary to insert the unprocessed part of these write requests into the end of the pending request queue as the pending write requests.

[0080] In one exemplary instance, the method further includes:

[0081] First, calculate the suspension duration of the pending write requests starting from the time when they are inserted into the pending request queue.

[0082] Secondly, calculate the suspension duration of the unprocessed write requests in the request processing queue that are inserted into the pending request queue as the pending write requests starting from the time when they are received.

[0083] Exemplarily, the suspension duration of all requests in the suspension queue is calculated starting from the time when they are received. It's just that some requests are put into the suspension queue as soon as they are received, and some are transferred from the request queue.

[0084] The key points of the method for scaling out and scaling in of the distributed database provided by the embodiments of the present application are as follows:

[0085] 1. Through the scaling - out mode / scaling - in mode, temporarily suspend the client requests. After waiting for a duration of T1 + T2, theoretically, the scaling - out or scaling - in is completed, and the requests are processed continuously. That is, the request duration during scaling - out is longer than that during non - scaling - out by (T1 + T2), and both T1 and T2 are in milliseconds. In this way, the client request response can be slightly longer, but it can be completed normally and is not affected.

[0086] 2. To ensure that the client requests do not time out, three other scaling - out parameters can be introduced, namely 1) the client request timeout duration T3, 2) the timeout threshold K, and 3) the length L of the pending request queue. The theoretical duration of a transaction request during scaling - out is T = T1+T2 + T1. When T < T3, the scaling - out can be completed normally. Considering the pending requests, when they are awakened, queuing may occur. Scaling - out can be aborted when the following situations occur: 1. The time that the first suspended request has been suspended exceeds T3*K; 2. The length of the suspension queue exceeds L.

[0087] 3. After the abort situation in 2 occurs, the scaling - out / scaling - in does not terminate. After解除扩容 / 缩容模式 (should be "lifting the scaling - out / scaling - in mode"), quickly process the suspended requests and continue with the N4 and N3 incremental catch - up; then enter the scaling - out / scaling - in mode again until the scaling - out / scaling - in is completed.

[0088] It should be noted that there seems to be an incorrect expression "解除扩容 / 缩容模式" in the original Chinese text. It is translated as "lifting the scaling - out / scaling - in mode" here for the purpose of translation. You may need to check and correct it according to the actual situation.4. Parameters such as T3, K, and L can be dynamically adjusted by maintenance personnel.

[0089] 5. Before the actual expansion or reduction process, the system can predict the success rate based on the above parameters. That is, based on the system throughput (Transactions Per Second, TPS) and the duration of a single transaction in a recent period (e.g., 5 minutes), the expansion time and the length of the suspended request queue during the expansion are predicted. The system can comprehensively judge the success rate of the current expansion / reduction and guide the operation and maintenance personnel to decide whether to start expansion / reduction.

[0090] This application also provides a scaling device for a distributed database, such as... Figure 3 As shown, it includes: a memory 21 and a processor 22, wherein the memory 21 is used to store executable programs;

[0091] The processor 22 is used to read and execute the executable program to perform the following steps:

[0092] Under the condition that the expansion and contraction conditions are met, the consistent hashing algorithm is used to expand or shrink the distributed database.

[0093] During the scaling up or scaling down process, write requests from clients to target physical nodes are suspended; wherein, the target physical node is the physical node involved in the scaling up or scaling down process of the distributed database; and, the characteristic information of the suspended write requests is obtained according to a preset strategy.

[0094] If the acquired feature information reaches the feature threshold, the expansion or reduction is paused, and new write requests from the client for the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the expansion or reduction is resumed when the expansion or reduction preparation conditions are met again.

[0095] In one exemplary instance, the feature information includes at least one of the following: the duration of the suspended write request, and the number of suspended write requests.

[0096] In one exemplary instance, when the feature information includes: the suspension duration of the suspended write request, the feature threshold includes: the product of the client request timeout duration and a preset timeout threshold.

[0097] In one exemplary instance, when the distributed database is expanded using a consistent hashing algorithm, the target physical node includes: the physical node that is closest to the new physical node in a clockwise direction;

[0098] When the distributed database uses a consistent hashing algorithm for scaling down, the target physical nodes include: physical nodes to be deleted.

[0099] In one exemplary instance, the processor 22 is configured to read and execute the executable program to perform the following steps:

[0100] According to the pre-allocation rules, a first number of threads are allocated to the new write request for concurrent processing, and a second number of threads are allocated to the suspended write request for concurrent processing.

[0101] New write requests from the client are processed concurrently using the first number of concurrent processing threads, and suspended write requests are processed concurrently using the second number of concurrent processing threads.

[0102] In one exemplary instance, the processor 22 is configured to read and execute the executable program to perform the following steps:

[0103] Whenever a suspended write request for a target physical node is received from a client, the received suspended write request is placed at the tail of the suspended request queue.

[0104] During the pause of the expansion or contraction, whenever a new write request for the target physical node is received from a client, the new write request is inserted at the tail of the request processing queue.

[0105] In one exemplary instance, the processor 22 is configured to read and execute the executable program to perform the following steps:

[0106] Determine whether all write requests in the request processing queue have been processed.

[0107] If there are unprocessed write requests in the request processing queue, the unprocessed write requests are inserted at the tail of the suspended request queue as suspended write requests.

[0108] In one exemplary instance, the scaling conditions include:

[0109] The unbacked-up index of the target physical node does not exceed a first preset value, and / or the longest transaction processing time of the distributed database within a predetermined time does not exceed a second preset value.

[0110] The distributed database scaling device provided in this application embodiment expands or shrinks the distributed database using a consistent hashing algorithm when the scaling conditions are met. During the scaling or shrinking process, write requests from clients to target physical nodes are suspended. The target physical node is the physical node involved in the scaling or shrinking process of the distributed database. The device also acquires feature information of the suspended write requests using a preset strategy. When the acquired feature information reaches a feature threshold, the scaling or shrinking is paused, and new write requests from the client to the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the scaling or shrinking is resumed when the scaling or shrinking preparation conditions are met again. Therefore, the distributed database can still provide normal service during scaling or shrinking, thereby avoiding service interruption of the distributed database and improving the service availability of the business distributed database.

[0111] This application also provides a scaling device for a distributed database, such as... Figure 4 As shown, it includes:

[0112] The scaling unit 31 is used to scale up or down the distributed database using a consistent hashing algorithm when the scaling conditions are met.

[0113] The suspension unit 32 is used to suspend write requests from clients to target physical nodes during the expansion or contraction process of the distributed database; wherein the target physical node is a physical node involved in the expansion or contraction process of the distributed database; and to obtain the characteristic information of the suspended write requests according to a preset strategy.

[0114] The request processing unit 33 is used to pause the expansion or reduction when the acquired feature information reaches the feature threshold, and to process new write requests from the client for the target physical node, as well as suspended write requests when the feature information reaches the feature threshold, until the expansion or reduction is resumed when the expansion or reduction preparation conditions are met again.

[0115] The distributed database scaling device provided in this application embodiment, when the scaling conditions are met, uses a consistent hashing algorithm to scale up or down the distributed database. During the scaling up or down process, write requests from clients to target physical nodes are suspended; wherein, the target physical node is the physical node involved in the scaling up or down process of the distributed database. Furthermore, feature information of the suspended write requests is obtained using a preset strategy. When the obtained feature information reaches a feature threshold, the scaling up or down process is paused, and new write requests from the client to the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the scaling up or down process resumes when the scaling up or down preparation conditions are met again. Therefore, the distributed database can still provide normal service during scaling up or down, thereby avoiding service interruption of the distributed database and improving the service availability of the business distributed database.

[0116] This application also provides a storage medium storing a computer program, which, when executed by a processor, can implement the scaling method of the distributed database as described in any of the above embodiments.

[0117] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A method for scaling up and down a distributed database, characterized in that, include: Under the condition that the expansion and contraction conditions are met, the consistent hashing algorithm is used to expand or shrink the distributed database. During the scaling up or scaling down process, write requests from clients to target physical nodes are suspended; wherein, the target physical node is the physical node involved in the scaling up or scaling down process of the distributed database; and, the characteristic information of the suspended write requests is obtained according to a preset strategy. If the acquired feature information reaches the feature threshold, the expansion or reduction is paused, and new write requests from the client for the target physical node, as well as suspended write requests whose feature information reaches the feature threshold, are processed until the expansion or reduction is resumed when the expansion or reduction preparation conditions are met again.

2. The method according to claim 1, characterized in that, The feature information includes at least one of the following: the suspension duration of the suspended write request, and the number of suspended write requests.

3. The method according to claim 2, characterized in that, When the feature information includes: the suspension duration of the suspended write request, the feature threshold includes: the product of the client request timeout duration and the preset timeout threshold.

4. The method according to claim 1, characterized in that, When the distributed database is expanded using the consistent hashing algorithm, the target physical node includes: the physical node that is closest to the new physical node in a clockwise direction; When the distributed database uses a consistent hashing algorithm for scaling down, the target physical nodes include: physical nodes to be deleted.

5. The method according to claim 1, characterized in that, The processing of new write requests from the client targeting the physical node, and suspended write requests whose feature information reaches a feature threshold, includes: According to the pre-allocation rules, a first number of threads are allocated to the new write request for concurrent processing, and a second number of threads are allocated to the suspended write request for concurrent processing. New write requests from the client are processed concurrently using the first number of concurrent processing threads, and suspended write requests are processed concurrently using the second number of concurrent processing threads.

6. The method according to claim 5, characterized in that, The suspension of write requests from the client to the target physical node includes: Whenever a suspended write request for a target physical node is received from a client, the received suspended write request is placed at the tail of the suspended request queue. During the pause of the expansion or contraction, whenever a new write request for the target physical node is received from a client, the new write request is inserted at the tail of the request processing queue. The restoration of the expanded or reduced capacity includes: Determine whether all write requests in the request processing queue have been processed. If there are unprocessed write requests in the request processing queue, the unprocessed write requests are inserted at the tail of the suspended request queue as suspended write requests.

7. The method according to any one of claims 1 to 6, characterized in that, The expansion / shrinkage conditions include: The unbacked-up index of the target physical node does not exceed a first preset value, and / or the longest transaction processing time of the distributed database within a predetermined time does not exceed a second preset value.

8. A scaling device for a distributed database, characterized in that, include: A memory and a processor, wherein the memory is used to store an executable program; The processor is used to read and execute the executable program to implement the scaling method of the distributed database as described in any one of claims 1-7.

9. A scaling device for a distributed database, characterized in that, include: The scaling unit is used to scale up or down the distributed database using a consistent hashing algorithm when the scaling conditions are met. A suspension unit is used to suspend write requests from clients to a target physical node during the expansion or contraction of the distributed database; wherein the target physical node is a physical node involved in the expansion or contraction of the distributed database; and to obtain characteristic information of the suspended write requests according to a preset strategy. The request processing unit is used to pause the expansion or contraction when the acquired feature information reaches the feature threshold, and to process new write requests from the client for the target physical node, as well as suspended write requests when the feature information reaches the feature threshold, until the expansion or contraction is resumed when the expansion or contraction preparation conditions are met again.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the scaling method for the distributed database as described in any one of claims 1-7.

Images