Distributed id generation method, distributed id generation device, and storage medium

Abstract

The application discloses a distributed ID generation method, a distributed ID generation device and a storage medium, and the method comprises the following steps: when the number of consumable distributed IDs in a cache queue is less than a preset number, obtaining a node ID corresponding to a current database, and obtaining a historical distributed ID generated last time; obtaining an auto-increment serial number of the historical distributed ID, and a first padding value of a time rollback field; if the current time is later than the time corresponding to the historical distributed ID, assembling a new distributed ID according to the node ID, the first padding value, the auto-increment serial number after auto-increment and the current time; and updating the new distributed ID to the consumable distributed ID in the cache queue. The application compares the time corresponding to the node ID with the time of the historical distributed ID, and then generates a unique new distributed ID based on the comparison result, so that the problem of repeated distributed IDs caused by time rollback is solved.

Description

Technical Field

[0001] This invention relates to the field of distributed systems, and more particularly to a distributed ID generation method, a distributed ID generation device, and a storage medium. Background Technology

[0002] In monolithic application scenarios, the most common approach is to use an auto-incrementing database ID (Identity Document) as a unique ID. However, with the evolution of distributed architectures, after data is sharded and partitioned, distributed applications need to generate globally unique IDs to identify each piece of data. The database's auto-incrementing ID can only guarantee uniqueness within the same database, which clearly cannot meet the requirement of globally unique IDs in distributed systems with multiple databases and application instances. There are several common algorithms for generating distributed unique IDs, such as UUID (Universally Unique Identifier), database or other middleware auto-incrementing algorithms, and the Snowflake algorithm.

[0003] In related distributed unique ID schemes, the distributed IDs generated by the snowflake algorithm depend on the machine's clock. When the device's clock rewinds from the current moment to a previous moment, the timestamp of the generated ID will also rewind. At this time, the generated ID may be the same as the ID generated at the current moment before the clock rewinds, resulting in duplicate distributed IDs and thus the problem of non-unique node IDs in the distributed system.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a distributed ID generation method, a distributed ID generation device, and a storage medium, thereby solving the problem in the prior art where distributed IDs generated based on the snowflake algorithm depend on timestamps, resulting in non-unique distributed IDs.

[0006] To achieve the above objectives, the present invention provides a distributed ID generation method, the method comprising the following steps:

[0007] When the number of consuming distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard, and obtain the historical distributed ID generated last time;

[0008] Obtain the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number;

[0009] If the current time is later than the time corresponding to the historical distributed ID, a new distributed ID is assembled based on the node ID, the first padding value, the auto-incrementing sequence number after auto-incrementing, and the current time;

[0010] Update the newly added distributed ID to the consumable distributed ID in the cache queue.

[0011] Optionally, after obtaining the first padding value of the time rollback field of the historical distributed ID and the random auto-incrementing sequence number, the method further includes:

[0012] If the current time is earlier than the time corresponding to the historical distributed ID, perform an increment operation based on the first padding value to generate a second padding value;

[0013] The newly added distributed ID is assembled based on the node ID, the second fill value, the auto-increment sequence number after auto-increment, and the current time.

[0014] Optionally, the step of generating a second padding value by performing an increment operation based on the first padding value if the current time is earlier than the time corresponding to the historical distributed ID includes:

[0015] If the current time is earlier than the time corresponding to the historical distributed ID, determine the difference between the current time and the time.

[0016] If the difference is greater than a preset threshold, then an increment operation is performed based on the first fill value to generate the second fill value; or

[0017] If the difference is less than the preset threshold, proceed to the step of obtaining the first padding value of the time rollback field of the historical distributed ID and the random auto-incrementing sequence number.

[0018] Optionally, the step of assembling a new distributed ID based on the node ID, the first padding value, the incremented sequence number after the auto-increment, and the current time if the current time is later than the time corresponding to the historical distributed ID includes:

[0019] If the current time is later than the time corresponding to the historical distributed ID, the auto-incrementing sequence number is incremented based on a preset value, and a third padding value is determined for the auto-incrementing sequence number after the auto-incrementing process.

[0020] Obtain the number of digits and the positional notation of the auto-incrementing sequence number, and determine the maximum fill value of the auto-incrementing sequence number based on the number of digits and the positional notation.

[0021] When the third padding value is less than the maximum padding value, a new distributed ID is assembled based on the node ID, the first padding value, the third padding value corresponding to the auto-incrementing sequence number, and the current time, using the snowflake algorithm.

[0022] Optionally, when the number of consuming distributed IDs in the cache queue is less than a preset number, the steps of obtaining the node ID corresponding to the current database shard and obtaining the historical distributed ID generated last time include:

[0023] When the number of consumable distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard and the registration status of the node ID;

[0024] If the registration status is "registered", then obtain the historical distributed ID.

[0025] Optionally, after the step of obtaining the node ID corresponding to the current shard and the registration status of the node ID when the number of consumable distributed IDs in the cache queue is less than the preset number, the method further includes:

[0026] If the registration status is unregistered, determine the number of database shards, and determine the step size of the current database shard based on the number of database shards;

[0027] Based on the step size, after performing the node registration action on the node ID, the historical distributed ID is obtained.

[0028] Optionally, before the steps of obtaining the node ID corresponding to the current database shard and obtaining the previously generated historical distributed ID when the number of consuming distributed IDs in the cache queue is less than a preset number, the method further includes:

[0029] Upon receiving a distributed ID call request, determine the response ID associated with the distributed ID call request from the consumable distributed IDs in the cache queue;

[0030] The response ID is sent to the business data corresponding to the distributed ID call request.

[0031] Optionally, before the step of determining the response ID associated with the distributed ID call request among the consumable distributed IDs in the cache queue when a distributed ID call request is received, the method further includes:

[0032] Determine the tail pointer and cursor pointer of the cache queue, wherein the tail pointer is used to identify the most recently populated distributed ID in the cache queue, and the cursor pointer is used to identify the consumed distributed ID in the cache queue;

[0033] When the tail pointer is at a position after the cursor pointer, or when the cursor pointer is at a position after the tail pointer, the newly added distributed ID or the extraction of the distributed ID is blocked based on a preset rejection policy.

[0034] In addition, to achieve the above objectives, the present invention also provides a distributed ID generation apparatus, which includes a memory, a processor, and a distributed ID generation program stored in the memory and executable on the processor. When the distributed ID generation program is executed by the processor, it implements the steps of the distributed ID generation method as described above.

[0035] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a distributed ID generation program, which, when executed by a processor, implements the steps of the distributed ID generation method as described above.

[0036] This invention provides a distributed ID generation method, a distributed ID generation device, and a storage medium. When the number of consuming distributed IDs in the cache queue is less than a preset number, the method obtains the node ID corresponding to the current database shard, the previously generated historical distributed ID, the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number. If the current time is later than the time corresponding to the historical distributed ID, a new distributed ID is assembled based on the node ID, the first padding value, the auto-incrementing sequence number after auto-incrementing, and the current time. This new distributed ID is then updated to the consuming distributed ID in the cache queue. Based on this, it can be seen that by comparing the time corresponding to the node ID with the time corresponding to the historical distributed ID, if the current time is later than the time corresponding to the historical distributed ID, it indicates that no clock rollback has occurred. Therefore, a unique distributed ID can be assembled based on the node ID, the first padding value of the time rollback field, the auto-incrementing sequence number after auto-incrementing, and the current time, thereby avoiding the problem of duplicate distributed IDs due to time rollback. This ensures that the currently generated distributed ID is unique. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the invention. To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0038] Figure 1This is a flowchart illustrating the first embodiment of the distributed ID generation method of the present invention;

[0039] Figure 2 This is a schematic diagram of the composition structure of the distributed ID in the distributed ID generation method of the present invention;

[0040] Figure 3 This is a schematic diagram of the database-splitting registration of the distributed ID generation method of the present invention;

[0041] Figure 4 This is a flowchart illustrating a second embodiment of the distributed ID generation method of the present invention;

[0042] Figure 5 This is a schematic diagram of the double-ring array structure of the distributed ID generation method of the present invention;

[0043] Figure 6 This is a flowchart illustrating the third embodiment of the distributed ID generation method of the present invention;

[0044] Figure 7 This is a schematic diagram of distributed ID generation according to the fourth embodiment of the distributed ID generation method of the present invention;

[0045] Figure 8 These are schematic diagrams of the terminal hardware structure of various embodiments of the distributed ID generation method of the present invention.

[0046] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] In related distributed unique ID schemes, the distributed IDs generated by the snowflake algorithm depend on the machine's clock. When the device's clock rewinds from the current moment to a previous moment, the timestamp of the generated ID will also rewind. At this time, the generated ID may be the same as the ID generated at the current moment before the clock rewinds, resulting in duplicate distributed IDs and thus the problem of non-unique node IDs in the distributed system.

[0049] To address the aforementioned shortcomings, this invention proposes a distributed ID generation method, the main solution of which includes the following steps:

[0050] When the number of consuming distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard, and obtain the historical distributed ID generated last time;

[0051] Obtain the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number;

[0052] If the current time is later than the time corresponding to the historical distributed ID, a new distributed ID is assembled based on the node ID, the first padding value, the auto-incrementing sequence number after auto-incrementing, and the current time;

[0053] Update the newly added distributed ID to the consumable distributed ID in the cache queue.

[0054] This invention compares the time corresponding to the obtained node ID with the time corresponding to the historical distributed ID. If the current time is later than the time corresponding to the historical distributed ID, it indicates that no clock rollback has occurred. Based on this, a unique distributed ID can be assembled according to the node ID, the first fill value of the time rollback field, the auto-incrementing sequence number after auto-incrementing processing, and the current time. This avoids the problem of duplicate distributed IDs caused by time rollback and ensures that the currently generated distributed ID is unique.

[0055] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0056] First Embodiment

[0057] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the distributed ID generation method of the present invention.

[0058] In this embodiment, the Snowflake algorithm is optimized by adding a clock rollback bit. When a clock rollback occurs, this bit is incremented by one, thus avoiding the clock rollback problem. In other words, the optimized Snowflake algorithm can address the defect of non-unique distributed IDs caused by the current machine's clock rollback issue through the clock rollback bit. Therefore, this embodiment can generate distributed IDs using the optimized Snowflake algorithm.

[0059] For example, the structure of a distributed ID generated based on the optimized Snowflake algorithm can be as follows: Figure 2As shown. This distributed ID is 20 characters long, using a base-36 system (numbers and letters). It includes 3 reserved spaces for a 3-digit application name to distinguish services (can also be customized); 1 clock rollback space to handle machine clock rollback issues; 8 timestamps in milliseconds, using the difference between the current time and the base time, with a maximum capacity of 36 to the power of 8, usable for approximately 89 years; 5 machine IDs, with a maximum capacity of 36 to the power of 5, approximately 60 million nodes; and 3 auto-incrementing sequence numbers, with a maximum capacity of 36 to the power of 3, approximately 40,000 per millisecond.

[0060] In this embodiment, the distributed ID generation method includes the following steps:

[0061] Step S10: When the number of consuming distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard and obtain the historical distributed ID generated last time.

[0062] In this embodiment, the distributed ID generation system is mainly divided into three modules: a registration module, a calculation module, and a caching module. The caching module is primarily responsible for dynamically adjusting the ID generation based on the ID generation and consumption status in real time, ensuring the continuity of ID generation. Consumable distributed IDs refer to unused distributed IDs in the current cache queue. The node ID corresponding to the current database shard is a machine ID registered based on the shard's step size. Since the step size is set according to the number of databases in each shard, the obtained node ID is unique. To obtain the time when the node ID was last obtained, it is also necessary to obtain the historical distributed IDs generated previously.

[0063] For example, the cache queue can be set with 10,000 consumable distributed IDs. When the number of consumable distributed IDs is less than 3,000, i.e., the preset threshold is 3,000, the data in the cache queue needs to be replenished to ensure normal data scheduling. Therefore, it is necessary to obtain the various parameters that assemble the distributed ID, i.e., obtain the node ID corresponding to the current database shard. It should be noted that the above data is for illustrative purposes only and is not intended to limit the invention.

[0064] Specifically, when a distributed ID is needed, the corresponding data can be retrieved from the cache queue of the caching module. That is, the consuming distributed IDs in the cache queue refer to those that can be directly obtained from the cache queue. Based on this, after a distributed ID in the cache queue is used, the number of remaining available IDs needs to be checked. When the number of consuming distributed IDs is less than a preset number, a new distributed ID needs to be generated to fill the empty space in the cache queue. At this time, it is necessary to obtain the node ID, which includes both registered and unregistered states. Further determination is needed based on the registration status of the node ID to determine whether a node registration action needs to be performed. That is, step S10 specifically includes:

[0065] Step S11: When the number of consumable distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard and the registration status of the node ID;

[0066] Step S12: If the registration status is "registered", then obtain the historical distributed ID.

[0067] In this embodiment, when the node ID has been registered, the historical distributed ID can be obtained directly.

[0068] Optionally, after step S11, the following steps are also included:

[0069] Step S13: If the registration status is unregistered, determine the number of database shards and determine the step size of the current database shard based on the number of database shards;

[0070] Step S14: Based on the step size, perform a node registration action on the node ID and then obtain the historical distributed ID.

[0071] In this embodiment, to ensure that the obtained node IDs are in an increasing trend and to ensure the uniqueness of the node IDs, it is necessary to determine the step size and corresponding number of the current sharding based on the number of sub-databases registered in the distributed system, so that the sharding can accurately register the corresponding node IDs.

[0072] To avoid the situation where a single point of failure in the middleware could cause the entire service to become unavailable, this embodiment uses a distributed ID registration method with database sharding. That is, to ensure that no duplicate IDs occur when each sub-database registers its own ID, the number of database shards needs to be determined, and the step size needs to be determined based on this number. For example, if the credit card authorization system CAX has 9 database shards, meaning there are 9 sub-databases, and the corresponding step size is 9, the allocation of node IDs in the distributed ID can be as follows: Figure 3 As shown.

[0073] Please continue to refer to Figure 3 If the current sub-database is CAXSDB01, the corresponding registered node IDs are 1, 10, 19, ..., etc. Therefore, to avoid duplicate distributed IDs, after obtaining the number of sub-databases, the step size can be determined based on the number, and the node IDs corresponding to the distributed IDs to be generated can be registered according to the step size.

[0074] It should be noted that the main server can access the database service only when the application starts to assign unique node IDs to nodes. Furthermore, by using a sharded database registration method, the pressure on the database to generate and allocate distributed IDs is reduced, and there are no database bottlenecks between the various sub-databases undergoing sharded registration. Compared to unified registration in a single database, this also significantly improves data throughput. The above data is for illustrative purposes only and is not intended to limit the scope of this invention.

[0075] Step S20: Obtain the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number;

[0076] In this embodiment, to ensure the uniqueness of the distributed ID, a clock rollback bit is set. When a clock rollback occurs, this bit is incremented by one, thus guaranteeing the uniqueness of the generated distributed ID. Based on this, it is necessary to obtain the current clock rollback count in advance, as this count corresponds to the first padding value of the clock rollback field in the historical distributed IDs. Furthermore, the distributed ID also includes an auto-incrementing sequence number. To avoid high repetition of the auto-incrementing sequence number, which could affect the uniformity of data modulo, a random auto-incrementing sequence number needs to be obtained.

[0077] It's important to note that the auto-incrementing sequence number is a component of the distributed ID. When calculating the sequence number, if the calculation spans milliseconds, the sequence number will mostly start from 0, leading to a large number of distributed IDs with an auto-incrementing sequence number of 0. This can result in uneven distribution of the generated distributed IDs after modulo operation. Therefore, the obtained auto-incrementing sequence number can be set to a random number between 0 and 9 to ensure a more even distribution after processing. Optionally, the auto-incrementing sequence number can also be set to a random number between 0 and Z (a base-36 number consisting of numbers and letters).

[0078] Step S30: If the current time is later than the time corresponding to the historical distributed ID, assemble a new distributed ID based on the node ID, the first padding value, the auto-incrementing sequence number after auto-incrementing, and the current time.

[0079] In this embodiment, when the current time is later than the time corresponding to the historical distributed ID, it means that the current system clock has not backtracked. In other words, the snowflake algorithm can be used directly to generate distributed IDs without worrying about ID duplication.

[0080] Specifically, step S30 includes:

[0081] Step S31: If the current time is later than the time corresponding to the historical distributed ID, increment the auto-incrementing sequence number based on a preset value, and determine the third padding value of the auto-incrementing sequence number after the auto-incrementing process.

[0082] In this embodiment, the incrementing process for the auto-incrementing sequence number can be either incrementing by one or adding other values. After the auto-incrementing process, a distributed ID needs to be assembled based on the incremented values; that is, the third padding value after the auto-incrementing process needs to be determined.

[0083] Step S32: Obtain the number of digits and the carry-based number system of the auto-incrementing sequence number, and determine the maximum fill value of the auto-incrementing sequence number based on the number of digits and the carry-based number system.

[0084] In this embodiment, to avoid the third padding value after auto-incrementing exceeding the maximum padding value, which would cause the data to be returned to zero or the recording to fail, it is necessary to determine the theoretical maximum value corresponding to the auto-incrementing sequence number, and then compare the third padding value with the theoretical maximum value.

[0085] For example, if the number of digits of the auto-incrementing sequence is 5 and the positional numeral system is 36 (a number system with letters), then the theoretical maximum value of the auto-incrementing sequence can be determined to be "ZZZZZ". The value of the auto-incrementing sequence after the auto-incrementing process can be obtained by adding one to the original random number.

[0086] Step S33: When the third padding value is less than the maximum padding value, a new distributed ID is assembled based on the snowflake algorithm according to the node ID, the first padding value, the third padding value corresponding to the auto-incrementing sequence number, and the current time.

[0087] In this embodiment, if the currently obtained third padding value is less than the maximum padding value, it means that the current auto-incrementing sequence number has not exceeded the theoretical storage range of the data. At this time, a new distributed ID can be assembled based on the snowflake algorithm according to the node ID, the first padding value, the third padding value corresponding to the auto-incrementing sequence number, and the current time.

[0088] Specifically, the target distributed ID can be generated using the snowflake algorithm based on the current time, the node ID, the third padding value, and the first padding value. The distributed ID generated by the snowflake algorithm also includes a business reserved space. After obtaining the distributed ID based on business data, corresponding business code can be added to the business reserved space based on specific business code.

[0089] Optionally, if the value of the auto-incrementing sequence number after auto-increment processing is greater than the maximum theoretical value, it means that the obtained random auto-incrementing sequence number is "ZZZZZ". In this case, if this data is incremented by one, it will result in 100000. However, since the current auto-incrementing sequence number is only compatible with five-digit numbers, the resulting six-digit number cannot be applied to the current scheme. In this case, it is necessary to regenerate the auto-incrementing sequence number. That is, when the value is greater than the theoretical maximum value, it is necessary to jump to the step of obtaining the first padding value of the time rollback field of the historical distributed ID and the random auto-incrementing sequence number.

[0090] Step S40: Update the newly added distributed ID to the consumable distributed ID in the cache queue;

[0091] In this embodiment, the generated target distributed ID can be fed back to the cache queue, thereby ensuring that there is still data available in the cache queue. This avoids increasing the load on the sub-database due to insufficient data in the cache queue when a large number of distributed IDs need to be generated, which would otherwise lead to low efficiency in distributed ID generation.

[0092] In the technical solution disclosed in this embodiment, when the number of consuming distributed ID data in the cache queue is less than a preset number, the current time of the currently obtained node ID is compared and verified with the time of the historical distributed ID, thereby avoiding the clock rollback phenomenon, making the generated new distributed ID unique, and having high practicality in high-concurrency scenarios.

[0093] Second Embodiment

[0094] Please refer to Figure 4 In the second embodiment, based on the first embodiment, after step S20, the method further includes:

[0095] Step S50: If the current time is earlier than the time corresponding to the historical distributed ID, perform an increment operation based on the first padding value to generate a second padding value;

[0096] In this embodiment, when the current time is earlier than the time corresponding to the historical distributed ID, it indicates that a time backtracking situation has occurred in the current distributed ID generation system. Based on this, it is necessary to perform corresponding setting processing on the clock rollback bit, that is, to perform auto-increment processing on the obtained first padding value to obtain the second padding value.

[0097] To avoid errors in timestamp acquisition or continuous clock rollback detection within a short period, it is necessary to calculate the difference between the current timestamp and the previous timestamp. This difference can then be used to set the clock rollback bit. Specifically, if the current time is earlier than the time corresponding to the historical distributed ID, the difference between the current time and the previous time is determined. If the difference is greater than a preset threshold, an increment operation is performed based on the first padding value to generate the second padding value. If the difference is less than the preset threshold, the process jumps to obtaining the first padding value of the time rollback field of the historical distributed ID and a random incrementing sequence number, without needing to perform the clock bit setting operation.

[0098] It should be noted that in this embodiment, the number of clock rollback bits is set to 1, meaning a maximum of 35 clock rollbacks are supported. Under normal circumstances, frequent clock rollbacks are unlikely; therefore, the number of clock rollbacks set in this embodiment basically meets the usage requirements. Optionally, in scenarios where clock rollbacks are more frequent, the number of clock rollback bits can be dynamically adjusted according to the scenario.

[0099] Step S60: Assemble the newly added distributed ID based on the node ID, the second fill value, the auto-increment sequence number after auto-increment, and the current time.

[0100] Specifically, the target distributed ID can be generated using the snowflake algorithm based on the current time, the node ID, the auto-incrementing sequence number after the auto-increment, and the second padding value. The distributed ID generated based on the snowflake algorithm also includes a business reserved space. After obtaining the distributed ID based on business data, corresponding business code can be added to the business reserved space based on specific business code.

[0101] In the technical solution disclosed in this embodiment, when the time corresponding to the current timestamp is earlier than the time corresponding to the timestamp of the largest historical machine ID, the difference between the two timestamps is determined, and a judgment is made on whether to set the clock rollback bit based on the difference. Based on this, by setting the clock rollback bit, when a clock rollback occurs, the field filling value corresponding to the clock rollback bit is incremented by one, and then a new distributed ID is generated. At this time, it can be ensured that the currently generated distributed ID has uniqueness, thereby solving the problem of duplicate distributed IDs generated due to clock rollback in the snowflake algorithm and improving the practicality of the distributed ID generation system.

[0102] Third Embodiment

[0103] Please refer to Figure 5 In the third embodiment, based on the first embodiment, before step S10, the following steps are further included:

[0104] Step S70: Determine the tail pointer and cursor pointer of the cache queue, wherein the tail pointer is used to identify the most recently filled distributed ID in the cache queue, and the cursor pointer is used to identify the consumed distributed ID in the cache queue;

[0105] In this embodiment, to ensure high availability of distributed ID generation and throughput in high-concurrency scenarios, it can be based on... Figure 6 The double-ring array queue shown caches and controls the generated IDs.

[0106] For details, please continue to refer to... Figure 6 In the cache queue, Tail pointers and Cursor pointers can be used for reading and writing to the circular array. The Tail pointer represents the maximum sequence number produced by the Producer (starting from 0 and continuously increasing). The Cursor pointer represents the minimum sequence number consumed by the Consumer (the sequence number sequence is the same as the Producer sequence). Because the array memory allocation is contiguous, the double circular array queue fully utilizes the CPU cache for efficient caching of IDs.

[0107] For example, the default capacity of the circular array is 10,000. It can be expanded through configuration to improve the read / write throughput of the cache queue. The two circular arrays are used to store the generated IDs and their states (whether they are populated and whether they are consumable), respectively.

[0108] Step S80: When the tail pointer is at the position after the cursor pointer, or when the cursor pointer is at the position after the tail pointer, the newly added distributed ID or the extraction of the distributed ID is blocked based on a preset rejection policy.

[0109] In this embodiment, please continue to refer to Figure 6 In a double-ring array, the Tail cannot exceed the Cursor, meaning the producer cannot overwrite unconsumed IDs. If the Tail catches up with the Cursor, a rejection policy or alarm can be set. The Cursor cannot exceed the Tail, meaning it cannot consume unproduced IDs. If the Cursor catches up with the tail, a rejection policy or alarm can also be set.

[0110] Step S90: When a distributed ID call request is received, determine the response ID associated with the distributed ID call request among the consumable distributed IDs in the cache queue;

[0111] Step S100: Send the response ID to the business data corresponding to the distributed ID call request.

[0112] In this embodiment, please continue to refer to Figure 6 After the application on the sharded database node starts, a pre-filled circular array needs to be initialized, which means pre-filling the entire queue. When an ID is cached in the cache queue or when a cached ID is consumed, the remaining number of available IDs (i.e., the number of consumed distributed IDs, Tail-Cursor) needs to be checked in real time. If the remaining number of available IDs is less than a set threshold, a distributed ID needs to be generated to fill the idle space. When a call request is received, the response ID associated with the distributed ID call request can be directly sent to the corresponding business processing data.

[0113] In the technical solution disclosed in this embodiment, setting a dual-ring array to ensure the distributed ID of each database shard calling the cache queue can improve calling efficiency. Furthermore, the dual-ring array design can further increase the throughput of the current cache queue, ensuring the stability of the distributed ID throughput even in high-concurrency scenarios.

[0114] Fourth embodiment

[0115] Please refer to Figure 7 In the fourth embodiment, based on any of the above embodiments, an ID generation process for the distributed ID generation method of the present invention is proposed.

[0116] For example, upon receiving a corresponding distributed ID retrieval request, a consumable distributed ID can be directly retrieved from the cache queue. Then, it checks whether the cache queue needs to be replenished with distributed IDs. If not, the process ends directly. If replenishment is needed, the node ID can be retrieved. If the node ID is not registered in the sharded database, registration is performed after determining the step size based on the number of sub-databases. After registration, the current timestamp can be retrieved, and the difference between the current timestamp and the time when the node ID was previously retrieved can be calculated. For millisecond-level time differences, if the current time is less than the previously retrieved time and the difference is less than a threshold, the current time is retrieved again. If the current time is less than the previously retrieved time and the difference is greater than the threshold, the clock rollback bit is set (i.e., incremented by one), and the current time is retrieved again. If the current time is greater than the last time it was obtained, the auto-incrementing sequence number is obtained and incremented by 1. Then, it is determined whether the sequence number is greater than the theoretical maximum value of the auto-incrementing sequence number. If the auto-incrementing sequence number is greater than the theoretical maximum value, the current time is obtained again. Otherwise, a distributed ID is generated and the distributed ID is assembled into the cache queue to ensure the data stability of the cache queue.

[0117] It should be noted that after assembling the ID into the cache queue, when a corresponding distributed ID retrieval request is received, the distributed ID can continue to be retrieved based on the supplemented cache queue. It does not mean that if the ID is assembled into the cache queue and no corresponding request is received, the ID will be directly retrieved from the cache queue.

[0118] In the technical solution disclosed in this embodiment, a detailed process for generating distributed IDs by a distributed ID generation system is provided. Based on the detailed distributed ID generation process, the stability of the current distributed ID generation system and its stability in high-concurrency scenarios are demonstrated.

[0119] Reference Figure 8 , Figure 8 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0120] like Figure 8 As shown, the terminal may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a network interface 1003, and a memory 1004. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0121] Those skilled in the art will understand that Figure 8 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0122] like Figure 8 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a data storage module, a network communication module, and a control program.

[0123] exist Figure 8 In the terminal shown, network interface 1003 is mainly used to connect to the backend server and communicate with it; processor 1001 can call the distributed ID generation program stored in memory 1004 and perform the following operations:

[0124] When the number of consuming distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard, and obtain the historical distributed ID generated last time;

[0125] Obtain the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number;

[0126] If the current time is later than the time corresponding to the historical distributed ID, a new distributed ID is assembled based on the node ID, the first padding value, the auto-incrementing sequence number after auto-incrementing, and the current time;

[0127] Update the newly added distributed ID to the consumable distributed ID in the cache queue.

[0128] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0129] If the current time is earlier than the time corresponding to the historical distributed ID, perform an increment operation based on the first padding value to generate a second padding value;

[0130] The newly added distributed ID is assembled based on the node ID, the second fill value, the auto-increment sequence number after auto-increment, and the current time.

[0131] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0132] If the current time is earlier than the time corresponding to the historical distributed ID, determine the difference between the current time and the time.

[0133] If the difference is greater than a preset threshold, then an increment operation is performed based on the first fill value to generate the second fill value; or

[0134] If the difference is less than the preset threshold, proceed to the step of obtaining the first padding value of the time rollback field of the historical distributed ID and the random auto-incrementing sequence number.

[0135] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0136] If the current time is later than the time corresponding to the historical distributed ID, the auto-incrementing sequence number is incremented based on a preset value, and a third padding value is determined for the auto-incrementing sequence number after the auto-incrementing process.

[0137] Obtain the number of digits and the positional notation of the auto-incrementing sequence number, and determine the maximum fill value of the auto-incrementing sequence number based on the number of digits and the positional notation.

[0138] When the third padding value is less than the maximum padding value, a new distributed ID is assembled based on the node ID, the first padding value, the third padding value corresponding to the auto-incrementing sequence number, and the current time, using the snowflake algorithm.

[0139] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0140] When the number of consumable distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard and the registration status of the node ID;

[0141] If the registration status is "registered", then obtain the historical distributed ID.

[0142] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0143] If the registration status is unregistered, determine the number of database shards, and determine the step size of the current database shard based on the number of database shards;

[0144] Based on the step size, after performing the node registration action on the node ID, the historical distributed ID is obtained.

[0145] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0146] Upon receiving a distributed ID call request, determine the response ID associated with the distributed ID call request from the consumable distributed IDs in the cache queue;

[0147] The response ID is sent to the business data corresponding to the distributed ID call request.

[0148] Furthermore, the processor 1001 can call the distributed ID generation program stored in the memory 1004 and also perform the following operations:

[0149] Determine the tail pointer and cursor pointer of the cache queue, wherein the tail pointer is used to identify the most recently populated distributed ID in the cache queue, and the cursor pointer is used to identify the consumed distributed ID in the cache queue;

[0150] When the tail pointer is at a position after the cursor pointer, or when the cursor pointer is at a position after the tail pointer, the newly added distributed ID or the extraction of the distributed ID is blocked based on a preset rejection policy.

[0151] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the control terminal to implement the process steps of the embodiments of the above methods.

[0152] Therefore, the present invention also provides a computer-readable storage medium storing a distributed ID generation program, which, when executed by a processor, implements the various steps of the distributed ID generation method described in the above embodiments.

[0153] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0154] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0159] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0160] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0161] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A distributed ID generation method, characterized in that, The distributed ID generation method includes: Determine the tail pointer and cursor pointer of the cache queue, wherein the tail pointer is used to identify the most recently populated distributed ID in the cache queue, and the cursor pointer is used to identify the consumed distributed ID in the cache queue; When the tail pointer is at the position after the cursor pointer, or when the cursor pointer is at the position after the tail pointer, the newly added distributed ID or the extraction of the distributed ID is blocked based on a preset rejection policy; Upon receiving a distributed ID call request, determine the response ID associated with the distributed ID call request from the consuming distributed IDs in the cache queue; Send the response ID to the business data corresponding to the distributed ID call request; When the number of consuming distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard, and obtain the historical distributed ID generated last time; Obtain the first padding value of the time rollback field of the historical distributed ID, and a random auto-incrementing sequence number, wherein the number of clock rollbacks in the system corresponds to the first padding value; If the current time is later than the time corresponding to the historical distributed ID, the auto-incrementing sequence number is incremented based on a preset value, and a third padding value is determined for the auto-incrementing sequence number after the auto-incrementing process. Obtain the number of digits and the positional notation of the auto-incrementing sequence number, and determine the maximum fill value of the auto-incrementing sequence number based on the number of digits and the positional notation. When the third padding value is less than the maximum padding value, a new distributed ID is assembled based on the node ID, the first padding value, the third padding value corresponding to the auto-incrementing sequence number, and the current time, using the snowflake algorithm. The distributed ID is a 20-character hexadecimal number. If the current time is earlier than the time corresponding to the historical distributed ID, perform an increment operation based on the first padding value to generate a second padding value; The newly added distributed ID is assembled based on the node ID, the second padding value, the incremented sequence number after the auto-increment, and the current time. Update the newly added distributed ID to the consumable distributed ID in the cache queue.

2. The distributed ID generation method as described in claim 1, characterized in that, The step of generating a second filler value by performing an increment operation based on the first filler value if the current time is earlier than the time corresponding to the historical distributed ID includes: If the current time is earlier than the time corresponding to the historical distributed ID, determine the difference between the current time and the time. If the difference is greater than a preset threshold, then an increment operation is performed based on the first fill value to generate the second fill value; or If the difference is less than the preset threshold, proceed to the step of obtaining the first padding value of the time rollback field of the historical distributed ID and the random auto-incrementing sequence number.

3. The distributed ID generation method as described in claim 1, characterized in that, When the number of consuming distributed IDs in the cache queue is less than a preset number, the steps of obtaining the node ID corresponding to the current database shard and obtaining the historical distributed ID generated last time include: When the number of consumable distributed IDs in the cache queue is less than the preset number, obtain the node ID corresponding to the current database shard and the registration status of the node ID; If the registration status is "registered", then obtain the historical distributed ID.

4. The distributed ID generation method as described in claim 3, characterized in that, After the step of obtaining the node ID corresponding to the current database shard and the registration status of the node ID when the number of consumable distributed IDs in the cache queue is less than the preset number, the method further includes: If the registration status is unregistered, determine the number of database shards, and determine the step size of the current database shard based on the number of database shards; Based on the step size, after performing the node registration action on the node ID, the historical distributed ID is obtained.

5. A distributed ID generation device, characterized in that, The distributed ID generation device includes: a memory, a processor, and a distributed ID generation program stored in the memory and executable on the processor. When the distributed ID generation program is executed by the processor, it implements the steps of the distributed ID generation method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a distributed ID generation program, which, when executed by a processor, implements the steps of the distributed ID generation method as described in any one of claims 1 to 4.

Images