Distributed ID generation method, device, electronic device and storage medium

Abstract

The present application provides a distributed ID generation method, device, electronic device, and storage medium, relating to the field of computer technology. This method utilizes the CAS algorithm to update a single global memory node for multiple service requests under the same unit timestamp, thereby preempting the latest timestamp recorded by the single global memory node. This generates a different request ID for each service request within the same unit timestamp. Since the CAS algorithm used in this application is a lock-free algorithm, the system can simultaneously process a large number of requests in high-concurrency scenarios, effectively improving system performance compared to the serial ID generation method of the snowflake algorithm.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a distributed ID generation method, device, electronic device, and storage medium. Background Art

[0002] In a traditional stand-alone environment, the database's auto-increment feature can be used to generate a unique identity document (ID) for data inserted into the database. However, in a distributed cluster environment, the same microservice is usually divided into multiple microservice instances deployed and running on different distributed nodes, that is, each node has a microservice instance. In a traditional stand-alone environment, the database's own ID auto-increment feature is often used to generate a globally unique and incrementing ID. However, in a distributed cluster environment, due to load balancing during concurrent requests, multiple distributed nodes may receive requests at the same time, that is, there is a situation where data is inserted into different databases at the same time. Although the database's auto-increment feature can ensure that the ID on the current distributed node is unique and incrementing, after the data is inserted, the synchronization of data between the databases of different distributed nodes will lead to ID conflicts and disorder. At this time, a system that can generate a globally unique ID is very necessary.

[0003] Existing technology generates distributed unique IDs based on a snowflake algorithm, and the generated IDs are generally sorted by time increments. However, in multi-threaded scenarios, to ensure the uniqueness of IDs generated by the snowflake algorithm, critical resources must be locked and protected, specifically the millisecond timestamp and sequence number used for the last generated ID. Threads that fail the competition will be suspended, resulting in serial ID generation and low system performance, making it unsuitable for high-concurrency scenarios that require processing a large number of requests simultaneously. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a distributed ID generation method, device, electronic device and storage medium to improve the problem that the existing snowflake algorithm generates IDs in a serial manner, has low performance, and cannot meet high concurrency scenarios.

[0005] In a first aspect, an embodiment of the present application provides a distributed ID generation method, which is applied to each distributed node, each distributed node including a global memory single node, the global memory single node being used to record the latest timestamp of the latest generated ID and the value of a counter, the counter being used to record the serial numbers of different IDs generated under the same unit timestamp, the method comprising:

[0006] Get multiple service requests within the same unit timestamp;

[0007] If the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, a new global memory single node is created, and the latest timestamp in the new global memory single node is the actual timestamps of the multiple service requests;

[0008] For each service request, update the new global memory single node to the old global memory single node using the compare-and-replace CAS algorithm, and obtain the CAS result corresponding to each service request;

[0009] According to the CAS result corresponding to each service request, obtain the serial number corresponding to each service request, wherein the serial number corresponding to each service request is different;

[0010] A request ID corresponding to each service request is generated according to the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node.

[0011] In the above implementation process, for multiple service requests under the same unit timestamp, the CAS algorithm is used to update the global memory single node to seize the latest timestamp recorded by the global memory single node, so that a different request ID can be generated for each service request within the same unit timestamp. Since the CAS algorithm used in this application is a lock-free algorithm, the system can process a large number of requests in high-concurrency scenarios in parallel at the same time, which effectively improves the system performance compared to the serial ID generation method of the snowflake algorithm.

[0012] Optionally, obtaining a serial number corresponding to each service request according to a CAS result corresponding to each service request includes:

[0013] If the CAS result corresponding to one of the service requests is successful, the default value 0 of the counter recorded in the new global memory single node is obtained as the serial number corresponding to the service request. If the CAS result corresponding to other service requests is failed, the serial number corresponding to each other service request is obtained by competing based on the current value of the counter in the new global memory single node and using the CAS algorithm.

[0014] In the above implementation process, the CAS algorithm is used to compete for serial numbers, so that a large number of requests can quickly obtain their corresponding serial numbers, thereby improving the efficiency of generating IDs.

[0015] Optionally, each service request is configured with a retry flag; if the CAS result corresponding to other service requests is failure, the method further includes:

[0016] The retry flag position corresponding to each other service request is set to a set value, where the set value is used to indicate that the other service request does not need to use the CAS algorithm again to update the new global memory single node to the old global memory single node.

[0017] In the above implementation process, by configuring a corresponding retry flag for each service request, it is possible to avoid the processing thread corresponding to the service request from constantly retrying the operation of updating a single node of global memory using the CAS algorithm, thereby avoiding it from falling into an infinite loop.

[0018] Optionally, after obtaining multiple service requests received with the same unit timestamp and before generating the request ID, the method further includes:

[0019] If the actual timestamps of the multiple service requests are less than or equal to the latest timestamp recorded in the global memory single node, the CAS algorithm is used to compete to obtain the serial number corresponding to each service request, and the latest timestamp in the global memory single node is used as the actual timestamp of the multiple service requests.

[0020] After the above implementation process, if the actual timestamp is less than or equal to the latest timestamp recorded in the global memory single node, it means that a clock rollback problem has occurred in the system. At this time, by using the latest timestamp in the global memory single node as the actual timestamp, the latest timestamp can be used to generate the request ID, avoiding the problem that the actual timestamp is inaccurate due to clock rollback, which may cause the generated ID to be duplicated or smaller than the generated ID.

[0021] Optionally, generating a request ID corresponding to each service request according to an actual timestamp corresponding to each service request, a serial number corresponding to each service request, and an ID of a distributed node includes:

[0022] Using the snowflake algorithm, the actual timestamp of each service request, the serial number of each service request, and the ID of the distributed node are combined to generate a request ID for each service request. This generates a globally unique and incrementing ID.

[0023] Optionally, the request ID corresponding to each service request consists of 53 bits, of which the first bit is a sign bit, which defaults to a positive number. Bits 2-39 represent the actual timestamp corresponding to the service request, bits 40-44 represent the ID of the distributed node, and bits 45-53 represent the sequence number corresponding to the service request. Since the maximum integer value that can be expressed by the JavaScript scripting language is 53 bits, the generated ID is also 53 bits. This avoids the problem of JavaScript precision loss caused by exceeding the maximum value that can be expressed by JavaScript.

[0024] Optionally, the actual timestamps of the multiple service requests are obtained based on a time offset. At system startup, if the maximum timestamp corresponding to the largest ID in the existing data is greater than the current timestamp at startup, the time offset is the value obtained by subtracting the current timestamp from the maximum timestamp plus 1. If the maximum timestamp is less than or equal to the current timestamp, the time offset is 0. This allows time calibration at system startup, thereby accurately determining the actual timestamps of service requests subsequently.

[0025] In a second aspect, an embodiment of the present application provides a distributed ID generation device that runs on each distributed node, each distributed node including a single global memory node, the single global memory node being used to record the latest timestamp of the latest generated ID and the value of a counter, the counter being used to record the serial numbers of different IDs generated under the same unit timestamp, the device comprising:

[0026] The request acquisition module is used to obtain multiple service requests within the same unit timestamp;

[0027] a node creation module, configured to create a new global memory single node if the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, wherein the latest timestamp in the new global memory single node is the actual timestamp of the multiple service requests;

[0028] A node update module is used to update the new global memory single node to the old global memory single node using a compare-and-replace CAS algorithm for each service request, and obtain a CAS result corresponding to each service request;

[0029] A serial number acquisition module is used to obtain the serial number corresponding to each service request according to the CAS result corresponding to each service request, wherein the serial number corresponding to each service request is different;

[0030] The ID generation module is used to generate a request ID corresponding to each service request according to the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node.

[0031] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect above are executed.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0033] Other features and advantages of the present application will be described in the following description and, in part, will become apparent from the description or be understood by practicing the embodiments of the present application. The objectives and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A flowchart of a distributed ID generation method provided in an embodiment of the present application;

[0036] Figure 2 A block diagram of a distributed ID generation device provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the structure of an electronic device for executing a distributed ID generation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application.

[0039] It should be noted that the terms "system" and "network" in the embodiments of the present invention are used interchangeably. "Multiple" refers to two or more. In view of this, in the embodiments of the present invention, "multiple" can also be understood as "at least two." "And / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the related objects are in an "or" relationship.

[0040] To facilitate understanding, the following is a brief explanation of the terms involved in this application.

[0041] Distributed ID: A unique ID that increases globally in a cluster environment;

[0042] CAS (Compare And Swap): Compare and replace, an atomic operation supported by the CPU. This operation is indivisible and guarantees thread safety.

[0043] Clock rollback: refers to the deviation of the system time of the operating system due to unexpected circumstances, such as forward or backward deviation;

[0044] JS precision issue: The maximum integer value that can be expressed in the JS scripting language is 9007199254740992, while the maximum integer value that can be expressed in JAVA can far exceed this value. This will cause the front-end to lose precision after receiving the data.

[0045] Global memory single node: refers to a data node that exists in memory. This node is a single mapping key-value pair, which is a timestamp and a counter.

[0046] Sequence number: used to distinguish IDs generated at the same millisecond in the distributed ID calculation process. That is, the IDs generated at the same millisecond by multiple service requests need to be accumulated to make the IDs different;

[0047] Distributed node ID: used to identify the identity ID between different microservice instances under the same microservice. The value is an integer.

[0048] The embodiment of the present application provides a distributed ID generation method, which is applied to each distributed node, each distributed node includes a global memory single node, the global memory single node is used to record the latest timestamp of the latest generated ID and the value of the counter, the counter is used to record the serial numbers of different IDs generated under the same unit timestamp, when multiple service requests within the same unit timestamp are obtained, if the actual timestamps of these service requests are greater than the latest timestamp recorded in the global memory single node, a new global memory single node is created, and then for each service request, the CAS algorithm is used to update the global memory single node to preempt the latest timestamp recorded by the global memory single node. According to the CAS result corresponding to each service request, the serial number corresponding to each service request can be obtained, and the serial number corresponding to each service request is different, so according to the latest timestamp, serial number and ID of the distributed node in the global memory single node, a different request ID for each service request within the same unit timestamp can be generated. Since the CAS algorithm used in this application is a lock-free algorithm, the system can simultaneously process a large number of requests in high-concurrency scenarios in parallel, which effectively improves system performance compared to the serial ID generation method of the snowflake algorithm.

[0049] Please refer to Figure 1 , Figure 1 A flowchart of a distributed ID generation method provided in an embodiment of the present application, the method comprising the following steps:

[0050] Step S110: Acquire multiple service requests within the same unit timestamp.

[0051] Among them, multiple service requests can refer to multiple service requests under microservices. For example, a cluster contains multiple distributed nodes, and microservices run in these distributed nodes. A distributed node needs to process a large amount of data. In order to uniquely identify this data, different IDs need to be generated for this data. These data can be understood as service requests.

[0052] In the embodiments of the present application, the unit timestamp may be in milliseconds, and multiple service requests are obtained within the same millisecond timestamp. Of course, in actual applications, the timestamp may also be in microseconds or seconds. When the unit is in microseconds, multiple service requests obtained within the same unit timestamp refer to multiple service requests obtained within the same microsecond timestamp. For ease of description, the following embodiments use millisecond timestamps as the unit timestamp for example.

[0053] Step S120: If the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, a new global memory single node is created.

[0054] If multiple service requests are obtained at the same unit timestamp, in order to distinguish the IDs of these multiple service requests, this application first compares the timestamps. The latest timestamp of the latest generated ID is maintained in the global memory single node, that is, the timestamp corresponding to the largest ID among the generated IDs is recorded in the global memory single node (because the ID is increasing, the largest ID is the latest generated ID).

[0055] If the actual timestamps of the multiple service requests currently received are greater than the latest timestamp recorded in the global memory single node, it means that the service requests received at this moment were received within the new millisecond, and the information recorded in the original global memory single node is in the past tense, so a new global memory single node needs to be created, and the latest timestamp in the new global memory single node is the actual timestamp of the multiple service requests.

[0056] For example, if the actual timestamps of multiple service requests currently received are at the 100th millisecond of the first second, and if the latest timestamp recorded in the current global memory node (i.e., the old global memory node) is at the 99th millisecond of the first second, then the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory node. In this case, a new global memory node is created, and the latest timestamp in the new global memory node is at the 100th millisecond of the first second. At this time, the counter value in the new global memory node is 0.

[0057] The reason why the actual timestamps of the service requests are compared here instead of the current timestamps is that there may be a clock rollback problem in the system, making the current timestamp inaccurate. Therefore, a time offset can be determined first, and then the actual timestamp can be determined based on the time offset and the current timestamp.

[0058] For example, when the system starts, the maximum timestamp corresponding to the largest ID in the existing data in the system is obtained. If the maximum timestamp is greater than the current timestamp at startup, it means that there is a deviation in the system time. The time deviation is obtained as the value obtained by subtracting the current timestamp plus 1 from the maximum timestamp. If the maximum timestamp is less than or equal to the current timestamp, the time deviation is 0.

[0059] For example, during the startup phase, the global memory single node is initialized, and the existing data on the distributed nodes is first obtained. If existing data exists, the maximum ID of the current existing data is obtained, and the maximum timestamp corresponding to the maximum ID can be calculated through the reverse operation of the snowflake algorithm. Then, the maximum timestamp corresponding to the maximum ID is compared with the current timestamp at startup. If there is no existing data, the maximum timestamp is 0, the latest timestamp in the initialized global memory single node is 0, and the value of the counter is also 0.

[0060] If the current timestamp at startup is less than the maximum timestamp, the system has experienced a clock rollback. For example, if the current timestamp is the 80th millisecond of the first second, and the maximum timestamp is the 90th millisecond of the first second, then the time deviation = 90 milliseconds - 80 milliseconds + 1 = 11 milliseconds, indicating a system time deviation of 11 milliseconds. At this point, the latest timestamp in the global memory single node is initialized to the maximum timestamp. If the current timestamp is greater than or equal to the maximum timestamp, the system has not experienced a clock rollback, so the time deviation is 0 and the actual timestamp is equal to the current timestamp.

[0061] The system may pre-store a time offset. If a service request is received, the current timestamp at the time of receipt may be added with the time offset to obtain the actual timestamp corresponding to the service request.

[0062] Therefore, by determining the time deviation in advance, an accurate actual timestamp can be obtained, which can solve the problem of inaccurate timestamps obtained due to clock rollback problems in the system, and avoid the situation where the calculated request ID is less than or equal to the ID already existing in the database after the service is restarted. After this processing, the request ID is calculated based on the actual timestamp of the service request, so the request ID generated on each distributed node will not be repeated or less than the ID already existing in the database.

[0063] Step S130: For each service request, the new global memory single node is updated to the old global memory single node using the CAS algorithm, and the CAS result corresponding to each service request is obtained.

[0064] In order to preempt the timestamp, the CAS algorithm is used to update the global memory single node for each service request, that is, each service request corresponds to a processing thread, and each processing thread uses the CAS algorithm to update the new global memory single node to the old global memory single node. Since the CAS algorithm is an atomic operation, an atomic operation refers to an operation that will not be interrupted by the thread scheduling mechanism. That is, once this operation starts, there will be no thread switching operation in the middle until the end of the operation. Therefore, if there are two service requests, the two processing threads can use the CAS algorithm to preempt the operation of updating the global memory single node.

[0065] The CAS algorithm is a lock-free algorithm that can detect interference from other threads. Therefore, it can implement atomic operations even without using locks. Threads that fail to preempt will not be suspended, but will be informed of their failure in the competition and can try again. Therefore, compared to the traditional snowflake algorithm, which needs to generate a request ID each time before processing the next request, and each time a request is processed, other requests are in a locked state (i.e., serial processing), this application uses the CAS algorithm to preempt the operation of updating a single node of global memory in order to preempt the latest timestamp. Multiple service requests can be processed in parallel. If a failure occurs, it only needs to be retried again, which is more efficient and can also ensure that the same ID will not be generated.

[0066] Step S140: Obtain the serial number corresponding to each service request according to the CAS result corresponding to each service request.

[0067] The CAS result corresponding to each service request includes failure or success. If the processing process corresponding to one of the service requests is preempted successfully, the CAS result is success. The CAS results of the processing processes corresponding to the remaining service requests are failure.

[0068] For service requests with successful CAS results, the default value 0 of the counter recorded in the new global memory single node can be obtained as the serial number corresponding to the service request. For other service requests with failed CAS results, the serial numbers corresponding to each other service request can be obtained based on the current value of the counter in the new global memory single node and the CAS algorithm.

[0069] For example, if there are three service requests simultaneously, and the CAS result for service request 1 is successful, while the CAS results for service request 2 and service request 3 are failed, and if the latest timestamp in the new global memory node is the 100th millisecond of the first second and its counter value is 0, service request 1 will prioritize updating the new global memory node to the old global memory node. Therefore, service request 1 competes for the counter value in the new global memory node, and the sequence number corresponding to service request 1 is 0. For service request 2 and service request 3, since the global memory node has already been updated, a comparison shows that the actual timestamps corresponding to service request 2 and service request 3 are equal to the latest timestamp recorded in the new global memory node (of course, their actual timestamps can also be re-compared to see if they are greater than the latest timestamp in the new global memory node). At this time, the two service requests can use the CAS algorithm to compete for the sequence number. If the counter value 0 has been preempted by service request 1, service request 2 and service request 3 need to use the CAS algorithm to preempt the counter value 1. If service request 2 preempts the sequence number 1, the counter value in the global memory node is updated to 1. For service request 3, the value of the counter in the global memory node can be directly increased by 1 as the sequence number of service request 3, that is, the sequence number of service request 3 is 2. Then the value of the counter in the global memory node can be updated to 2. In this way, the three service requests can obtain their own sequence numbers through the CAS algorithm. Of course, if there are more service requests with failed CAS results, the CAS algorithm can be used repeatedly to compete for sequence numbers until the last service request is reached. In this way, each service request can obtain a different sequence number.

[0070] It should be noted that if there are only two service requests, and the CAS result of one of the two service requests is successful, then for the other service request, the serial number can be directly obtained as the value of the counter in the new global memory single node plus 1.

[0071] Step S150: Generate a request ID corresponding to each service request according to the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the distributed node ID.

[0072] When generating a request ID, a snowflake algorithm can be used to generate it. For example, the ID generated in the snowflake algorithm contains 64 bits, the first bit is not used and is used to record positive numbers, the second to the 42nd bits are used to represent the actual timestamp corresponding to the service request (in the above embodiment, the latest timestamp in the new global memory single node is the actual timestamp), the 43rd to the 52nd bits are used to represent the distributed node ID (the distributed node ID is obtained by the micro instance deployed on the distributed node during the startup phase, and the distributed node ID can also uniquely identify a microservice instance under the same microservice), and the 53rd to the 64th bits are used to represent the serial number corresponding to the service request.

[0073] Therefore, for multiple service requests with the same actual timestamp, their distributed node IDs are the same, and only the serial numbers are different. In this way, the generated request IDs are also different and unique.

[0074] In some embodiments, in order to solve the problem of JS accuracy, the number of ID bits generated by the traditional snowflake algorithm is reduced. For example, the first 11 bits of the ID generated by the traditional snowflake algorithm are discarded and only 53 bits are retained. Of course, the request ID can also be generated according to the logic of the traditional snowflake algorithm, but the final generated request ID includes 53 bits, of which the first is the sign bit, which is a positive number by default, the second to the 39th bits (38 bits in total) are used to represent the actual timestamp corresponding to the service request, the 40th to the 44th bits (5 bits in total) are used to represent the ID of the distributed node, and the 45th to the 53rd bits (9 bits in total) are used to represent the serial number corresponding to the service request.

[0075] In this way, under the same millisecond and the same sequence number, the difference in the size of the generated request ID is 512 for every increase of the distributed node ID by 1; under the same millisecond and the same distributed machine ID, the request ID generated is the difference in the sequence number, and the maximum difference is 511; if it is a different millisecond and the same distributed machine ID, the difference in the generated request ID is at least 32 times 512.

[0076] Because the maximum integer value that JS can express is 53 bits, the request ID is reduced to 53 bits in this application so that the obtained ID value does not exceed the maximum number of bits that JS can express, avoiding the problem of loss of precision.

[0077] It should be noted that the request ID on each distributed node is generated in the above manner, so that when each distributed node synchronizes the database, the ID corresponding to each piece of data is different, thus achieving the uniqueness of the ID.

[0078] In the above implementation process, for multiple service requests under the same unit timestamp, the CAS algorithm is used to update the global memory single node to seize the latest timestamp recorded by the global memory single node, so that a different request ID can be generated for each service request within the same unit timestamp. Since the CAS algorithm used in this application is a lock-free algorithm, the system can process a large number of requests in high-concurrency scenarios in parallel at the same time, which effectively improves the system performance compared to the serial ID generation method of the snowflake algorithm.

[0079] On the basis of the above embodiment, when the CAS results corresponding to other service requests are failed, in order to avoid these service requests from continuing to retry the operation of updating the global memory single node and falling into an infinite loop, a retry flag can be configured for each service request in advance. If the CAS results corresponding to other service requests are failed, the retry flag corresponding to each other service request can be set to a set value, which is used to indicate that other service requests do not need to use the CAS algorithm again to update the new global memory single node to the old global memory single node.

[0080] For example, if three service requests are received within the same unit timestamp, the retry flags corresponding to the three service requests are configured first. The initial configuration value of the retry flag can be false. If the CAS result corresponding to service request 1 is successful and the CAS result corresponding to service request 2 and service request 3 is failed, the retry flags corresponding to service request 2 and service request 3 are set to true, and true is the set value. If service request 2 and service request 3 use the CAS algorithm to update the global memory single node again, the retry flag can be judged. If the CAS result is successful or the retry flag is true, the CAS result can be returned. However, since the retry flag is true at this time, no retries will be made anyway, and the latest timestamp and counter of the global memory single node will be directly read for the next calculation.

[0081] Of course, after service request 1 is preempted successfully, the global memory single node is updated. At this time, the actual timestamps of service request 2 and service request 3 are equal to the latest timestamps in the global memory single node, so the CAS algorithm can be directly used to compete for the sequence number.

[0082] The following is a specific example to illustrate the process of generating the request ID.

[0083] For example, assume that microservice A has three microservice instances, deployed on distributed nodes a1, a2, and a3, with one microservice instance deployed on each distributed node. After starting, each of the three microservice instances obtains the ID of its own distributed node and initializes the global memory node on each distributed node.

[0084] If three service requests are load balanced to a1, a2, and a3 at the same time, and the actual timestamps obtained are the same, then the difference in the request IDs corresponding to the three service requests is at least 512. If the actual timestamps obtained are different, then the difference in the request IDs corresponding to the three service requests is the difference in the actual timestamps multiplied by 32 times 512.

[0085] The following example uses a distributed node a1 as an example, which includes the following situations:

[0086] If distributed node a1 receives three service requests at the 100th millisecond of the 1st second (current timestamp), and if the latest timestamp in the current global memory single node is the 99th millisecond of the 1st second, and assuming the time deviation is 0, then the current timestamp is the actual timestamp. At this time, if the actual timestamp is greater than the latest timestamp in the current global memory single node, it means that the service request is within the new timestamp.

[0087] At this point, a new global memory node is created. The latest timestamp in the new global memory node is the 100th millisecond of the 1st second, and the counter value is 0. Then, all three service requests use the CAS algorithm to update the global memory node. If the CAS result of service request 1 is successful, the obtained sequence number is 0. Then, service request 2 and service request 3 compete for the sequence number. If service request 2 obtains sequence number 1 and service request 3 obtains sequence number 2, the request ID of service request 1 is generated based on the actual timestamp: 100th millisecond of the 1st second, distributed node ID: a1, and sequence number 0; the request ID of service request 2 is generated based on the timestamp: 100th millisecond of the 1st second, distributed node ID: a1, and sequence number 1; and the request ID of service request 3 is generated based on the timestamp: 100th millisecond of the 1st second, distributed node ID: a1, and sequence number 2.

[0088] If distributed node a1 receives service request 4 at the 101st millisecond of the 1st second (the current timestamp), and the latest timestamp in the current global memory node is the 100th millisecond of the 1st second (already updated in the above example), and the actual timestamp of service request 4 is greater than the latest timestamp in the global memory node, a new global memory node is created with the latest timestamp of the 101st millisecond of the 1st second, its counter value is 0, and the old global memory node is updated. In this case, the request ID for service request 4 is generated based on the actual timestamp: the 101st millisecond of the 1st second, the distributed node ID: a1, and the sequence number 0.

[0089] If distributed node a1 experiences time regression when obtaining service request 5, the current timestamp is now the 100th millisecond of the 1st second (the pre-calculated time deviation is 0, but a clock rollback problem occurred, so the actual timestamp is not accurate at this time), and the latest timestamp in the current global memory single node is the 101st millisecond of the 1st second (the second example has been updated), and its counter value is 0. The minimum timestamp in the current global memory single node is used as the actual timestamp corresponding to service request 5, and its sequence number is 1 (the value of the counter in the global memory single node is increased by 1). The request ID of service request 5 is generated based on the actual timestamp: 101st millisecond of the 1st second, the ID of the distributed node: a1, and the sequence number 1.

[0090] Similarly, if service request 6 is received at the 100th millisecond of the 1st second, service request 6 and service request 5 will compete for the sequence number using the CAS algorithm. If the sequence number obtained by service request 5 is 1, the sequence number obtained by service request 6 is 2. The request ID of service request 6 is generated based on the actual timestamp: the 101st millisecond of the 1st second, the ID of the distributed node: a1 and the sequence number 2.

[0091] Therefore, the final request ID generated is: request ID of service request 1 < request ID of service request 2 < request ID of service request 3 < request ID of service request 4 < request ID of service request 5 < request ID of service request 6. Therefore, even if clock rollback and concurrent requests occur in the above process, the generated request IDs can ultimately be guaranteed to be increasing and unique.

[0092] Please refer to Figure 2 , Figure 2 This is a block diagram of a distributed ID generation device 200 provided in an embodiment of the present application, which runs on each distributed node. Each distributed node includes a global memory node, which is used to record the latest timestamp of the latest generated ID and the value of the counter. The counter is used to record the serial numbers of different IDs generated under the same unit timestamp. The device 200 can be a module, program segment or code on an electronic device. It should be understood that the device 200 is similar to the above-mentioned Figure 1 The method embodiment corresponds to the embodiment that can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device 200 can be found in the description above. To avoid repetition, detailed description is appropriately omitted here.

[0093] Optionally, the apparatus 200 includes:

[0094] The request acquisition module 210 is used to acquire multiple service requests within the same unit timestamp;

[0095] A node creation module 220 is configured to create a new global memory single node if the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, wherein the latest timestamp in the new global memory single node is the actual timestamp of the multiple service requests;

[0096] The node update module 230 is configured to update the new global memory single node to the old global memory single node using a compare-and-replace (CAS) algorithm for each service request, and obtain a CAS result corresponding to each service request;

[0097] A sequence number acquisition module 240 is configured to acquire a sequence number corresponding to each service request based on the CAS result corresponding to each service request, wherein the sequence number corresponding to each service request is different;

[0098] The ID generation module 250 is configured to generate a request ID corresponding to each service request according to an actual timestamp corresponding to each service request, a serial number corresponding to each service request, and an ID of a distributed node.

[0099] Optionally, the ID generation module 250 is used to obtain the default value 0 of the counter recorded in the new global memory single node as the serial number corresponding to the service request if the CAS result corresponding to one of the service requests is successful; if the CAS result corresponding to other service requests is failed, the serial number corresponding to each other service request is obtained based on the current value of the counter in the new global memory single node and by using the CAS algorithm to compete.

[0100] Optionally, each service request is configured with a retry flag; if the CAS result corresponding to other service requests is a failure, the apparatus 200 further includes:

[0101] A flag reset module is used to set the retry flag position corresponding to each other service request to a set value, and the set value is used to indicate that the other service request does not need to use the CAS algorithm again to update the new global memory single node to the old global memory single node.

[0102] Optionally, the apparatus 200 further includes:

[0103] A timestamp acquisition module is used to use the CAS algorithm to compete to obtain the serial number corresponding to each service request if the actual timestamps of the multiple service requests are less than or equal to the latest timestamp recorded in the global memory single node, and use the latest timestamp in the global memory single node as the actual timestamp of the multiple service requests.

[0104] Optionally, the ID generation module 250 is configured to use a snowflake algorithm to generate a request ID corresponding to each service request by using an actual timestamp corresponding to each service request, a serial number corresponding to each service request, and an ID of a distributed node.

[0105] Optionally, the request ID corresponding to each service request includes 53 bits, where the first bit is a sign bit, which defaults to a positive number, the second to the 39th bits are used to represent the actual timestamp corresponding to the service request, the 40th to the 44th bits are used to represent the ID of the distributed node, and the 45th to the 53rd bits are used to represent the serial number corresponding to the service request.

[0106] Optionally, the actual timestamps of the multiple service requests are obtained based on the time deviation. When the system starts, if the maximum timestamp corresponding to the maximum ID in the existing data is greater than the current timestamp at startup, the time deviation is the value of the maximum timestamp minus the current timestamp plus 1. If the maximum timestamp is less than or equal to the current timestamp, the time deviation is 0.

[0107] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0108] Please refer to Figure 3 , Figure 3 A structural diagram of an electronic device for executing a distributed ID generation method provided in an embodiment of the present application, the electronic device may be a distributed node in the above embodiment, which may include: at least one processor 310, such as a CPU, at least one communication interface 320, at least one memory 330 and at least one communication bus 340. Among them, the communication bus 340 is used to realize direct connection and communication between these components. Among them, the communication interface 320 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 330 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 330 may optionally be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 330. When the computer-readable instructions are executed by the processor 310, the electronic device executes the above Figure 1 The method process shown.

[0109] I understand. Figure 3 The structure shown is only for illustration, and the electronic device may also include Figure 3 More or fewer components than shown, or with Figure 3 Different configurations shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0110] The embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program performs the following operations: Figure 1 The method process in the illustrated method embodiment is executed by the electronic device.

[0111] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining multiple service requests within the same unit timestamp; if the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, creating a new global memory single node, and the latest timestamp in the new global memory single node is the actual timestamp of the multiple service requests; for each service request, using a compare and replace CAS algorithm to update the new global memory single node to the old global memory single node, and obtain a CAS result corresponding to each service request; according to the CAS result corresponding to each service request, obtaining a serial number corresponding to each service request, wherein the serial number corresponding to each service request is different; generating a request ID corresponding to each service request according to the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node.

[0112] In summary, the embodiments of the present application provide a distributed ID generation method, device, electronic device and storage medium. For multiple service requests under the same unit timestamp, the CAS algorithm is used to update the global memory single node to seize the latest timestamp recorded by the global memory single node. In this way, a different request ID can be generated for each service request within the same unit timestamp. Since the CAS algorithm used in this application is a lock-free algorithm, the system can simultaneously process a large number of requests in high-concurrency scenarios, which effectively improves system performance compared to the serial ID generation method of the snowflake algorithm.

[0113] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] In addition, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] Furthermore, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0116] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any actual relationship or order between these entities or operations.

[0117] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A distributed ID generation method, characterized in that: Applied to each distributed node, each distributed node creates a global memory single node, the global memory single node is used to record the latest timestamp of the latest generated ID and the value of the counter, the counter is used to record the serial numbers of different IDs generated under the same unit timestamp, the method includes: Get multiple service requests within the same unit timestamp; If the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, a new global memory single node is created, and the latest timestamp in the new global memory single node is the actual timestamps of the multiple service requests; For each service request, update the new global memory single node to the old global memory single node using the compare-and-replace CAS algorithm, and obtain the CAS result corresponding to each service request; According to the CAS result corresponding to each service request, obtain the serial number corresponding to each service request, where different service requests have different serial numbers; Generate a request ID for each service request based on the actual timestamp, serial number, and ID of the distributed node. Wherein, obtaining the serial number corresponding to each service request according to the CAS result corresponding to each service request includes: If the CAS result corresponding to one of the service requests is successful, the default value 0 of the counter recorded in the new global memory single node is obtained as the serial number corresponding to the service request. If the CAS result corresponding to other service requests is failed, the serial number corresponding to each other service request is obtained by competing based on the current value of the counter in the new global memory single node and using the CAS algorithm.

2. The method according to claim 1, characterized in that Each service request is configured with a retry flag; If the CAS result corresponding to the other service request is failure, the method further includes: The retry flag position corresponding to each other service request is set to a set value, where the set value is used to indicate that the other service request does not need to use the CAS algorithm again to update the new global memory single node to the old global memory single node.

3. The method according to claim 1, characterized in that After obtaining the multiple service requests received with the same unit timestamp and before generating the request ID, the method further includes: If the actual timestamps of the multiple service requests are less than or equal to the latest timestamp recorded in the global memory single node, the CAS algorithm is used to compete to obtain the serial number corresponding to each service request, and the latest timestamp in the global memory single node is used as the actual timestamp of the multiple service requests.

4. The method according to claim 1, wherein Generating a request ID corresponding to each service request according to the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node includes: Using the snowflake algorithm, the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node are combined to generate a request ID corresponding to each service request.

5. The method according to claim 4, characterized in that The request ID corresponding to each service request consists of 53 bits, of which the first is the sign bit, which defaults to a positive number, the second to the 39th bits are used to represent the actual timestamp corresponding to the service request, the 40th to the 44th bits are used to represent the ID of the distributed node, and the 45th to the 53rd bits are used to represent the serial number corresponding to the service request.

6. The method according to any one of claims 1 to 5, characterized in that The actual timestamps of the multiple service requests are obtained based on the time deviation. When the system starts, if the maximum timestamp corresponding to the maximum ID in the existing data is greater than the current timestamp at startup, the time deviation is the value of the maximum timestamp minus the current timestamp plus 1. If the maximum timestamp is less than or equal to the current timestamp, the time deviation is 0.

7. A distributed ID generation device, characterized in that: Running on each distributed node, each distributed node includes a global memory single node, the global memory single node is used to record the latest timestamp of the latest generated ID and the value of the counter, the counter is used to record the serial numbers of different IDs generated under the same unit timestamp, the device includes: The request acquisition module is used to obtain multiple service requests within the same unit timestamp; a node creation module, configured to create a new global memory single node if the actual timestamps of the multiple service requests are greater than the latest timestamp recorded in the global memory single node, wherein the latest timestamp in the new global memory single node is the actual timestamp of the multiple service requests; A node update module is used to update the new global memory single node to the old global memory single node using a compare-and-replace CAS algorithm for each service request, and obtain a CAS result corresponding to each service request; A serial number acquisition module is used to obtain the serial number corresponding to each service request according to the CAS result corresponding to each service request, wherein the serial number corresponding to each service request is different; An ID generation module is used to generate a request ID corresponding to each service request based on the actual timestamp corresponding to each service request, the serial number corresponding to each service request, and the ID of the distributed node; Among them, the serial number acquisition module is specifically used to obtain the default value 0 of the counter recorded in the new global memory single node as the serial number corresponding to the service request if the CAS result corresponding to one of the service requests is successful; if the CAS result corresponding to other service requests is failed, the serial number corresponding to each other service request is obtained based on the current value of the counter in the new global memory single node and the CAS algorithm.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images