Global unique identifier generation method and system, electronic equipment and storage medium
By rewriting the self-increment command in the persistent file and generating the worker node identifier when the issuer is restarted, combining the timestamp and serial number, the shortcomings of the distributed global unique ID generation method are solved, and efficient and reliable ID generation is achieved , improves performance and reliability.
Patent Information
- Application Number
- CN202411085322.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-22
AI Technical Summary
The existing distributed global unique ID generation methods are insufficient in functionality, spatial structure, security, and operation and maintenance costs, and it is difficult to meet the immediacy and high concurrency requirements in the big data environment.
By reading and rewriting the self-increment command in the persistent file when the issuer is restarted, the worker node identifier is generated, and a globally unique identifier is generated based on the current timestamp and serial number, avoiding duplication problems and reducing restart time consumption, combined with Redis and the characteristics of the snowflake algorithm ensure the reliability and distributed characteristics of the ID.
It effectively avoids the duplication of global unique identifiers, reduces the consumption of restart time, improves the sender performance, and ensures the reliability and distributed characteristics of the generated global unique identifiers.
Smart Images

Figure CN119011532A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of distributed data processing technology, and in particular to a method, system, electronic device and storage medium for generating a globally unique identifier. Background Art
[0002] With the rapid development of information technology, distributed systems have become the cornerstone of supporting big data processing, cloud computing services and high-concurrency Internet applications. However, with the continuous expansion of system scale and the growing user demand, the design and implementation of distributed systems are facing many challenges. Traditional ID (Identity) generation strategies, such as database auto-increment or simple random generation, can no longer meet the requirements of immediacy, high concurrency and distributed deployment in big data environments. Therefore, efficient and reliable generation of distributed globally unique IDs is a key link to ensure data consistency and system scalability.
[0003] In the related technologies, there are mainly the following technologies for generating distributed globally unique IDs: 1. Java UUID (Universally Unique Identifier) generation, which is generated through Java API (Java Application Programming Interface), but UUID occupies 16 characters, occupies more space and has low indexing efficiency; 2. Database primary key auto-increment, using the characteristics of relational database management system to generate ID, but limited by database performance, the concurrency performance is not high, and it is easy to cause data leakage; 3. Snowflake algorithm, which generates ID through multi-node deployment, but there are also clock dial-back problems and ID conflicts; 4. Meituan Leaf, which uses open source framework performance to generate ID in a pre-distributed manner, but the cost of use and deployment is high and the framework is complex. In summary, the existing distributed globally unique ID generation methods have deficiencies in functionality, spatial structure, security, and operation and maintenance costs.
[0004] Therefore, it is necessary to propose a distributed globally unique ID generation solution that is more efficient, reliable, easy to deploy and maintain. Summary of the invention
[0005] In view of the above-mentioned shortcomings of the prior art, the present application discloses a method, system, device and storage medium for generating a globally unique identifier, which are used to solve at least one of the above-mentioned technical problems.
[0006] In the first aspect, the present application provides a method for generating a globally unique identifier, the method comprising: if a restart of a number generator is detected, reading the auto-increment command in a persistent file and rewriting it, the persistent file recording all the auto-increment commands executed by the number generator; generating a working node identifier based on the rewriting result, and obtaining a current first timestamp and a first serial number, wherein the first serial number is generated based on the first timestamp, and the rewriting result includes a previous working node identifier; combining the first timestamp, the first serial number and the working node identifier to generate a globally unique identifier.
[0007] In one embodiment of the present application, before reading the self-increment command in the persistent file and rewriting it, it also includes: recording all write operation commands in the issuer to generate the persistent file, wherein the write operation command includes the self-increment command; constructing a self-increment command rewrite function, and setting the execution condition of the self-increment command rewrite function, wherein the execution element includes restarting the issuer.
[0008] In one embodiment of the present application, the reading of the auto-increment command in the persistent file and rewriting it includes: calling the auto-increment command rewrite function, traversing each of the write operation commands in the persistent file, and detecting whether there is a preset field in each of the write operation commands, wherein the preset field includes a key field and an auto-increment command identification field; if the preset field exists in any write operation command, and the key field is a string field, then the arbitrary write operation command is added to a pre-built auto-increment command set, and all the auto-increment commands in the auto-increment command set are rewritten.
[0009] In one embodiment of the present application, generating a working node identifier based on the rewriting result includes: determining a first key value of the key in the numbering device based on the previous working node identifier, wherein the data type of the previous working node identifier is the string field; adding a preset auto-increment step size to the first key value to obtain a second key value; writing the second key value back to the key to refresh the first key value, and using the second key value as the working node identifier.
[0010] In one embodiment of the present application, the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier, including: obtaining a globally unique identifier structure, the globally unique identifier structure including a positive integer identifier field, a timestamp field, a working node identifier field and a serial number field for distinguishing multiple different globally unique identifiers generated in the same timestamp, wherein the working node identifier field is a string field; according to the globally unique identifier structure, the first timestamp, the first serial number and the working node identifier are combined to generate the globally unique identifier.
[0011] In one embodiment of the present application, after the first timestamp is obtained, it also includes: obtaining the second timestamp stored in the number generator when the last globally unique identifier was generated; comparing the second timestamp with the first timestamp; if the second timestamp is greater than the first timestamp, an alarm is issued for failure to generate the globally unique identifier; if the second timestamp is less than or equal to the first timestamp, the generation operation of the first serial number is entered, and the first timestamp is stored in the number generator, and the second timestamp is refreshed.
[0012] In one embodiment of the present application, the method for generating the first serial number includes: if the second timestamp is less than the first timestamp, the first serial number is generated from 0; if the second timestamp is equal to the first timestamp, the second serial number stored in the generator when the previous global unique identifier was generated is obtained, and the second serial number is compared with a preset maximum serial number; if the second serial number is less than the maximum serial number, one is added to the second serial number to generate the first serial number; if the second serial number is equal to the maximum serial number, the first serial number is generated from 0 when the next timestamp is waited for.
[0013] In the second aspect, the present application provides a globally unique identifier generation system, the system comprising: a rewrite module, for reading the auto-increment command in the persistent file and rewriting it if a restart of the issuer is detected, the persistent file recording all the auto-increment commands executed by the issuer; a first generation module, for generating a working node identifier based on the rewrite result, and obtaining a current first timestamp and a first serial number, wherein the first serial number is generated based on the first timestamp, and the rewrite result includes a previous working node identifier; a second generation module, for combining the first timestamp, the first serial number and the working node identifier to generate a globally unique identifier.
[0014] In a third aspect, the present application provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the globally unique identifier generation method described in the first aspect.
[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the globally unique identifier generation method described in the first aspect.
[0016] As described above, the embodiments of the present application provide a method, system, electronic device, and storage medium for generating a globally unique identifier, which have the following beneficial effects:
[0017] By persisting the self-increment command, when the restart of the number generator is detected, only the self-increment command in the persistence file is read and rewritten, and then the working node identifier is generated according to the rewriting result, and the current first timestamp and the first serial number are obtained. The first serial number is generated according to the first timestamp. The rewriting result includes the previous working node identifier, and finally the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier. After the number generator hangs up and restarts, only the self-increment command is rewritten, which not only avoids the problem of duplicate global unique identifiers, but also reduces the consumption of restart time and improves the performance of the number generator. In addition, the distributed characteristics of the number generator and the global uniqueness of the snowflake algorithm are combined to ensure the reliability of the generated global unique identifier.
[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0020] Figure 1 It is a schematic diagram of an implementation environment of a globally unique identifier generation system shown in an exemplary embodiment of the present application;
[0021] Figure 2 is a flowchart of a method for generating a globally unique identifier shown in an exemplary embodiment of the present application;
[0022] Figure 3 It is a structural schematic diagram of a globally unique identifier generation system shown in an exemplary embodiment of the present application;
[0023] Figure 4 is a flowchart of a method for generating a globally unique identifier based on clock dialback, shown in an exemplary embodiment of the present application;
[0024] Figure 5 is a block diagram of a globally unique identifier generation system shown in an exemplary embodiment of the present application;
[0025] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0026] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.
[0027] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0028] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.
[0029] Distributed systems have become the cornerstone of supporting big data processing, cloud computing services and high-concurrency Internet applications. However, with the continuous expansion of system scale and the growing user demand, the design and implementation of distributed systems are facing many challenges. At present, there are mainly the following technologies for generating distributed globally unique IDs: 1. Java UUID generation, generated through Java API, but UUID occupies 16 characters, occupies more space and has low indexing efficiency; 2. Database primary key auto-increment, using the characteristics of relational database management system to generate ID, but limited by database performance, the concurrency performance is not high, and it is easy to cause data leakage; 3. Snowflake algorithm, generates ID through multi-node deployment, but there are also clock dialback problems and ID conflicts; 4. Meituan Leaf, using open source framework performance, generates ID in a pre-distributed manner, but the use and deployment costs are high and the framework is complex. Therefore, the existing distributed globally unique ID generation methods have deficiencies in functionality, spatial structure, security, and operation and maintenance costs.
[0030] Therefore, see Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a globally unique identifier generation system shown in an exemplary embodiment of the present application. Figure 1As shown, the implementation environment may include a globally unique identifier generation system 110 and a computer device 120. The globally unique identifier generation system 110 may be set in the computer device 120 to generate a globally unique identifier. The computer device 120 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The globally unique identifier generation system 110, by persisting the self-increment command, only rewrites the self-increment command after the number generator hangs up and restarts, which not only avoids the problem of duplicate globally unique identifiers, but also reduces the consumption of restart time, improves the performance of the number generator, and combines the distributed characteristics of the number generator and the global uniqueness of the snowflake algorithm to ensure the reliability of the generated globally unique identifier.
[0031] See also Figure 2 , Figure 2 is a flowchart of a method for generating a globally unique identifier as shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is a schematic diagram. It should be understood that the method may also be applicable to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.
[0032] like Figure 2 As shown, in an exemplary embodiment, the method for generating a globally unique identifier includes at least steps S210 to S230, which are described in detail as follows:
[0033] Step S210: If it is detected that the number generator is restarted, the self-increment command in the persistent file is read and rewritten. The persistent file records all the self-increment commands executed by the number generator.
[0034] Among them, the number generator is the Redis number generator, and the generation efficiency of WorkId (i.e., work node identifier) is high. By changing the underlying logic of the persistent rewriting of Redis's auto-increment command, the non-repetitive and consistent acquisition of WorkId is guaranteed, and Redis resources are not excessively occupied. The Redis algorithm is simple and there is no problem of data leakage.
[0035] Redis persistence refers to saving the data in Redis memory to the hard disk to prevent data loss after a failure and restart. Redis persistence supports two persistence methods: RDB (Redis Database Backup, snapshot persistence) and AOF (AppendOnly File, incremental persistence). RDB persistence generates a data snapshot of the data in memory and saves it as an RDB file on the hard disk. When Redis restarts, the file can be read to restore the data; AOF persistence records all write operations in a separate log file and stores them in the form of commands. When Redis restarts, these commands will be re-executed to restore the data, which can ensure that the data is not lost.
[0036] In addition, the INCR command (i.e., the auto-increment command) is a command in Redis that is used to atomically increase the value stored in Redis. It allows the auto-increment operation to be performed on a specific key, generally increasing the integer value corresponding to the key by 1, and returning the increased value.
[0037] In the embodiment of the present application, the persistence file is an AOF file, which is equivalent to persisting each write operation command (including INCR command). If Redis hangs up and restarts, it can ensure that WorkId will not be repeated, and only the INCR command is rewritten without considering other write operation commands. It can effectively reduce the restart recovery time of WorkId data and improve Redis performance.
[0038] In one embodiment, before reading the auto-increment command in the persistent file and rewriting it, it also includes: recording all write operation commands in the issuer to generate a persistent file, wherein the write operation commands include the auto-increment command; constructing an auto-increment command rewrite function, and setting the execution conditions of the auto-increment command rewrite function, wherein the execution conditions include restarting the issuer.
[0039] It should be noted that the source code file for processing AOF files in Redis is aof.c file, and its functions include: 1. Opening and closing AOF files: including the function aofOpen for opening AOF files, and aofClose for closing AOF files; 2. AOF buffer: Redis uses a buffer to temporarily save commands to be written to the AOF file, and appends commands to the buffer through the function aofRewriteBufferAppend; 3. Writing AOF files: write the contents of the buffer to the AOF file through the function flushAppendOnlyFile; 4. Rewriting AOF files: Redis will rewrite AOF files regularly to reduce the size of AOF files and improve reading efficiency, and rewrite AOF files through the function feedAppendOnlyFile; 5. Loading AOF files: When Redis starts, it will read commands from the AOF file and restore the data to memory, and perform this task through the function loadAppendOnlyFile.
[0040] In an embodiment of the present application, when configuring Redis, the AOF rewrite logic is modified: AOF persistence involves recording each write operation command executed. During the AOF rewrite process, Redis will create a new AOF file that only contains the commands required for the current database state; insert custom logic: in the feedAppendOnlyFile function, insert custom logic to check each command, and for the INCR command, check whether its key has a specific prefix or contains specific parameters; compile and deploy: through the adjustment of AOF persistence and related configurations, it is ensured that WorkId generation is coherent and non-repetitive in various abnormal situations.
[0041] In this embodiment, the seq_id value (incrementing sequence number) and step (step length) in the Redis node are first initialized, and then the INCR command is used to obtain a different self-incrementing WorkId for each request. When Redis works normally, all write operation commands will be recorded and an AOF file will be generated, that is, the INCR command is persisted in time when the WorkId is allocated, and in the persistence configuration of Redis, in addition to the above configuration, a self-increment command rewrite function is constructed, and the execution condition of the self-increment command rewrite function is set, that is, the restart of the number generator, so that when the restart of the number generator is detected, only the self-increment command in the persistence file is read and rewritten. In this way, the restart recovery time of the WorkId data is effectively reduced, and the performance of Redis is improved.
[0042] Specifically, the auto-increment command in the persistence file is read and rewritten, including: calling the auto-increment command rewrite function, traversing each write operation command in the persistence file, and detecting whether there is a preset field in each write operation command, wherein the preset field includes a key field and an auto-increment command identification field; if there is a preset field in any write operation command, and the key field is a string field, then any write operation command is added to a pre-built auto-increment command set, and all auto-increment commands in the auto-increment command set are rewritten.
[0043] It should be noted that in the embodiment of the present application, Redis and the snowflake algorithm are combined to generate a globally unique identifier. Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a globally unique identifier generation system shown in an exemplary embodiment of the present application. Figure 3 As shown in the figure, a Redis self-increasing ID generator is built, and the ID is used as WorkId. The timestamp and serial number are generated in combination with SnowFakeService (i.e., snowflake algorithm) to form a globally unique identifier. When the business logic layer provides services, the system load generates a globally unique identifier to identify each request or transaction.
[0044] In the embodiment of the present application, the snowflake algorithm is improved, that is, the structure of the global unique identifier generated by the traditional snowflake algorithm is a Long data structure, which is a 64-bit identifier, and its highest 1 bit is fixed to 0, indicating that the generated global unique identifier is a positive integer; the next 41 bits store the millisecond timestamp; the next 10 bits store the machine code, including a 5-bit machine ID and a 5-bit service ID, and a maximum of 2^10=1024 machines can be deployed; the last 12 bits store the serial number, and the same millisecond timestamp is distinguished by this incremental serial number, that is, for the same machine, under the same millisecond timestamp, 2^12=4096 non-repeating global unique identifiers can be generated, where WorkId=machine ID+service ID. The data structure of the global unique identifier will be improved, and WorkId is no longer allocated based on the machine node as the dimension. Therefore, each time the service restarts, the service node must get a new WorkId, so WorkId can no longer be limited to 10 bits, and will be upgraded to a String data structure, and other fields remain unchanged.
[0045] In this way, by improving the binary data type and increasing the number of bits of WorkId, WorkId can be infinitely expanded while not taking up too much space. One more bit can allocate 1024 more machines or 1024 more globally unique identifiers. Therefore, there is no need to design complex allocation algorithm logic for the allocation of WorkId. Each time the service is restarted, a new WorkId is directly obtained, which can ensure distributed global uniqueness and greatly reduce the complexity of the generator. The generation of the new WorkId is combined with Redis. Considering that Redis is basically used in every cloud service, Redis is used to build the number generator, and some commonly used component service resources are used to build it, which can save resource costs. Of course, based on your own business, other easy-to-use components can also be selected. In addition, Redis and the snowflake algorithm are combined to use Redis to manage the allocation of WorkId in the snowflake algorithm, ensuring that non-repetitive globally unique identifiers can be correctly generated even in a cluster environment. The distributed characteristics of Redis and the global uniqueness of the snowflake algorithm are combined to ensure the reliability of distributed ID generation.
[0046] Therefore, in this embodiment, considering that the WorkId in the generated global unique identifier is a string field, the key field in the self-increment command is also a string field. Therefore, in the process of reading the self-increment command in the persistent file and rewriting it, in addition to detecting whether the key field and the self-increment command identification field exist in each write operation command, it is also necessary to determine whether the key field is a string field. In this way, the reliability of the self-increment command rewrite is guaranteed.
[0047] Step S220, generating a working node identifier according to the rewriting result, and obtaining a current first timestamp and a first serial number, wherein the first serial number is generated according to the first timestamp, and the rewriting result includes a previous working node identifier.
[0048] It should be noted that the INCR command rewrite is to restore the WorkId data, so the rewrite result includes a generated WorkId. In addition, if an identifier generation request is received, the current timestamp will be obtained and then a serial number will be generated based on the timestamp.
[0049] Specifically, a working node identifier is generated according to the rewriting result, including: determining a first key value of a key in a number generator according to a previous working node identifier, wherein the data type of the previous working node identifier is a string field; increasing a preset auto-increment step size on the basis of the first key value to obtain a second key value; writing the second key value back to the key to refresh the first key value, and using the second key value as the working node identifier.
[0050] In this embodiment, the first key value of the key in the number generator is determined according to the previous WorkId, and the INCR command is executed to perform self-increment, wherein the self-increment step can be set. Generally, the self-increment step is 1. In addition, the key value needs to be refreshed. When the next WorkId is generated, the key value is self-incremented on the refreshed key value.
[0051] In one embodiment, after the first timestamp is obtained, it also includes: obtaining the second timestamp stored in the number generator when the last globally unique identifier was generated; comparing the second timestamp with the first timestamp; if the second timestamp is greater than the first timestamp, an alarm is issued for failure to generate the globally unique identifier; if the second timestamp is less than or equal to the first timestamp, entering the first serial number generation operation, and storing the first timestamp in the number generator, and refreshing the second timestamp.
[0052] In this embodiment, the current first timestamp is compared with the stored second timestamp. If the second timestamp is greater than the first timestamp, a clock dial-back problem has occurred. If the globally unique identifier continues to be generated, the generated globally unique identifier will have problems. Therefore, a failure alarm or exception processing is performed. Only when the second timestamp is less than or equal to the first timestamp, only the identifier generation operation needs to be performed. In this way, the clock dial-back problem of the generated globally unique identifier is avoided. In addition, the first timestamp is stored in the number generator and the second timestamp is refreshed to ensure the reliability of the next identifier generation.
[0053] Furthermore, the method for generating the first serial number includes: if the second timestamp is less than the first timestamp, the first serial number is generated from 0; if the second timestamp is equal to the first timestamp, the second serial number stored in the generator when the last global unique identifier was generated is obtained, and the second serial number is compared with a preset maximum serial number; if the second serial number is less than the maximum serial number, one is added to the second serial number to generate the first serial number; if the second serial number is equal to the maximum serial number, the first serial number is generated from 0 when the next timestamp is waited for.
[0054] In this embodiment, the sequence number represents multiple different globally unique identifiers generated within the same millisecond timestamp. If the second timestamp is less than the first timestamp, it indicates an identifier generated for the next millisecond timestamp, and the first sequence number is generated starting from 0. If the second timestamp is equal to the first timestamp, it indicates an identifier generated by the same timestamp. Since the sequence number has a bit limit, there is a maximum sequence number for the identifier within the same timestamp. Therefore, the second sequence number is compared with the maximum sequence number. If the second sequence number is less than the maximum sequence number, the first sequence number can also be generated by adding one based on the second sequence number. If the second sequence number is equal to the maximum sequence number, the sequence numbers within the timestamp have been arranged, and it is necessary to wait until the next timestamp when the first sequence number is generated starting from 0. In this way, the reliability of the generated globally unique identifier is guaranteed.
[0055] Step S230: Combine the first timestamp, the first sequence number and the working node identifier to generate a globally unique identifier.
[0056] In an embodiment of the present application, the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier, and there is no restriction on their order.
[0057] Specifically, the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier, including: obtaining a globally unique identifier structure, the globally unique identifier structure including a positive integer identifier field, a timestamp field, a working node identifier field and a serial number field for distinguishing multiple different globally unique identifiers generated in the same timestamp, wherein the working node identifier field is a string field; according to the globally unique identifier structure, the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier.
[0058] It should be noted that the positive integer identification field can be a 1-bit field representing a globally unique identifier as a positive integer; the timestamp field can be a 41-bit field representing a millisecond timestamp; the working node identification field can be a string field representing the working node identifier, and the string field supports unlimited expansion; the serial number field can be a 12-bit field representing multiple different globally unique identifiers generated within the same millisecond timestamp.
[0059] In this embodiment, a globally unique identifier is generated by combining a work node identifier of a string field, which reduces the complex allocation algorithm logic in the design of WorkId allocation and greatly reduces the complexity of the generator.
[0060] See also Figure 4 , Figure 4 FIG. 1 is a flowchart of a method for generating a globally unique identifier based on clock dialback, which is shown in an exemplary embodiment of the present application. Figure 4As shown, first, start the SnowFlake Service; then use WorkId as the Key to read the timestamp of the previous periodic upload from Redis. If it is empty, the current time is used by default; then determine whether the cached timestamp is greater than the current time of the local machine; if the cached timestamp is greater than the current time of the local machine, it means that a clock dial-back problem has occurred. At this time, a failure alarm / exception processing is performed. The specific exception processing process can be arranged according to its own business; if it is not greater than, a globally unique identifier is generated according to the process; then the current time value is stored in the local service cache to replace the previously cached value; finally, determine whether there is a new request to generate a globally unique identifier. If so, continue to obtain and determine the timestamp to avoid the clock dial-back problem. Even if the service crashes and the clock is reset after restart, the latest time can be pulled from the Redis cache to determine whether a clock dial-back problem has occurred.
[0061] The above-mentioned globally unique identifier generation method persists the self-increment command. When the restart of the number generator is detected, only the self-increment command in the persistent file is read and rewritten, and then the working node identifier is generated according to the rewriting result, and the current first timestamp and the first serial number are obtained. The first serial number is generated according to the first timestamp, and the rewriting result includes the previous working node identifier. Finally, the first timestamp, the first serial number and the working node identifier are combined to generate a globally unique identifier. After the number generator hangs up and restarts, only the self-increment command is rewritten, which not only avoids the problem of duplicate globally unique identifiers, but also reduces the consumption of restart time and improves the performance of the number generator. In addition, the distributed characteristics of the number generator and the global uniqueness of the snowflake algorithm are combined to ensure the reliability of the generated globally unique identifier.
[0062] See also Figure 5 , Figure 5 is a block diagram of a globally unique identifier generation system shown in an exemplary embodiment of the present application. The system can be applied to Figure 1 The implementation environment shown is a schematic diagram. It should be understood that the system may also be applicable to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the system is applicable.
[0063] like Figure 5 As shown, in an exemplary embodiment, the globally unique identifier generation system 500 includes at least a rewriting module 510, a first generation module 520 and a second generation module 530, which are described in detail as follows:
[0064] The rewriting module 510 is used to read the auto-increment command in the persistent file and rewrite it if it is detected that the number generator is restarted. The persistent file records all the auto-increment commands executed by the number generator;
[0065] A first generating module 520, configured to generate a working node identifier according to the rewriting result, and obtain a current first timestamp and a first sequence number, wherein the first sequence number is generated according to the first timestamp, and the rewriting result includes a previous working node identifier;
[0066] The second generating module 530 is used to combine the first timestamp, the first sequence number and the working node identifier to generate a globally unique identifier.
[0067] It should be noted that the globally unique identifier generation system provided in the above embodiment belongs to the same concept as the globally unique identifier generation system provided in the above embodiment, and the contents of the operations performed by each module have been described in detail in the method embodiment and will not be repeated here.
[0068] See also Figure 6 , Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. Figure 6 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0069] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603, such as executing the method in the above embodiment. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through the bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.
[0070] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.
[0071] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, various functions defined in the system of the present application are executed.
[0072] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the method for generating a globally unique identifier as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.
[0073] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0074] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0075] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0076] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A method for generating a globally unique identifier, characterized in that: The method comprises: If the restart of the number generator is detected, the auto-increment command in the persistent file is read and rewritten. The persistent file records all the auto-increment commands executed by the number generator; Generate a working node identifier according to the rewriting result, and obtain a current first timestamp and a first sequence number, wherein the first sequence number is generated according to the first timestamp, and the rewriting result includes a previous working node identifier; The first timestamp, the first sequence number and the working node identifier are combined to generate a globally unique identifier.
2. The method for generating a globally unique identifier according to claim 1, characterized in that: Before reading the auto-increment command in the persistent file and rewriting it, the method further includes: Recording all write operation commands in the number generator to generate the persistent file, wherein the write operation commands include the auto-increment command; Construct a self-increment command rewrite function, and set the execution condition of the self-increment command rewrite function, wherein the execution condition includes the restart of the generator.
3. The method for generating a globally unique identifier according to claim 2, characterized in that: The reading of the auto-increment command in the persistence file and rewriting it includes: Calling the auto-increment command rewriting function, traversing each of the write operation commands in the persistent file, and detecting whether there is a preset field in each of the write operation commands, wherein the preset field includes a key field and an auto-increment command identification field; If the preset field exists in any write operation command, and the key field is a string field, the any write operation command is added to a pre-built auto-increment command set, and all auto-increment commands in the auto-increment command set are rewritten.
4. The method for generating a globally unique identifier according to claim 3, characterized in that: The generating of the working node identifier according to the rewriting result includes: Determine the first key value of the key in the numbering device according to the last working node identifier, wherein the data type of the last working node identifier is the string type field; Adding a preset auto-increment step size to the first key value to obtain a second key value; The second key value is written back to the key to refresh the first key value, and the second key value is used as the working node identifier.
5. The method for generating a globally unique identifier according to claim 1, characterized in that: The combining the first timestamp, the first sequence number and the working node identifier to generate a globally unique identifier includes: Obtain a globally unique identification structure, wherein the globally unique identification structure includes a positive integer identification field, a timestamp field, a work node identification field, and a serial number field used to distinguish multiple different globally unique identifications generated within the same timestamp, wherein the work node identification field is a string field; According to the globally unique identifier structure, the first timestamp, the first serial number and the working node identifier are combined to generate the globally unique identifier.
6. The method for generating a globally unique identifier according to any one of claims 1 to 5, characterized in that: After the first timestamp is obtained, the method further includes: Obtaining a second timestamp stored in the number generator when the last globally unique identifier was generated; comparing the second timestamp with the first timestamp; If the second timestamp is greater than the first timestamp, an alarm is issued for failure to generate a globally unique identifier; if the second timestamp is less than or equal to the first timestamp, the first serial number generation operation is started, the first timestamp is stored in the number generator, and the second timestamp is refreshed.
7. The method for generating a globally unique identifier according to claim 6, characterized in that: The method for generating the first serial number includes: If the second timestamp is less than the first timestamp, the first sequence number is generated starting from 0; If the second timestamp is equal to the first timestamp, obtaining the second sequence number stored in the number generator when the last global unique identifier was generated, and comparing the second sequence number with a preset maximum sequence number; If the second sequence number is less than the maximum sequence number, one is added to the second sequence number to generate the first sequence number; if the second sequence number is equal to the maximum sequence number, the first sequence number is generated from 0 when the next timestamp is reached.
8. A globally unique identifier generation system, characterized in that: The system comprises: A rewriting module is used to read and rewrite the auto-increment command in the persistent file if it is detected that the number generator is restarted. The persistent file records all the auto-increment commands executed by the number generator; A first generating module, configured to generate a working node identifier according to the rewriting result, and obtain a current first timestamp and a first sequence number, wherein the first sequence number is generated according to the first timestamp, and the rewriting result includes a previous working node identifier; The second generating module is used to combine the first timestamp, the first serial number and the working node identifier to generate a globally unique identifier.
9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method for generating a globally unique identifier as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method for generating a globally unique identifier as described in any one of claims 1 to 7.
