Distributed service ID generation system

Abstract

The invention relates to a distributed business ID generation system, which comprises an order service module located at a local end and an ID generation service module located at a remote end, and is characterized in that the order service module is built in local application service equipment; when local application service equipment is started, a built-in order service module preloads and calls an interface of the ID generation service module; the ID generation service module inquires a database according to the business type to which the current order belongs to obtain an ID data record under the current business type and returns an ID batch number segment to the order service module, and meanwhile, the ID generation service module updates a current ID value under the current business type according to a set step length; all IDs in the ID batch number segment obtained by the order service module are sequentially stored in a local synchronous blocking queue, and when the order service module needs to use the IDs, ID data are taken from the local synchronous blocking queue. Compared with the prior art, the method has the advantage that the performance problem of a single node in a high-concurrency scene is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of ID generation, and in particular to a distributed service ID generation system. Background Art

[0002] The globally unique ID of a distributed system is a scenario that all systems will encounter. It is often used in search and storage as a unique identifier or sorting, such as a globally unique order number, a coupon code, etc. If two identical order numbers appear, it will undoubtedly be a huge bug for users.

[0003] In a monolithic system, since there is only one machine and one application, generating a unique ID is not a challenge. You can simply use a singleton plus an atomic operation to increment it. In a distributed system, however, there are different applications, different computer rooms, and different machines. If you need to generate a unique ID, it is extremely difficult.

[0004] At present, the distributed ID generation solutions on the market have the following defects: single-point performance problems, global orderliness problems, and ID discontinuity problems after power failure and restart. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a distributed business ID generation system, wherein the distributed local storage of each node and the early warning preloading settings solve the performance problem of a single node in a high concurrency scenario.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a distributed service ID generation system, comprising an order service module located at a local end and an ID generation service module located at a remote end, wherein the order service module is built in a local application service device;

[0008] When the local application service device is started, the built-in order service module preloads the interface for calling the ID generation service module;

[0009] The ID generation service module queries the database to obtain the ID data record under the current business type according to the business type of the current order and returns the ID batch number segment to the order service module. At the same time, the ID generation service module updates the current ID value under the current business type according to the set step size;

[0010] All IDs in the ID batch number segment acquired by the order service module are stored in the local synchronous blocking queue in sequence. When the order service module needs to use the ID, it retrieves the ID data from the local synchronous blocking queue.

[0011] Preferably, after the order service module obtains the ID data from the local synchronous blocking queue, the asynchronous thread determines whether the number of IDs remaining in the current queue is less than the set value. If it is less than the set value, the order service module calls the interface of the ID generation service module to obtain a new ID batch number segment;

[0012] After obtaining the new ID batch number segments, the order service module writes the newly obtained ID batch number segments into the local synchronous blocking queue in sequence.

[0013] Preferably, the determining whether the number of IDs remaining in the current queue is less than a set value is specifically: determining whether the number of IDs remaining in the current queue is less than 50%.

[0014] Preferably, the order service module takes ID data from a local synchronous blocking queue and asynchronously stores the current ID in Redis.

[0015] Preferably, the Redis stores the current machine number, current ID value and current ID maximum value corresponding to each business type biz_type.

[0016] Preferably, when the order service module is powered off and restarted, the order service module obtains the current ID value and the current ID maximum value from Redis according to the current business type biz_type and the current machine number. After processing the data, all IDs in the ID batch number segment with an interval range of [current ID value + 1, current ID maximum value] are written into the local blocking queue of the order service module at once.

[0017] Preferably, the storage space size of the local queue in the order service module is twice the step size corresponding to the business type biz_type.

[0018] Preferably, the ID generation service module queries the database to obtain the ID data records under the current business type according to the business type of the current order and returns the ID batch number segments to the order service module. Specifically, the ID generation service module obtains the ID data records under the current business type from the Mysql database and returns the ID batch number segments to the order service module.

[0019] Preferably, the ID generation service module updates the current ID value under the current service type according to the set step size, specifically: the ID generation service module updates the current ID value under the current service type according to the set step size by adding the current ID value to the set step size.

[0020] Preferably, the order service module is built into a local application service device via a jar package.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1) The batch ID segments are preloaded in advance, and each ID is stored in a local synchronous blocking queue. When using it, you only need to directly call the operation from the local queue. The distributed local storage and early warning preloading settings of each node solve the performance problem of a single node in a high-concurrency scenario, improve the response efficiency of the service, and effectively ensure that the ID generation service module does not become a performance bottleneck when used as a centralized service.

[0023] 2) The business type classification integration model is used to effectively distinguish the different ID segments generated by different business scenarios. The business scenarios are distinguished by ID segmentation, which can ensure the uniqueness of the generated ID.

[0024] 3) The current ID and the current maximum ID are stored in redis. When the machine loses power, it will obtain the historical unused ID segments from Redis after restart and restore PUT to the synchronous blocking queue. The use of Redis and memory storage solves the problem of discontinuous ID generation when the machine loses power and restarts.

[0025] 4) When the number of IDs in the synchronous blocking queue is insufficient, the remote ID generation service is immediately called to obtain the number segment to add to the synchronous blocking queue.

[0026] 5) The global orderliness problem of ID generation is solved by utilizing the characteristics of MySQL database.

[0027] 6) Use the jar package model to directly introduce it into the business service equipment that needs to use the ID, making system deployment more convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the system framework structure of the present invention;

[0029] Figure 2 This is a schematic diagram of the storage of ID batch segments in memory;

[0030] Figure 3 Schematic diagram of the process of obtaining ID. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] Example

[0033] like Figure 1As shown, this embodiment provides a distributed business ID generation system, including an order service module located at a local end and an ID generation service module located at a remote end, the order service module is built in a local application service device (built in the local application service device through a jar package, and each local application service device that needs to use the ID introduces the jar package);

[0034] When the local application service device is started, the built-in order service module preloads the interface for calling the ID generation service module;

[0035] The ID generation service module queries the database to obtain the ID data record under the current business type according to the business type of the current order and returns the ID batch number segment to the order service module. At the same time, the ID generation service module updates the current ID value under the current business type according to the set step size;

[0036] All IDs in the ID batch segment obtained by the order service module are stored in the local synchronous blocking queue in sequence. When the order service module needs to use the ID, it retrieves the ID data from the local synchronous blocking queue block queue stored in the jvm memory.

[0037] Next, combine Figure 3 , the specific ID generation and use process of the distributed service ID generation system of this embodiment is described in detail, including:

[0038] 1) When the application service device is started, the order service module preloads the interface for calling the ID generation service module;

[0039] 2) The ID generation service module queries the database and finds the corresponding data row record based on the biz_type of the current order service. The current value is 0 and the step length is 1000.

[0040] 3) The ID generation service module returns current and step to the order service module, and updates current to: current+step, that is, updates it to 1000.

[0041] 4) The order service module obtains current=0, step=1000.

[0042] 5) The ID interval that the order service module can obtain is [current+1, current+step], that is, [1,1000].

[0043] 6) The order service module processes data locally and stores all IDs between [1,1000] in sequence into the local JVM's synchronous blocking queue block queue. The storage diagram is shown in the figure below. Figure 2 shown.

[0044] 7) When the order service module needs to use the ID, it can directly take the TAKE data from the local jvm memory queue. The TAKE operation has a dynamic loading mechanism. After the order service module obtains the data from the local queue, the asynchronous thread will determine whether the number of IDs remaining in the current queue is sufficient (<50%).

[0045] 8) If the number of IDs in the current queue is insufficient, the order service module will call the interface of the ID generation service module.

[0046] 9) After obtaining the new batch number segment, the algorithm in the local jar package in the order service module will write the newly pulled batch number segment into the local queue in turn and PUT it to ensure that the ID in the local queue of the order service module is always sufficient.

[0047] 10) After the order service module obtains the ID from the local queue, it will asynchronously store the current ID in redis:

[0048] The redis key is: (biz_type + current machine number)

[0049] The redis value is: {current: current ID value, max_id: current ID maximum value}.

[0050] 11) When the order service module is powered off and restarted, the order service module will obtain the current ID value and max_id value from redis based on the current biz_type and machine number. After processing the data in the next step, all IDs in the interval [ID+1, max_id] will be written into the order service module jvm local queue at once through PUT, so as to achieve the purpose of not losing the ID batch segment after the machine is powered off and restarted.

[0051] 12) The storage space size of the local block queue of the order service module is twice the step corresponding to the biz_type.

[0052] The ID batch segments are cached in the local business service device. As the business service instances increase, the corresponding global QPS also increases, which can fully adapt to the rapid growth of business volume.

[0053] Traditional ID generation (batch number segment mode) stores the current AtomicLong type number, and it is necessary to go through AtomicLong again when retrieving the number. The getAndIncrement method of the auto-increment calculation also consumes resource performance. The method of preloading the ID batch segments in advance stores each ID in the local synchronous blocking queue block queue, such as Figure 2 When using it, you only need to directly call the TAKE operation from the local queue, which effectively solves the problem of AtomicLong under high concurrency. The performance bottleneck problem of getAndIncrement.

[0054] The current ID and the current maximum ID are stored in redis. When the machine loses power, after restarting, the historical unused ID segments will be obtained from redis and restored to the synchronous blocking queue by PUT, ensuring that the ID segments are not lost after the machine loses power.

[0055] By using the jar package model, it can be directly introduced into the business service that needs to use the ID. As the business service machine instance nodes expand, the acquisition of the ID will not become a business bottleneck.

[0056] In this embodiment, a data table of starting support data for ID generation is created, specifically:

[0057] CREATE TABLE`sequence_id_info`(

[0058] `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Auto-increment primary key',

[0059] `biz_type`varchar(63)NOT NULL DEFAULT"COMMENT'Business type, unique',

[0060] `current` bigint(20) NOT NULL DEFAULT '0' COMMENT 'Current value',

[0061] `step` int(11) DEFAULT'0' COMMENT'Step length',

[0062] `create_time` timestamp NOT NULL DEFAULT'2010-01-01 00:00:00' COMMENT'Creation time',

[0063] `update_time` timestamp NOT NULL DEFAULT'2010-01-01 00:00:00' COMMENT'Update time',

[0064] PRIMARY KEY(`id`),

[0065] UNIQUE KEY`uniq_biz_type`(`biz_type`)

[0066] )ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT 'id information table';

[0067] The construction process of the ID generation service module in this embodiment is specifically as follows:

[0068] This service is mainly used to connect to the database. When each business scenario needs to generate an ID, it will call this service to achieve the purpose. Each time the service is requested for a different business scenario, the service will generate a batch of number segments for the scenario (biz_type) and return them to the caller.

[0069] For example, when the step size is set to 1000, and the service is requested for the business scenarios of scene 1, scene 2, and scene 3, the service will return a number segment for each scenario:

[0070] (1,1000)->biz_type: scene 1, step: 1000, current: 0;

[0071] (1,1000)->biz_type: scene 2, step: 1000, current: 0;

[0072] (1,1000]->biz_type: scene 3, step: 1000, current: 0.

[0074] The local memory storage of ID batch number segments in this embodiment is specifically as follows:

[0075] Local storage makes ID issuance more efficient. BlockQueue is introduced as a local storage container for IDs to ensure synchronization between threads in a high-performance concurrent environment. If the number of IDs in the current block queue is less than half of the capacity of the block queue after the operation, the remote ID generator service will be requested to request a new batch of number segments, and these number segments will be stored in the blockQueue. Each time a business needs to use an ID, a new ID will be taken from the blockQueue and returned to the caller.

[0076] This embodiment uses redis storage as a back-end guarantee. Redis will store the current id value corresponding to each business scenario biz_type, with biz_type as the key and the value as the current id value of the business scenario. For example, for scene 1, a business scenario with a step size of 1000, when the current value is 996, the corresponding storage in redis is r.set(scene1+machine number, 996, the maximum value of the current number segment). If the current machine instance loses power and shuts down at this time, when the machine is restarted again, the value in redis will be read, and the current value will be used as the starting value to perform a write PUT operation on the synchronous blocking queue block Queue.

[0077] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A distributed service ID generation system, characterized in that: It includes an order service module located at the local end and an ID generation service module located at the remote end, wherein the order service module is built in the local application service device; When the local application service device is started, the built-in order service module preloads the interface for calling the ID generation service module; The ID generation service module queries the database to obtain the ID data record under the current business type according to the business type of the current order and returns the ID batch number segment to the order service module. At the same time, the ID generation service module updates the current ID value under the current business type according to the set step size; All IDs in the ID batch number segment acquired by the order service module are stored in the local synchronous blocking queue in sequence. When the order service module needs to use the ID, it retrieves the ID data from the local synchronous blocking queue.

2. A distributed service ID generation system according to claim 1, characterized in that: After the order service module obtains the ID data from the local synchronous blocking queue, the asynchronous thread determines whether the number of IDs remaining in the current queue is less than the set value. If it is less than the set value, the order service module calls the interface of the ID generation service module to obtain a new ID batch number segment; After obtaining the new ID batch number segments, the order service module writes the newly obtained ID batch number segments into the local synchronous blocking queue in sequence.

3. A distributed service ID generation system according to claim 2, characterized in that: The determining whether the number of IDs remaining in the current queue is less than a set value specifically includes: determining whether the number of IDs remaining in the current queue is less than 50%.

4. A distributed service ID generation system according to claim 1, characterized in that: The order service module takes ID data from the local synchronous blocking queue and asynchronously stores the current ID in Redis.

5. A distributed service ID generation system according to claim 4, characterized in that: The Redis stores the current machine number, current ID value and current ID maximum value corresponding to each business type biz_type.

6. A distributed service ID generation system according to claim 5, characterized in that: When the order service module is powered off and restarted, the order service module obtains the current ID value and the current ID maximum value from Redis according to the current business type biz_type and the current machine number. After processing the data, all IDs in the ID batch number segment with the interval range of [current ID value + 1, current ID maximum value] are written into the local blocking queue of the order service module at once.

7. A distributed service ID generation system according to claim 5, characterized in that: The storage space size of the local queue in the order service module is twice the step size corresponding to the business type biz_type.

8. A distributed service ID generation system according to claim 1, characterized in that: The ID generation service module queries the database to obtain the ID data records under the current business type according to the business type of the current order and returns the ID batch number segments to the order service module. Specifically: the ID generation service module obtains the ID data records under the current business type from the Mysql database and returns the ID batch number segments to the order service module.

9. A distributed service ID generation system according to claim 1, characterized in that: The ID generation service module updates the current ID value under the current service type according to the set step length, specifically: the ID generation service module updates the current ID value under the current service type according to the set step length by adding the current ID value to the set step length.

10. A distributed service ID generation system according to claim 1, characterized in that: The order service module is built into the local application service device through a jar package.

Images