MongoDB database sharding method, electronic device, and storage medium

Abstract

This application discloses a MongoDB database sharding method, electronic device, and computer-readable storage medium. The method includes: obtaining a target ID field, where the target ID field is an auto-incrementing ID field or a auto-decrementing ID field; converting the target ID field, where adjacent target ID fields after conversion belong to different ranges; assigning the converted target ID field to the data to be sharded in a MongoDB collection; creating an index for the MongoDB collection based on at least one field of the data to be sharded, where at least one field includes the converted target ID field; using the index as a shard key, and using the shard key to perform range sharding on the MongoDB collection. This approach can fully utilize the write capabilities of a MongoDB sharding cluster.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a MongoDB database sharding method, an electronic device, and a computer-readable storage medium. Background Art

[0002] As a distributed file storage database, MongoDB is widely used in computer technology due to its high performance, high availability, and high scalability. Applications can write / store data to a MongoDB database through a server, and can also read / query data from a MongoDB database through a server.

[0003] When a user needs to store data, it sends a storage instruction to the server. The server responds to the storage instruction and stores the data in the MongoDB database. When a user needs to query data, it sends a query instruction to the server. The server responds to the query instruction, retrieves the data from the MongoDB database, and returns it to the user.

[0004] For large data sets and high-throughput business scenarios, MongoDB sharding clusters are used to scale data horizontally, writing different data blocks to different shard servers. However, the existing sharding method cannot fully utilize the writing capabilities of MongoDB sharding clusters. Summary of the Invention

[0005] The present application provides a MongoDB database sharding method, electronic device, and computer-readable storage medium, which can solve the problem that existing sharding methods cannot fully utilize the writing capabilities of MongoDB sharding clusters.

[0006] To solve the above technical problems, the present application adopts a technical solution: providing a feature matching method. The method includes: obtaining a target ID field, where the target ID field is an auto-incrementing ID field or a auto-decrementing ID field; converting the target ID field, whereby adjacent target ID fields after conversion belong to different ranges; assigning the converted target ID field to the data to be sharded in a MongoDB collection; creating an index for the MongoDB collection based on at least one field of the data to be sharded, where the at least one field includes the converted target ID field; using the index as a shard key, and performing range sharding on the MongoDB collection using the shard key.

[0007] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, which includes a processor and a memory connected to the processor, wherein the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0008] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium storing program instructions, which can implement the above method when executed.

[0009] Through the above method, this application does not directly perform range sharding based on the target id field, but first converts the target id field so that the difference between adjacent target id fields after conversion is greater than the difference threshold, and then performs range sharding based on the converted target id field, that is, creates an index for the MongoDB collection based on the converted target id field, and uses the index as the shard key for range sharding. Therefore, under the range sharding method, adjacent data to be sharded will most likely not be written to the same shard server, and the MongoDB shard cluster will not have the problem of write bottlenecks, and can give full play to the write capabilities of the MongoDB shard cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of an embodiment of the MongoDB database sharding method of the present application;

[0011] Figure 2 This is a flow chart of another embodiment of the MongoDB database sharding method of the present application;

[0012] Figure 3 yes Figure 2 Specific process diagram of S22;

[0013] Figure 4 This is a schematic diagram of converting the target ID field;

[0014] Figure 5 It is a structural diagram of a self-incrementing ID acquisition device;

[0015] Figure 6 This is a flowchart of another embodiment of the MongoDB database sharding method of the present application;

[0016] Figure 7 This is a flowchart for obtaining the ID field sequence;

[0017] Figure 8 This is a schematic diagram of the existing range sharding method;

[0018] Figure 9 It is a schematic diagram of the sharding method of this application;

[0019] Figure 10 This is a structural diagram of an embodiment of an electronic device of the present application;

[0020] Figure 11 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically specified.

[0023] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments unless there is a conflict.

[0024] Before introducing the sharding method provided in this application, let's first briefly introduce the principles of MongoDB sharding clusters:

[0025] A MongoDB shard cluster consists of multiple shard servers, each hosting multiple MongoDB databases. Each MongoDB database stores data in chunks, each containing zero to multiple data records. Later in this application, a collection of data to be written to a MongoDB database is referred to as a MongoDB collection. When a single shard server has insufficient storage capacity, insufficient load, or reaches its throughput limit, the MongoDB collection needs to be sharded. This means sharding the data in the MongoDB collection so that it can be written to different shard servers.

[0026] Sharding a MongoDB collection is based on the shard key. A shard key is a single or multiple fields that must be included in every data record and are indexed. There are two existing sharding methods: range sharding and hash sharding. Hash sharding selects a single field as the index field, calculates the hash value of the index field, and uses the hash value as the shard key. Range sharding selects one or more fields as the index field and uses the index field as the shard key. Each shard server corresponds to a shard key range. Hash sharding / range sharding writes data to different shard servers based on the shard key range to which the shard key belongs.

[0027] With hash sharding, even if the values ​​of index fields are very close, the corresponding hash values ​​may not belong to the same index range. Therefore, hash sharding can evenly distribute data. However, hash sharding is only suitable for sharding using a single field as the index field, and data query efficiency is low.

[0028] In range sharding, data query efficiency is high. When the shard keys of the data in a MongoDB collection are unevenly distributed, the data written to each shard server is unevenly distributed. In particular, for data with monotonic shard keys, data to be sharded with adjacent shard keys is likely to be written to the same shard server. For example, if the shard key is monotonically increasing, all newly inserted data to be sharded will be written to the shard server with the shard key range with maxkey as the upper limit. If the index field is monotonically decreasing, all subsequent data to be sharded will be written to the shard server with minkey as the lower limit. As a result, the shard server with the shard key range with maxkey as the upper limit, or the shard server with the shard key range with minkey as the lower limit, will become the bottleneck of the entire MongoDB shard cluster's write capacity.

[0029] Therefore, the existing range sharding method will cause a write bottleneck in the MongoDB sharded cluster, and the write capability of the MongoDB sharded cluster cannot be fully utilized.

[0030] To address the write bottleneck issue in MongoDB sharded clusters under range sharding and fully utilize the write capabilities of MongoDB sharded clusters, this application provides the following sharding methods:

[0031] Figure 1 This is a flow chart of an embodiment of the MongoDB database sharding method of the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment may include:

[0032] S11: Get the target ID field.

[0033] The target ID field is an auto-increment ID field or a auto-decrement ID field.

[0034] The auto-increment id field increases by 1 each time, and the auto-decrement id field decreases by 1 each time.

[0035] S12: Convert the target ID field.

[0036] The difference between the converted adjacent target ID fields is greater than the difference threshold.

[0037] The target ID field can be converted by using bit operations such as shift, inversion, phase OR and phase AND. By converting the target ID field, the jump of adjacent target ID fields is achieved.

[0038] S13: Assign the converted target ID field to the data to be sharded in the MongoDB collection.

[0039] S14: Creating an index for the MongoDB collection based on at least one field of the data to be sharded.

[0040] The at least one field includes a converted target id field.

[0041] In S14, if the at least one field only includes the converted target ID field, the converted target ID field can be directly used as the index. If the at least one field includes multiple fields, that is, in addition to the converted target ID field, it also includes other integer fields, then the combination of the multiple fields can be used as the index (composite index), where the converted target field is placed first in the combination.

[0042] S15: Use the index as the shard key and use the shard key to range shard the MongoDB collection.

[0043] It is understandable that the target id field is self-increasing or self-decrementing (with monotonicity), and the difference between two adjacent target id fields is 1. Therefore, if range sharding is performed based on the target id field, that is, the target id field is directly assigned as an index to the data to be sharded, and the index is used as the shard key to implement range sharding of the data to be sharded, adjacent data to be sharded will most likely be written to the same shard server, which cannot fully utilize the write capability of the MongoDB sharding cluster.

[0044] To this end, this application does not perform range sharding directly based on the target ID field. Instead, it first converts the target ID field so that the difference between adjacent target ID fields after conversion is greater than the difference threshold. Then, range sharding is performed based on the converted target ID field. That is, an index is created for the MongoDB collection based on the converted target ID field, and the index is used as the shard key for range sharding. Therefore, under the range sharding method, adjacent data to be sharded will most likely not be written to the same shard server. The MongoDB shard cluster will not have the problem of write bottlenecks, and the write capability of the MongoDB shard cluster can be fully utilized.

[0045] Furthermore, the converted target ID field has a periodic increasing or decreasing property. Accordingly, the converted target ID field is used as an index / shard key, and the index / shard key has a periodic increasing or decreasing property. If the target ID field is an auto-incrementing ID field, the converted target ID field has a periodic increasing property, and the index / shard key has a periodic increasing property. If the target ID field is a auto-decrementing ID field, the converted target ID field has a periodic decreasing property, and the index / shard key has a periodic decreasing property. Thus, by converting the target ID field, a periodic jump of the target ID field is achieved. The periodic jump algorithm has the characteristics of a single-valued function, and the domain and range are the same. Thus, by creating an index based on the converted target ID field and using the index as a shard key for sharding, the writing capability of the MongoDB cluster can be improved while also improving the reading capability of the MongoDB cluster (the specific principle is explained in the following example). Thus, the advantages of range sharding can be retained while inheriting the advantages of hash sharding.

[0046] In the case where the index / shard key is periodically increasing or decreasing, the above embodiment can be further expanded as follows:

[0047] Figure 2 This is a flow chart of another embodiment of the MongoDB database sharding method of the present application. It should be noted that if there is substantially the same result, this embodiment does not Figure 2 The process sequence shown is limited. This embodiment is a further extension of S12. Figure 2 As shown, this embodiment may include:

[0048] S21: intercept the low n bits of the m-bit binary number of the target ID field.

[0049] Among them, n+1<m.

[0050] The AND operation can be used to truncate the m-bit binary number 0 to n-1 of the target ID field as the low n-bit value. n can be 4, 5, 6, 7, 8, etc.

[0051] For example, id_increase represents the target ID field, and id_jump represents the converted target ID field. If you want to extract the lower 8 bits of the m-bit binary number in id_increase, you can add 0xff to the m-bit binary number in id_increase to obtain the lower 8 bits of the m-bit binary number in id_increase.

[0052] S22: Reverse the order of the lower n bits.

[0053] Reversing the order can increase the range of shard keys converted from two adjacent target ID fields, facilitating subsequent range sharding based on shard keys. Reversing the order can be achieved through AND, OR, and shift operations.

[0054] See also Figure 3 , S22 may include the following sub-steps:

[0055] S221: Truncate the lowest bit value of the lower n bits.

[0056] Assuming id_low_n represents the lower n bits, id_low_1 represents the lowest bit, and id_invert represents the reversed result, id_low_1 can be extracted from id_low_n using an AND operation, for example, id_low_n&1.

[0057] S222: Update the reverse order result to the OR result of the reverse order result and the lowest bit value.

[0058] For example, update id_invert to the result of the OR of id_invert and id_low_1.

[0059] S223: Shift the reversed result left by one bit, and shift the lower n-bit value right by 1 bit.

[0060] S224: Determine whether to loop and execute N times.

[0061] If yes, then end; if not, jump to S221 to loop through the above steps until the number of loops reaches n times.

[0062] The initial value of the reverse result is 0, and the final value is the nth reverse result. The low n bits before and after the reverse have a one-to-one correspondence.

[0063] S23: Shift the lower n-bit value after the inversion to the left by mn-1 bits, and shift the truncated m-bit binary number to the right by n bits.

[0064] The processing of the m-bit binary number in the target ID field in steps S21 to S23 above only involves the numeric bits, not the sign bits. Therefore, the processing in steps S21 to S23 is relative to the numeric bits.

[0065] S24: Perform a phase OR operation on the left shift result of the inverted lower n-bit value and the right shift result of the truncated m-bit binary number to obtain a converted target ID field.

[0066] Assume that id_left represents the result of left-shifting the lower n bits of the reversed value, and id_right represents the result of right-shifting the truncated target ID field. We can OR id_left and id_right to obtain the m-bit binary number of the converted target ID field, and thus obtain the converted target ID field.

[0067] The following combination Figure 4 , S21 to S24 are described in detail using a specific example.

[0068] The target ID field is 256020, represented as a 64-bit binary number as "0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0011 1110 1000 0001 0100." Bits 0 to 7 of this binary number, "0001 0100," are used as the first part, bits 8 to 62 as the second part, and bit 63 as the third part. The first part is the low-order 8 bits, and the third part is the sign bit. The first part of this binary number is truncated and reversed to obtain "00101000." Move "0010 1000" to after the third part and before the second part to obtain the binary number of the shard key "0 00101000 000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0011 1110 1000". Convert the binary number of the shard key to decimal form to obtain 1441151880758559720.

[0069] Table 1 below shows the conversion results for target ID fields 256000 to 256511:

[0070] Table 1

[0071] id_increase id_jump 256000 1000 256001 4611686018427388904 256002 2305843009213694952 256255 9187343239835812840 256256 1001 256257 4611686018427388905 256258 2305843009213694953 256511 9187343239835812841

[0072] As shown in Table 1, 256000 is converted to 1000, and 256001 is converted to 4611686018427388904. Before the conversion, 256000 and 256001 are adjacent, and the difference is 1. After the conversion, the difference between 1000 and 4611686018427388904 is 4611686018427387904. ... Therefore, after the adjacent id_increase is converted to id_jump, a huge jump occurs.

[0073] Therefore, if sharding is performed based on id_increase, 256000, 256001, and 256002 are adjacent, so they will most likely be assigned to the same shard server. However, if sharding is performed based on id_jump, 1000, 4611686018427388904, and 2305843009213694952 are not adjacent, so they will most likely be written to different shard servers. Therefore, sharding based on id_jump can fully utilize the write capacity of the MongoDB cluster.

[0074] Moreover, id_jump has a periodic increment, that is, id_jump is incremented every 2 8 Increase by 1. Therefore, the data to be sharded with id_jump 1000 and 1001 is likely to be assigned to the same shard server. Therefore, when querying data to be sharded with id_jump 1000 and 1001, since this data is distributed on the same shard server, it does not need to be broadcast to every shard server for querying, resulting in high query efficiency. This improves the write and read capabilities of the MongoDB cluster.

[0075] In order to more intuitively demonstrate the conversion effect, the following is an explanation in conjunction with Table 2:

[0076] Table 2

[0077] interval No. 63 No. 55-62 0-54th place scope 0 0 0000 0000 000…00–111…11 00000000000000000-36028797018963967 1 0 1000 0000 000…00–111…11 4611686018427387904-4647714815446351871 2 0 0100 0000 000…00–111…11 2305843009213693952-2341871806232657919 … … … … … 254 0 0111 1111 000…00–111…11 4575657221408423936-4611686018427387903 255 0 1111 1111 000…00–111…11 9187343239835811840-9223372036854775807

[0078] As shown in Table 2, id_jump is divided into 256 intervals / ranges, and id_jump is divided into 256 intervals / ranges. 8Increment by 1, so that id_jump cycles through a range of 256. For example, the 0th id_jump is in range 0, the first id_jump jumps to range 1, and so on, and the 256th id_jump returns to range 0. Therefore, sharding based on id_jumps ensures that data with adjacent id_jumps is assigned to different shard servers. Furthermore, the periodic increment ensures that the data to be sharded is distributed as evenly as possible within the MongoDB cluster.

[0079] Furthermore, in the above S11, the target id field can be obtained from the self-incrementing id field obtaining device of the MongDB database. Figure 5 This is a schematic diagram of a self-incrementing ID acquisition device. Figure 5 As shown, the self-incrementing id acquisition device includes a storage unit and an acquisition unit. The storage unit is a set stored in the MongoDB database (hereinafter referred to as the self-incrementing id field set), which is used to store the name of the MongoDB collection to be sharded and the next available id field (hereinafter also referred to as the second starting id field). The data structure stored in the storage unit can be schematically shown as follows:

[0080]

[0081] The _id field stores the name of the MongoDB collection, file_info represents the name of the MongoDB collection to be sharded, and the next_id field stores the next available id field, where 256000 represents the next available id field.

[0082] As one embodiment, an acquisition unit can be used to obtain the next available ID field from the storage unit as the target ID field; the next available ID field stored in the storage unit is modified to the next available ID field plus 1, for use the next time a new target ID field is needed. In this manner, for each piece of data to be sharded, the next available ID field needs to be obtained from the storage unit as the corresponding target ID field, and the next available ID field stored in the storage unit is updated. Therefore, for each piece of data to be sharded, the storage unit / MongoDB database needs to be accessed, and obtaining the target ID field consumes a lot of resources.

[0083] As another embodiment, in order to reduce the number of accesses to the MongoDB database and reduce the resources consumed in obtaining the target ID field, the obtaining unit may obtain the target ID field in the following manner:

[0084] Figure 6 This is a flow chart of another embodiment of the MongoDB database sharding method of the present application. It should be noted that if there is substantially the same result, this embodiment does not Figure 6The process sequence shown is limited. This embodiment is a further extension of S11. Figure 6 As shown, this embodiment may include:

[0085] S31: Determine whether the remaining amount of available ID fields is less than or equal to 0.

[0086] For example, remain is used to represent the remaining amount, and the acquisition unit can be used to determine whether remain≤0 is satisfied.

[0087] If yes, execute S32 to S35; if no, execute S34 to S35.

[0088] S32: Obtain the second starting ID field, update the first starting ID field to the second starting ID field, and reset the remaining amount to the preset amount.

[0089] S33: Update the second starting ID field to the sum of the second starting ID field and the preset number.

[0090] For example, the preset number is 1000, and the second starting id field next_id=next_id+1000 can be set.

[0091] S34: Calculate the target id field based on the first start id field and the remaining amount.

[0092] The difference between the preset quantity and the remaining quantity can be calculated as the offset; the sum of the offset and the first starting ID field can be calculated as the self-incrementing ID field.

[0093] For example, offset is used to represent the offset, and start is used to represent the first starting id field. Therefore, the auto-increment id field id_increase can be calculated using the following formula:

[0094] offset=1000-remain;

[0095] id_increase=start+offset.

[0096] S35: Update the remaining amount to the remaining amount minus 1.

[0097] That is, let remain=remain-1.

[0098] The initial values ​​of the first start ID field (start), the remaining available ID field (remain), and the offset (offset) are all 0. The initial value of the second start ID field can be 0 or a preset number (the value that can be used as the target ID field is reserved for other needs).

[0099] Through the implementation of this embodiment, each time the acquisition unit obtains the second starting ID field from the storage unit as the first starting ID field, it modifies the second starting ID field stored in the storage unit to the sum of the second starting ID field and a preset number. Thus, the acquisition unit obtains the preset number of available ID fields from the storage unit each time, and only accesses the storage unit again when the preset number of available ID fields has been used up. Therefore, the number of accesses to the MongoDB database can be reduced, reducing the resources consumed in obtaining the target ID field.

[0100] Combine as follows Figure 7 , S31-S35 are explained in detail in the form of an example.

[0101] Set the initial values ​​of remain, start, and offset to 0, the initial value of next_id to 1000, and the preset number to 1000.

[0102] 1) Determine whether remain≤0 is satisfied. If so, proceed to 2); if not, proceed to 4).

[0103] 2) Get next_id, start=next_id, remain=1000.

[0104] 3)next_id=next_id+1000.

[0105] 4)offset=1000-remain, id_increase=start+offset.

[0106] 5)remain=remain-1.

[0107] Combine as follows Figure 8 and Figure 9 Taking the auto-increment ID fields of 256000, 256001, and 256002 as an example, the existing range sharding method is compared with the sharding method of this application:

[0108] Figure 8 This is a diagram of the existing range sharding method, combined with reference to Figure 8 The range of shard server A is minkeyx≤id≤100000, and the data block is ChunkA. The range of shard server B is 100000≤id≤256000, and the data block is ChunkB. The range of shard server C is 256000≤id≤maxkey, and the data block is ChunkC. When range sharding is performed based on the auto-increment id field, all three data to be sharded are written to ChunkC.

[0109] Figure 9This is a schematic diagram of the sharding method of this application, please refer to Figure 9 Shard server A's shard key range is minkeyx ≤ id ≤ 100000, and its data chunk is ChunkA. Shard server B's shard key range is 100000 ≤ id ≤ 2341871806232657919, and its data chunk is ChunkB. Shard server C's shard key range is 4611686018427388904 ≤ id ≤ maxkey, and its data chunk is ChunkC. First, the auto-increment ID fields 256000, 256001, and 256002 are converted (periodically jumping), resulting in the converted auto-increment ID fields 1000, 4611686018427388904, and 23058435843009213694952, respectively. When sharding based on the converted target ID field, the three data to be sharded are written to ChunkA, ChunkC, and ChunkB, respectively.

[0110] Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application. Figure 10 As shown, the electronic device includes a processor 21 and a memory 22 coupled to the processor 21 .

[0111] The memory 22 stores program instructions for implementing the method of any of the above embodiments; the processor 21 is used to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. The processor 21 can also be called a CPU (Central Processing Unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0112] Figure 11 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present application. Figure 11As shown, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and when the program instructions 31 are executed, the method provided in the above embodiment of the present application is implemented. Among them, the program instructions 31 can form a program file and be stored in the above-mentioned computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) executes all or part of the steps of the various embodiments of the present application. The aforementioned computer-readable storage medium 30 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0114] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A MongoDB database sharding method, characterized in that: include: Get the target ID field, which is a self-incrementing ID field or a self-decrementing ID field; Converting the target ID field, wherein a difference between adjacent target ID fields after conversion is greater than a difference threshold, includes: truncating low-order n-bit values ​​of an m-bit binary number of the target ID field, where n+1<m; reversing the low-order n-bit values; left-shifting the reversed low-order n-bit values ​​by mn-1 bits, and right-shifting the truncated m-bit binary number by n bits; performing a phase OR operation on the left-shift result of the reversed low-order n-bit values ​​and the right-shift result of the truncated m-bit binary number to obtain the converted target ID field; Assign the converted target id field to the data to be sharded in the MongoDB collection; Creating an index for the MongoDB collection based on at least one field of the data to be sharded, wherein the at least one field includes the converted target id field; The index is used as a shard key, and the MongoDB collection is range-sharded using the shard key.

2. The method according to claim 1, characterized in that The creating an index for the MongoDB collection based on at least one field of the data to be sharded includes: The converted target ID field is used as the index.

3. The method according to claim 1, characterized in that If the target ID field is an auto-incrementing ID field, the target ID field after conversion has a periodic increment property; if the target ID field is a auto-decrementing ID field, the target ID field after conversion has a periodic decrement property.

4. The method according to claim 1, wherein The reversing of the lower n-bit values ​​includes: Cutting off the lowest bit value of the lower n bits; The updated reverse order result is the OR result of the reverse order result and the lowest bit value; Shift the reversed result left by one bit, and shift the lower n-bit value right by 1 bit; The above steps are repeated until the number of cycles reaches n.

5. The method according to claim 1, wherein The target ID field is the self-incrementing ID field, and the acquisition target ID field includes: Determine whether the remaining number of available ID fields in the current range is less than or equal to 0; If so, obtain the second starting ID field, update the first starting ID field to the second starting ID field, and reset the remaining amount to the preset amount; update the second starting ID field to the sum of the second starting ID field and the preset amount; calculate the target ID field based on the first starting ID field and the remaining amount; update the remaining amount to the remaining amount minus 1, where the second starting ID field is the starting ID field of the next range, and the first starting ID field is the starting ID field of the current range; If not, the target id field is calculated based on the first start id field and the remaining amount; and the remaining amount is updated to be the remaining amount minus 1.

6. The method according to claim 5, characterized in that The calculating the target ID field based on the first start ID field and the remaining amount includes: Calculating the difference between the preset quantity and the remaining quantity as an offset; The sum of the offset and the first starting ID field is calculated as the self-incrementing ID field.

7. The method according to claim 1, wherein the at least one field comprises a plurality of the fields, and the step of creating an index for the data to be sharded based on the at least one field comprises: A combination of multiple fields is used as the index, wherein the converted target ID field is at the first position of the combination.

8. An electronic device, characterized in that: comprising a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 7 is implemented.

Images