Redis cluster query method, device, computer-readable storage medium, and program product
By analyzing the node index and array index in the cursor parameters, the traversal query of the Redis cluster is realized, which solves the problem that the scan instruction cannot traverse multiple nodes in the existing technology, simplifies the data migration process and improves the query efficiency.
Patent Information
- Application Number
- CN202011283742.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-17
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-11-17
AI Technical Summary
In the existing Redis cluster, the scan instruction can only query the data of a single node and cannot traverse the entire cluster, resulting in compatibility issues during the data migration process.
By optimizing cursor parameters, parsing out node index and array index, thereby obtaining data in the Redis cluster, realizing traversal query of multiple nodes, and is compatible with the standard version of Redis client protocol.
A comprehensive query of the Redis cluster is realized, which simplifies the migration process from a single node to multiple nodes, avoids compatibility issues, and improves the efficiency and reliability of data queries.
Smart Images

Figure CN114510500B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to the field of computer technology, and more specifically to a Redis cluster query method, a Redis cluster query device, a computer-readable storage medium, and a computer program product. Background Art
[0002] Remote Dictionary Server (Redis) is an open source key-value database. In Redis, you can execute the scan command to traverse a single node to query the relevant data in this node. In addition, in order to meet the growing demand for data processing, a Redis cluster containing multiple nodes can be built. However, currently, the scan command can only query the data in the current single node in the Redis cluster, but cannot traverse multiple nodes in the entire Redis cluster at once. In this way, when the data in Redis containing only a single node is migrated to a Redis cluster containing multiple nodes, there will be compatibility issues if it involves data traversal query (use of the scan command). Summary of the invention
[0003] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.
[0004] According to one aspect of the present disclosure, a Redis cluster query method is provided, including: obtaining a query request, wherein the query request includes a cursor parameter; parsing a node index and an array index according to the cursor parameter, and determining a node corresponding to the node index as a starting node for iterative query data, and a position corresponding to the array index as a starting position for iterative query data; and, starting from the starting position, obtaining data from the Redis cluster according to the query request.
[0005] According to another aspect of the present disclosure, a Redis cluster query device is provided, including: a memory on which instructions are stored; and a processor configured to execute the instructions stored in the memory to execute the above-mentioned Redis cluster query method.
[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which includes computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors execute the above-mentioned Redis cluster query method.
[0007] According to another aspect of the present disclosure, a computer program product is provided, including computer instructions, which implement the steps of the above-mentioned Redis query cluster method when executed by one or more processors. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0009] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
[0010] Figure 1 A schematic diagram of a Redis cluster query method according to an exemplary embodiment of the present disclosure is shown;
[0011] Figure 2 Shows Figure 1 A flow chart of step S200 of the Redis cluster query method;
[0012] Figure 3 Shows Figure 1 A flow chart of step S300 of the Redis cluster query method in FIG. 1 ;
[0013] Figure 4 An exemplary configuration of a Redis cluster query device that can implement an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0014] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and not to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.
[0015] In the Redis database, a query request can be initiated through the scan command to query related data. Among them, the data is stored in the Redis node in the form of key-value pairs. Traversing Redis means filtering out the data whose key values meet the corresponding conditions and returning them in the form of an array. In the scan command, a cursor parameter (cursor) and optional condition parameters (pattern) and count parameters (count) can usually be included. Among them, the cursor parameter (cursor) can be used to represent the current query position in the traversal query process (the identifier of a specific position of a data in the array currently queried); the condition parameter can be a query condition in the form of a regular expression, which is used to represent the specific conditions satisfied by the data to be queried; the count parameter can be a preset number, which is used to limit the maximum number of data queried in this traversal, so as to avoid the traversal query process from being unable to end and causing process blocking when there are too many data that meet the condition parameters.
[0016] Generally, cursor parameters only include array indexes, which are used to indicate a specific position in an array in a node. However, in a Redis cluster containing multiple nodes, a cursor not only indicates a specific position in an array in the node, but also needs to indicate a specific position in a node.
[0017] In the technical solution of the present disclosure, the cursor parameters are optimized so that not only the array index indicating a specific position in the array can be parsed from the cursor parameters, but also the node index indicating the node can be parsed from the cursor parameters to distinguish multiple nodes in the Redis cluster. In addition, the technical solution of the present disclosure also optimizes the relevant steps in the Redis cluster query method, so that multiple nodes in the Redis cluster can be traversed through a query request (scan instruction), thereby realizing a comprehensive query of the Redis cluster, simplifying the migration of services from a Redis containing only a single node to a Redis cluster containing multiple nodes.
[0018] According to an exemplary embodiment of the present disclosure, a Redis cluster query method is proposed, such as Figure 1 As shown, the Redis cluster query method may include:
[0019] Step S100: obtaining a query request, wherein the query request may include a cursor parameter.
[0020] The cursor parameters of this embodiment include information related to the node index and array index, so that the corresponding node index and array index can be parsed according to the cursor parameters in subsequent steps, and uniquely indicate the specific position of data in a certain array in a certain node in the Redis cluster.
[0021] Specifically, the node index can uniquely indicate the node in the Redis cluster, and under the premise of determining the node, the array index can uniquely indicate the specific location of the data in the array of this node. It should be noted that the same array index can indicate the specific location of data in multiple arrays in different nodes, so it is necessary to combine the node index to uniquely determine the specific location of data in a specific array.
[0022] In some embodiments, the cursor parameters may include a first preset number of node index bits configured to represent the node index and a second preset number of array index bits configured to represent the array index. The first preset number of bits may be set according to the total number of nodes in the Redis cluster, and the second preset number of bits may be set according to the maximum number of data in the array in the node, to ensure that both the node index bits and the array index bits are sufficient to represent the data in the nodes and arrays that may be involved. In different embodiments, the first preset number of bits and the second preset number of bits may be equal or unequal as needed.
[0023] In some embodiments, in order to facilitate parsing the node index and array index according to the cursor parameter in subsequent steps, and to make the Redis query method (scan instruction) in the technical solution of the present disclosure compatible with the existing open source standard client protocol, the node index bits of the first preset number of bits can be arranged continuously in the cursor parameter, and the array index bits of the second preset number of bits can be arranged continuously in the cursor parameter. In this way, the cursor parameter can be divided into two continuous parts representing the node index and the array index, respectively. It is understandable that in other embodiments, the positions of the node index bits and the array index bits in the cursor parameter may also be allocated according to other rules.
[0024] In some embodiments, the node index bit of the cursor parameter can be in front of the array index bit, that is, the first part of the cursor parameter represents the node index, and the second part of the cursor parameter represents the array index. In this way, the Redis query method (scan instruction) is compatible with the standard version of the Redis client protocol to facilitate the migration of services from a single-node Redis to a Redis cluster.
[0025] In a specific example, the cursor parameter can be in string format, and its maximum length can be 20 bits. In the existing cursor parameters, these 20 bits can all be used to represent array indexes. Then, if the cursor parameter is specified as a non-negative integer, the value range of the array index can be 0 to 18446744073709551616. However, in the actual Redis database, so many array indexes are rarely used. Therefore, in the present disclosure, the first 10 (first preset number of digits) bits of the cursor parameter can be used to represent the node index, and only the last 10 (second preset number of digits) bits of the cursor parameter can be used to represent the array index. In fact, even so many node indexes and array indexes are usually sufficient in the Redis cluster. When converting the existing cursor parameters to the optimized cursor parameters in the present disclosure, the valid number of digits representing the array index can be moved to the last ten bits of the cursor parameter, and the first ten bits of the cursor parameter can be determined according to the nodes in the Redis cluster where the array is located.
[0026] In another specific example, considering that the total number of nodes in the Redis cluster may be less than the maximum number of data that can be contained in the array in each node, and in order to save the number of bits occupied by the node index in the cursor parameter and avoid the situation where the number of bits of the array index is insufficient, the first preset number of bits and the second preset number of bits may also have other values. For example, the first preset number of bits for the node index may be determined according to the total number of nodes in the Redis cluster, and the remaining number of bits in the cursor parameter may be allocated to the array index.
[0027] In one calculation method, the first preset number of digits N1 may satisfy N1=[lg(N0)]+1, where N0 is the total number of nodes in the Redis cluster, and [lg(N0)] represents the logarithm of N0 with a base of 10 rounded. For example, when the total number of nodes in the Redis cluster is 12, according to the above formula, the first preset number of digits N1 is 2. Then, in this 12-node Redis cluster, the cursor parameter 050000000000000010000 may represent the data in the corresponding array with a node index of 5 and an array index of 10000.
[0028] return Figure 1 , the Redis cluster query method can also include:
[0029] Step S200, parse the node index and array index according to the cursor parameter, and determine the node corresponding to the node index as the starting node of iterative query data, and the position corresponding to the array index as the starting position of iterative query data.
[0030] Normally, if this traversal is the first query, the current node index and array index can be 0; if this traversal is not the first query, the current node index and array index can correspond to the node index and array index of the last query, that is, this query will start from the position where the last query ended to avoid missing queries.
[0031] like Figure 2 As shown, parsing the node index and array index according to the cursor parameter may include:
[0032] Step S210, obtaining node information in the Redis cluster;
[0033] Step S220, determining a first preset number of bits and a second preset number of bits according to the node information; and
[0034] Step S230, determining the node index according to a first preset number of node index bits, and determining the array index according to a second preset number of array index bits.
[0035] That is to say, in the process of parsing the cursor parameters, the key is to obtain the node index position and array index position occupied by the node index and array index in the cursor parameters respectively, and then the node index and array index can be determined according to the specific values on the node index position and array index position. The determination of the node index position and array index position can be carried out according to the node information. And in different embodiments, there can be different node information.
[0036] In some embodiments, the node information may include the total number of nodes N0 in the Redis cluster. In this case, the first preset number of bits N1 in the cursor parameter can be calculated according to N1=[lg(N0)]+1 described above, and then the second preset number of bits N2 can be determined according to the difference between the total number of bits of the cursor parameter and the first preset number of bits N1, and then the node index and the array index can be determined.
[0037] In other embodiments, the node information may include the current query node and a non-zero cursor parameter corresponding to the current query node. It is understandable that in the non-zero cursor parameter, it is easy to determine the first preset bit number of the node index. That is, according to the known node index of the current query node, the corresponding node index bit can be found in the cursor parameter. For example, assuming that the node index of the current query node is known to be 011, and the corresponding cursor parameter is known to be 01100000000000001010, then, according to the corresponding relationship, it can be concluded that the first 3 bits of the cursor parameter correspond to the node index bit, and accordingly, the remaining 17 bits correspond to the array index bit, that is, the first preset bit number N1 is 3, and the second preset bit number N2 is 17.
[0038] return Figure 1, the Redis cluster query method can also include:
[0039] Step S300, starting from the starting position, obtaining data from the Redis cluster according to the query request.
[0040] As described above, the query request can be implemented through the scan instruction in Redis. That is, the query request can also include a condition parameter and a preset number (equivalent to the count parameter in the scan instruction) to obtain data that meets the condition.
[0041] During the query process, the traversal starts from a specific position in the array of the specified node parsed from the cursor parameter. If a preset number of data can be obtained in this node, these data can be output as result data, or these data can be merged into the result set to form the result data output. In addition, during the traversal process, the cursor parameter will be updated as the query position changes, and it always corresponds to the latest queried position to facilitate the next traversal. If the data in the array of the preset number cannot be obtained in this node, the node index in the cursor parameter can be changed, and the array index can be set to 0, and the traversal can be continued in the next node, and so on, more nodes can be traversed until the data in the array of the preset number is queried, and these data can be output as result data, or these data can be merged into the result set to form the result data output. If the traversal of all nodes has been completed and the data of the array of the preset number has not been obtained, these data can also be output as result data, or these data can be merged into the result set to form the result data output, and the query is terminated.
[0042] The flowchart of the above traversal process is as follows Figure 3 As shown, starting from the starting position, obtaining data from the Redis cluster according to the query request may include:
[0043] Step S310, in the node where the starting position is located, starting from the starting position, determine in sequence whether the data in the array meets the conditional parameters, and update the array index to correspond to the current position, wherein the starting position is a specific position in the array of the designated node obtained by parsing the cursor parameter;
[0044] Step S320, when data satisfying the condition parameters is found, the data is merged into the result data;
[0045] Step S330, when the number of data items in the result data reaches a preset number, stop scanning the node and output the result data;
[0046] Step S340, when the query of the current node is completed and the number of data items in the result data does not reach the preset number, the node index is updated to correspond to the next node, and the array index is updated to zero;
[0047] Step S350, obtaining data from the Redis cluster according to the updated node index and array index and conditional parameters until the number of data items in the result data reaches a preset number, and outputting the result data.
[0048] In a specific example, in the online number selection service, you can query the Redis cluster for numbers with 6 consecutive 8s. Assume that it has been concluded based on other information that in the cursor parameter of the scan instruction, the first 10 bits are the node index and the last 10 bits are the array index. The current cursor parameter is 00000000100000000111, the condition parameter is "*888888*" expressed in the form of a regular expression, and the count parameter (preset number) is 20. Then, first in node 10, start taking data from the data at position 111 in the array. Whenever an array of data with a key value that meets the regular expression of "*888888*" is queried, when the data is not less than 20 pieces, the data therein will be merged into the result data, and these data will be output directly or after certain processing. If 20 pieces of data are not found after traversing node 10, and assuming that the cursor parameter is 00000000100000000113 when the traversal ends, the cursor parameter is updated to 00000000110000000000, that is, traversing node 11 begins, and so on, until 20 pieces of data are found and output.
[0049] In the Redis cluster query method disclosed in the present invention, by optimizing the cursor parameters so that it can represent both the node and the specific position in the array, the defect that the existing scan instruction cannot traverse the Redis cluster is compensated, and the scan instruction in the Redis cluster version that is compatible with the standard version of the Redis client protocol is implemented, so that the Redis users of a single node can be seamlessly migrated to the Redis cluster, simplifying the transformation and adaptation of the user migration business system.
[0050] The present disclosure also proposes a Redis cluster query device. Figure 4 An exemplary configuration of a Redis cluster query device 900 capable of implementing an embodiment of the present disclosure is shown.
[0051] The Redis cluster query device 900 is an example of a hardware device to which the above aspects of the present disclosure can be applied. The Redis cluster query device 900 can be any machine configured to perform processing and / or computing. The Redis cluster query device 900 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smart phone, a vehicle-mounted computer, or a combination thereof.
[0052] like Figure 4As shown, the Redis cluster query device 900 may include one or more elements that can be connected or communicated with the bus 902 via one or more interfaces. The bus 902 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. The Redis cluster query device 900 may include, for example, one or more processors 904, one or more input devices 906, and one or more output devices 908. The one or more processors 904 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). The processor 902 may be configured to, for example, implement a query request, parse out a node index and an array index according to a cursor parameter in the query request, and determine a specific position in an array in the Redis cluster that uniquely corresponds to the parsed node index and array index, and use the specific position as a starting position; and, starting from the starting position, obtain data from the Redis cluster according to the query request. The input device 906 may be any type of input device capable of inputting information to the Redis cluster query apparatus 900, and may include but is not limited to a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 908 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer.
[0053] The Redis cluster query device 900 may also include or be connected to a non-transitory storage device 914, which may be any non-transitory storage device that can implement data storage, and may include but is not limited to a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a tape or any other magnetic medium, a compressed disk or any other optical medium, a cache memory and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions and / or codes. The Redis cluster query device 900 may also include a random access memory (RAM) 910 and a read-only memory (ROM) 912. The ROM 912 may store programs, utilities or processes to be executed in a non-volatile manner. The RAM 910 may provide volatile data storage and store instructions related to the operation of the Redis cluster query device 900. The Redis cluster query device 900 may also include a network / bus interface 916 coupled to a data link 918. The network / bus interface 916 may be any type of device or system capable of enabling communication with an external device and / or a network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth TM equipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.).
[0054] The present disclosure may be implemented as any combination of an apparatus, a system, an integrated circuit, and a computer program on a non-transitory computer-readable medium. One or more processors may be implemented as an integrated circuit (IC), an application specific integrated circuit (ASIC), or a large scale integrated circuit (LSI), a system LSI, a super LSI, or a super LSI component that performs some or all of the functions described in the present disclosure.
[0055] The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, applications, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to enable a computer such as one or more processors to perform the above steps and the steps described in the accompanying drawings. For example, one or more memories store the software or algorithm in executable instructions, and one or more processors may associate a set of instructions to execute the software or algorithm to provide various functions according to the embodiments described in the present disclosure.
[0056] Software and computer programs (which may also be referred to as programs, software applications, applications, components or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or an assembly language or a machine language. The term "computer-readable medium" refers to any computer program product, apparatus or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.
[0057] For example, computer readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer readable program code in the form of instructions or data structures and can be accessed by a general or special purpose computer or a general or special purpose processor. As used herein, disk or disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically by laser. Combinations of the above are also included in the scope of computer readable media.
[0058] The subject matter of the present disclosure is provided as an example of devices, systems, methods and programs for performing the features described in the present disclosure. However, in addition to the above-mentioned features, other features or variations may also be expected. It is expected that the implementation of the components and functions of the present disclosure may be completed with any emerging technology that may replace any of the above-mentioned implementation technologies.
[0059] In addition, the above description provides examples without limiting the scope, applicability or configuration set forth in the claims. Without departing from the spirit and scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various embodiments may appropriately omit, replace or add various processes or components. For example, features described with respect to certain embodiments may be combined in other embodiments.
[0060] In addition, in the description of the present disclosure, the terms “first”, “second”, “third”, etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance and order.
[0061] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing can be advantageous.
Claims
1. A Redis cluster query method, comprising: Obtaining a query request, wherein the query request includes a cursor parameter, the cursor parameter includes a first preset number of node index bits representing a node index, and a second preset number of array index bits representing an array index, the cursor parameter being divided into two consecutive parts representing the node index and the array index respectively; Parsing the node index and the array index according to the cursor parameter, and determining the node corresponding to the node index as the starting node of the iterative query data, and the position corresponding to the array index as the starting position of the iterative query data; and Starting from the starting position, data is obtained from the Redis cluster according to the query request.
2. The Redis cluster query method according to claim 1, wherein: The node index bit of the cursor parameter is located before the array index bit.
3. The Redis cluster query method according to claim 1, wherein: The first preset number of bits N1 satisfies: N1=[lg(N0)]+1, Wherein, N0 is the total number of nodes in the Redis cluster.
4. The Redis cluster query method according to claim 1, wherein: Parsing the node index and array index according to the cursor parameter includes: Obtain node information in the Redis cluster; Determining the first preset number of bits and the second preset number of bits according to the node information; and The node index is determined according to the first preset number of node index bits, and the array index is determined according to the second preset number of array index bits.
5. The Redis cluster query method according to claim 4, wherein: The node information includes: The total number of nodes in the Redis cluster; or A current query node and a non-zero cursor parameter corresponding to the current query node.
6. The Redis cluster query method according to claim 1, wherein: The query request also includes condition parameters and a preset number; Starting from the starting position, obtaining data from the Redis cluster according to the query request includes: In the node where the starting position is located, starting from the starting position, determining in sequence whether the data in the array meets the conditional parameters, and updating the array index to correspond to the current position; When data satisfying the condition parameters is found, the data is merged into the result data; When the number of data items in the result data reaches the preset number, scanning of the node is stopped and the result data is output.
7. The Redis cluster query method according to claim 6, wherein: Starting from the starting position, acquiring data from the Redis cluster according to the query request further includes: When the query of the current node is completed and the number of data items in the result data does not reach the preset number, the node index is updated to correspond to the next node, and the array index is updated to zero; According to the updated node index and array index and the conditional parameters, data is obtained from the Redis cluster until the number of data items in the result data reaches the preset number, and the result data is output.
8. A Redis cluster query device, comprising: a memory having instructions stored therein; as well as A processor is configured to execute instructions stored on the memory to perform the Redis cluster query method according to any one of claims 1 to 7.
9. A computer-readable storage medium, comprising computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the Redis cluster query method according to any one of claims 1 to 7.
10. A computer program product, comprising computer instructions, which, when executed by one or more processors, implement the steps of the Redis cluster query method according to any one of claims 1 to 7.
