Data storage method, device and equipment

Abstract

The present application provides a data storage method, apparatus, and device, which are applied to a server, including: receiving a storage location query request sent by a client, the storage location query request carrying key-value information of data to be processed; the data to be processed includes at least one key-value information; based on the storage location query request, determining the target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list represents each key-value information in each storage shard and links them in the form of a bidirectional redundant chain, the bidirectional redundant linked list including label information corresponding to the target storage shard; based on the label information of the target storage shard, determining the first storage location information of the data to be processed, and sending the first storage location information to the client for the client to query or store the data to be processed. The present application can improve the scalability and persistence of data storage, thereby improving data storage efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing, and relates to but is not limited to a data storage method, device and equipment. Background Art

[0002] Currently, storage systems for Key-Value (key-value-data) data include stand-alone storage systems and distributed storage systems. Stand-alone Key-Value storage systems are limited by the storage capacity of a single machine, resulting in a limited storage scale. Therefore, stand-alone storage systems are suitable for embedded scenarios or Key-Value data storage scenarios with smaller data volumes. Distributed Key-Value storage systems focus on caching and have weaker persistence capabilities. Therefore, distributed storage systems are suitable for small-scale Key-Value data storage scenarios with lower persistence requirements.

[0003] Therefore, for storage scenarios that process large-scale Key-Value data, the storage efficiency of Key-Value data is low due to the weak scalability and persistence capabilities of single-machine storage systems and distributed storage systems. Summary of the Invention

[0004] In view of this, the data storage method, device and equipment provided in this application are suitable for large-scale Key-Value data storage scenarios, and can improve the scalability and persistence of data storage, thereby improving the storage efficiency of Key-Value data.

[0005] According to one aspect of an embodiment of the present application, a data storage method is provided, which is applied to a server, including:

[0006] Receive a storage location query request sent by a client, wherein the storage location query request carries key value information of data to be processed; the data to be processed includes at least one key value information;

[0007] Based on the storage location query request, determining a target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list represents that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes label information corresponding to the target storage shard;

[0008] According to the tag information of the target storage shard, first storage location information of the data to be processed is determined, and the first storage location information is sent to the client for the client to query or store the data to be processed.

[0009] According to one aspect of an embodiment of the present application, a data storage method is provided, which is applied to a client and includes:

[0010] Sending a storage location query request to the server, wherein the storage location query request carries key value information of the data to be processed; the data to be processed includes at least one key value information;

[0011] Receive first storage location information sent by the server in response to the storage location query request, and query or store the data to be processed based on the first storage location information.

[0012] According to one aspect of an embodiment of the present application, a server is provided, including a management node, at least one cache node, and at least one storage point, wherein:

[0013] The cache node is configured to receive a storage location query request sent by a client, wherein the storage location query request carries key value information of data to be processed; the data to be processed includes at least one key value information;

[0014] The cache node is further configured to determine, based on the storage location query request, a target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list indicates that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes tag information corresponding to the target storage shard;

[0015] The cache node is further used to determine the first storage location information of the data to be processed based on the tag information of the target storage shard, and send the first storage location information to the client for the client to query or store the data to be processed.

[0016] According to one aspect of an embodiment of the present application, a client is provided, including a sending unit and a receiving unit, wherein:

[0017] The sending unit is configured to send a storage location query request to the server, wherein the storage location query request carries key value information of the data to be processed; the data to be processed includes at least one key value information;

[0018] The receiving unit is configured to receive first storage location information sent by the server in response to the storage location query request, and perform query or storage processing on the data to be processed based on the first storage location information.

[0019] According to one aspect of an embodiment of the present application, a data storage device is provided, including: a memory for storing executable instructions; a processor for implementing the method described in the server side of the embodiment of the present application when executing the executable instructions stored in the memory; or implementing the method described in the client side of the embodiment of the present application.

[0020] According to one aspect of an embodiment of the present application, a server is provided, comprising a first memory and a first processor; wherein the first memory is used to store executable instructions; and the first processor is used to implement the method described in the server side of the embodiment of the present application when executing the executable instructions stored in the first memory.

[0021] According to one aspect of an embodiment of the present application, a client is provided, comprising a second memory and a second processor; wherein the second memory is used to store executable instructions; and the second processor is used to implement the method described on the client side of the embodiment of the present application when executing the executable instructions stored in the second memory.

[0022] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores executable instructions. When the executable instructions are executed, the first processor is used to execute the method described on the server side of the embodiment of the present application; or the second processor is used to execute the method described on the client side of the embodiment of the present application.

[0023] In an embodiment of the present application, a data storage method is proposed, which is applied to a server. First, the server receives a storage location query request sent by a client; then, based on the storage location query request, the server determines the target storage shard corresponding to the key-value information in the constructed bidirectional redundant linked list; finally, the server determines the first storage location information of the data to be processed based on the tag information of the target storage shard, and sends the first storage location information to the client for the client to query or store the data to be processed. On the one hand, the bidirectional redundant linked list represents that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, that is, the cache node caches each key-value information in each storage shard in a redundant manner in the memory of different cache nodes, thereby achieving the purpose of scalability and fast indexing. On the other hand, the cache node organizes the locally ordered and overall disordered Key-Value data (data to be processed) in the form of a bidirectional redundant chain. The cache node stores each key-value information in each storage shard in the order of each key-value information, and links all storage shards in the order of each key-value information using a bidirectional redundant chain, and redundancies each key-value information in each storage shard. In this way, when a storage node or cache node fails, the data can be restored from the redundant copy of the storage node, thereby improving the accuracy of data storage or data query.

[0024] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, serve to illustrate the technical solutions of the present application. Obviously, the drawings described below are merely some embodiments of the present application. Those skilled in the art can, without inventive effort, derive other drawings from these drawings.

[0026] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0027] Figure 1 A schematic diagram of a network architecture that may be applicable to an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 1 ;

[0029] Figure 3 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 2 ;

[0030] Figure 4 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 3 ;

[0031] Figure 5 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 4 ;

[0032] Figure 6 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 5 ;

[0033] Figure 7 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 6 ;

[0034] Figure 8 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 7 ;

[0035] Figure 9 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 8 ;

[0036] Figure 10 A schematic diagram of a data storage method provided in an embodiment of the present application Figure 9 ;

[0037] Figure 11 A schematic diagram of the structure of a cache node provided in an embodiment of the present application;

[0038] Figure 12 A schematic diagram of the structure of another cache node provided in an embodiment of the present application;

[0039] Figure 13 A schematic diagram of the structure of a server provided in an embodiment of the present application Figure 1 ;

[0040] Figure 14 A schematic diagram of the structure of a client provided in an embodiment of the present application Figure 1 ;

[0041] Figure 15 A schematic diagram of the structure of a server provided in an embodiment of the present application Figure 2 ;

[0042] Figure 16A schematic diagram of the structure of a client provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0043] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but are not intended to limit the scope of the present application.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0045] In the following description, references to “some embodiments,” “this embodiment,” “embodiments of the present application,” and examples, etc., describe a subset of all possible embodiments. However, it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0046] If similar descriptions of "first / second" appear in the application documents, the following explanation is added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0047] The network architecture and service scenarios described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Persons skilled in the art will appreciate that, with the evolution of network architecture and the emergence of new service scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0048] In some embodiments of the present application, a data storage method is provided, which is applied to a data storage device and a data storage equipment. The data storage device includes a server and a client, and the data storage equipment includes a server and a client. The data storage equipment is a physical device corresponding to the data storage device.

[0049] Figure 1 A schematic diagram of a network architecture that may be applicable to the embodiment of the present application is provided, such as Figure 1As shown, the application environment involved in the embodiment of the present application includes a client 100 and a server 200, wherein the client 100 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. Figure 1 As shown, the server 200 is implemented by a server cluster composed of multiple server clusters. The server 200 can also be clustered and deployed on the cloud. The server 200 can also be clustered and deployed on multiple physical machines. This embodiment does not impose any restrictions on this. Figure 1 As shown, the server 200 includes a management node 210, at least one cache node 220 ( Figure 1 221, cache node 222, and cache node 223) and at least one storage node 230 ( Figure 1 231, storage node 232, storage node 233, and storage node 234 are shown in FIG. 1 . It should be noted that: Figure 1 The management node, each cache node and each storage node are only an example. The management node, each cache node and each storage node can be divided into different nodes in the same server, and this embodiment does not impose any limitation on this.

[0050] In some embodiments of the present application, the client is used to send a read request or a write request to the server to store the Key-Value data in the server, or to query the Key-Value data in the server.

[0051] In some embodiments of the present application, the management node is responsible for managing all cache nodes (cache node 221, cache node 222 and cache node 223). The management node is also used to divide the capacity range for each cache node. The management node is also used to provide the client and each cache node with relevant information of each storage node, for example, providing the IP address (Internet Protocol Address) and port information of each storage node.

[0052] In some embodiments of the present application, the cache node is used to cache the key value information of all Key-Value data, and provide the client with the storage location information of the storage node of the Key-Value data when the client stores or writes the Key-Value data.

[0053] In some embodiments of the present application, the storage node is a storage engine for Key-Value data, and is used to store Key-Value data.

[0054] It should be noted that when the cache node in the server is enabled, the cache node can connect to the network by accessing the interface of the management node, thereby realizing communication between the management node and the cache node.

[0055] Figure 2 A flow chart of a data storage method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, Figure 1 Taking the application environment shown in FIG. 1 as an example, the method may include the following steps 301 to 303:

[0056] Step 310: Receive a storage location query request sent by the client, where the storage location query request carries key-value information of the data to be processed; the data to be processed includes at least one key-value information.

[0057] In some embodiments of the present application, a client sends a storage location query request to a cache node in a server, and the cache node receives the storage location query request sent by the client. Storage location query requests are divided into read requests and write requests, wherein a read request indicates that the client performs query processing on the data to be processed, and a write request indicates that the client performs storage processing on the data to be processed. Here, the data to be processed is key-value data, which can also be understood as typical data. The data to be processed includes at least one key value information and one data information, and the key value information and the data information are in one-to-one correspondence.

[0058] In some embodiments of the present application, the client may first query the target cache node corresponding to the key-value information of the data to be processed through a locally stored bidirectional redundant linked list, and then the client may send a storage location query request to the target cache node, where the storage location query request carries the key-value information of the data to be processed. It should be noted that the bidirectional redundant linked list can be obtained by the client through the management node, and the bidirectional redundant linked list can be kept updated in real time. In other words, after the client stores the key-value data through the server, the management node sends the updated bidirectional redundant linked list to the client, thereby ensuring that the bidirectional redundant linked list stored locally by the client is the latest list.

[0059] Step 320: Based on the storage location query request, determine the target storage shard corresponding to the key-value information in the constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list represents that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes label information corresponding to the target storage shard.

[0060] In some embodiments of the present application, based on a storage location query request, a target cache node in a server determines the target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list. It should be noted that the server may include multiple cache nodes, each of which includes multiple storage shards, each of which stores multiple key-value information. Here, the key-value information within each storage shard in different cache nodes is linked in the form of a bidirectional redundant chain.

[0061] In some embodiments of the present application, each storage slice has a tag information, and the tag information is a unique identifier of the storage slice.

[0062] In some embodiments of the present application, the double-layer redundant linked list is pre-built, and the cache node builds a bidirectional redundant linked list based on each historical key value information in the historical data to be processed, such as Figure 3 As shown, it includes steps 321 to 324:

[0063] Step 321: The management node divides the capacity range for at least one cache node according to each historical key value information in the historical data to be processed.

[0064] In some embodiments of the present application, the management node divides the capacity range of at least one cache node according to each historical key-value information in the historical data to be processed. The historical data to be processed is key-value data, having multiple historical key-value information, and the historical key-value information corresponds one-to-one with the historical data information.

[0065] For example, if the number of historical data to be processed is 100 and the server has 3 cache nodes, then the management node divides the capacity range of 40 key-value information for each cache node according to the number of historical data to be processed. This can maintain the capacity balance of each cache node, thereby improving the scalability of the cache node and achieving the purpose of fast indexing.

[0066] It is understandable that each cache node contains multiple storage nodes, and the range divided by the management node for each cache node can be considered as an ordered list containing multiple storage shards.

[0067] Step 322: Based on the capacity range of each cache node, store each historical key-value information in the order of each historical key-value information in the historical data to be processed into each storage shard in each cache node.

[0068] In some embodiments of the present application, based on the capacity ranges of each cache node, each cache node stores each historical key-value information in the order of the historical key-value information in the historical data to be processed, in each storage shard within each cache node. It is understood that each historical key-value information can be in alphabetical or numeric type, and this application does not impose any limitations on this. Here, the order of the historical key-value information can also be considered to be the order of the dictionary.

[0069] Step 323: Determine the label information of each storage shard based on the historical key value information of the historical data to be processed.

[0070] In some embodiments of the present application, a cache node in a server determines the tag information of each storage shard based on the historical key value information of the historical data to be processed. It is understood that each storage node has a unique tag information, and once the tag information of a storage shard is determined, it will not be modified again.

[0071] Step 324: Link the historical key-value information in each storage shard in the form of a bidirectional redundant chain, and construct a bidirectional redundant linked list with the index information of each storage shard. Send the constructed bidirectional redundant linked list to the client. The index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs.

[0072] In some embodiments of the present application, the cache node first links the historical key-value information in each storage shard in the form of a bidirectional redundant chain.

[0073] As an implementable approach, first, for each historical key-value information in each storage shard, the cache node links each key-value information in each storage shard in the form of a bidirectional redundant chain based on the order of each historical key-value information. Subsequently, for each storage shard in each cache node, the cache node links each storage shard in different cache nodes in the form of a bidirectional redundant chain based on the storage shard's tag information and the location information of the cache node to which the storage shard belongs, and constructs a bidirectional redundant linked list with the index information of each storage shard.

[0074] For example, when historical key-value information (Key-Value data) is written to the storage system (storage node), the cache node (multi-layer cache system) stores it in different storage shards in the order of each historical key-value information. Each storage shard is organized in the form of a bidirectional redundant chain, and all historical key-value data are organized in a continuous and ordered linked list (bidirectional redundant linked list) in the order of each historical key-value information, and the index information of the adjacent storage shards of each storage shard is redundant.

[0075] In some embodiments of the present application, for each historical key-value information in each storage shard, the cache node can add a bidirectional pointer to each historical key-value information according to the order of each key-value information to link each historical key-value information in the form of a bidirectional redundant chain.

[0076] In some embodiments of the present application, for each storage shard at each cache node, the cache node adds the index information of each storage node's neighboring storage nodes to a bidirectional redundant linked list organized with each historical key value information. On this basis, each storage shard also stores the historical key value information of the previous storage node, thereby linking the storage shards at different cache nodes through a bidirectional redundant chain.

[0077] Exemplarily, for each storage shard, each storage shard will store the index information of the previous storage shard, and the index information includes the tag information of the storage shard and the location information of the cache node to which the storage shard belongs. In addition, based on the bidirectional linked list, each storage shard will also store the index information of at least one subsequent storage shard, such as the index information of the subsequent storage shard, the subsequent two storage shards, or the subsequent three storage shards. On this basis, each storage shard also stores a complete data copy of the previous (previous) storage shard. When a cache node is damaged, the cache node (multi-layer cache system) can find the corresponding redundant copy by querying the link order of each storage shard, thereby completing data recovery.

[0078] It can be understood that the bidirectional redundant linked list (also called a bidirectional redundant linked list system) for each storage shard provides continuous linked list capability (also called list capability) by adding consecutive index information to each storage shard based on the order of historical key value information. When a client (user) initiates a linked list query, the subsequent storage shard can directly obtain the location of the key value information of the subsequent storage shard (also called the subsequent storage shard) by using the index information of the subsequent storage shard (also called the subsequent storage shard) recorded in the bidirectional redundant linked list.

[0079] Step 330: Determine the first storage location information of the data to be processed based on the tag information of the target storage shard, and send the first storage location information to the client for the client to query or store the data to be processed.

[0080] In some embodiments of the present application, since each storage shard in a bidirectional redundant linked list has unique tag information, the target cache node first determines the first storage location information of the data to be processed based on the tag information of the target storage shard. The first storage location information is the location information of the target storage node corresponding to the target storage shard. The target cache node then sends the first storage location information to the client for the client to query or store the data to be processed. Thus, the client can obtain the location information of the target storage node corresponding to the key value information of the data to be processed based on the first storage location information, so that the client can query or store the data to be processed.

[0081] In some embodiments of the present application, on the one hand, a bidirectional redundant linked list represents that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, that is, the cache node caches each key-value information in each storage shard in a redundant manner in the memory of different cache nodes, thus achieving the purpose of scalability and fast indexing. On the other hand, the cache node organizes the locally ordered and overall disordered Key-Value data (data to be processed) in the form of a bidirectional redundant chain, and the cache node stores each key-value information in each storage shard in the order of each key-value information, and links all storage shards in the order of each key-value information in the form of a bidirectional redundant chain, and redundancies each key-value information in each storage shard. In this way, when a storage node or a cache node fails, the data can be recovered from the redundant copy of the storage node, thus improving the accuracy of data storage or data query.

[0082] like Figure 4 As shown, step 323 includes steps 401 to 404:

[0083] Step 401: For the current storage slice, obtain the starting key value information in the current storage slice and the previous ending key value information in the previous storage slice.

[0084] In some embodiments of the present application, for the current storage shard, the cache node (also called a multi-layer cache system) obtains the starting key value information in the current storage shard and the previous ending key value information in the previous storage shard. Exemplarily, the range of each historical key value information (also called Key value) in the current storage shard is "egt-fab", and the range of each historical key value information in the previous storage shard is "aaa-efa". At this time, the starting key value information in the current storage shard is "egt", and the previous ending key value information in the previous storage shard is "efa".

[0085] Step 402: Obtain the longest common prefix of the starting key value information and the previous ending key value information in the current storage shard.

[0086] In some embodiments of the present application, the longest common prefix of the starting key information and the previous ending key information in the current storage shard of the cache node is stored. For example, for the label information "Aaaaf" and the label information "Aaaae", the longest common prefix is ​​"Aaaa". For the starting key information "egt" and the previous ending key information "efa", the longest common prefix is ​​"e".

[0087] Step 403: Based on the longest common prefix and the preset number of label bits, determine the shortest different prefix of the current storage shard, where the shortest different prefix is ​​the label information of the current storage shard.

[0088] In some embodiments of the present application, based on the longest common prefix and the preset number of label bits, the cache node determines the shortest different prefix of the current storage shard, where the shortest different prefix is ​​the label information of the current storage shard. The preset number of label bits may be the number of label bits added to the longest common prefix, or the number of label bits of the label information of the current storage shard, and this application does not impose any restrictions on this.

[0089] For example, the preset number of tag bits is the number of tag bits added to the longest common prefix. For the starting key value "egt" and the previous ending key value "efa", the longest common prefix is ​​"e". Since the preset number of tag bits is 1, one bit "g" is added to the longest common prefix "e" to obtain the shortest different prefix "eg" for the current storage shard. This shortest different prefix is ​​the tag information for the current storage shard.

[0090] Step 404: Continue to obtain the starting key value information of the next storage shard and the ending key value information of the current storage shard, thereby determining the tag information of the next storage shard, until the tag information of all storage shards is obtained, and obtain the tag information of each storage shard.

[0091] In some embodiments of the present application, the cache node continues to obtain the starting key value information of the next storage shard and the ending key value information of the current storage shard, thereby determining the label information of the next storage shard, until the label information of all storage shards in the bidirectional redundant linked list is obtained, and the label information of each storage shard is obtained.

[0092] like Figure 5 As shown, step 330 includes steps 331 to 332:

[0093] Step 331: According to the label information of the target storage shard, the target storage node information corresponding to the target storage shard is determined by a preset algorithm. The target storage node information represents the location information of the target storage node.

[0094] In some embodiments of the present application, a cache node determines the target storage node information corresponding to the target storage shard based on the tag information of the target storage shard using a preset algorithm, wherein the target storage node information represents the location information of the target storage node. It should be noted that each storage node in the server has a unique identifier, which can be a serial number, for example, 0 to 100.

[0095] In some embodiments of the present application, a management node may store a storage location list of storage nodes, which records the storage node identifiers and storage node address information. It is understood that there is a one-to-one correspondence between the storage node identifiers and the storage node location information. For example, storage node 0 corresponds to location information 192.XXX.XXX.1, and storage node 2 corresponds to location information 192.XXX.XXX.2.

[0096] As a feasible method, for the target storage shard, the target cache node first converts the label information of the target storage shard to obtain the corresponding target digital label; then, the target cache node determines the target storage node information based on the maximum serial number identifier and target digital label in each storage node.

[0097] Exemplarily, a cache node (also called a multi-layer cache system) randomly selects a storage node as the storage location for the data to be processed through a preset algorithm. Each storage node has a serial number, and the serial number (also called an identifier) ​​of each storage node can be a continuous range of numbers starting from 0. First, the target cache node converts the tag information (also called a hash tag) of the target cache node to obtain the corresponding target data tag (number). Subsequently, the target cache node performs a modulo calculation using the target data tag (number) and the maximum serial number identifier of each storage node, and the resulting digital result is the storage location of the target storage shard (target storage node information).

[0098] Step 332: Based on the target storage node information, obtain the first storage location information of the data to be processed.

[0099] In some embodiments of the present application, since the target storage node information is the location information of the target storage node corresponding to the data to be processed, the target cache node obtains the first storage location information of the data to be processed based on the target storage node information.

[0100] In some embodiments of the present application, in the process of storing the current historical key-value information in the current storage shard, the cache node (multi-layer cache system) determines whether the various historical key-value information in the current storage shard exceeds a preset quantity range.

[0101] Here, the preset number range is the maximum number of key-value information that can be stored in the storage shard, and the preset number range can be set in advance. It is understandable that the client can set the preset number range of the storage shard through the management node.

[0102] In some embodiments of the present application, if the historical key-value information in the current storage shard does not exceed a preset quantity range, the cache node stores the current historical key-value information in the current storage shard.

[0103] In some embodiments of the present application, if the historical key-value information in the current storage shard exceeds a preset number range, the cache node will treat the part of the current storage shard that exceeds the preset number range as a new storage shard and migrate it to the next cache node based on the tag information of the current storage shard; the storage shards after the current storage shard are migrated in turn so that the capacity of each cache node is balanced.

[0104] It should be noted that each storage shard has a certain limit on the number of Key-Value data (key-value information or historical key-value information). When the cache node (multi-layer cache system) is initialized, it establishes a storage shard with a fixed-bit short label based on the number of each historical key-value information and the order of each historical key-value information (lexicographic order). For example, if the number of bits of the label information of the storage shard is specified to be 3, and the legal name characters of the label information are only lowercase letters, then the label information of the storage shard is "aaa" to "zzz". If the current storage shard is inserted with the increase of Key-Value data, it may exceed the limit of the preset number range. At this time, the current storage shard can be split from the middle, and the latter half of the storage shard will be migrated to the new location (cache node) as the new storage shard, and the new storage shard will also generate a new label.

[0105] In some embodiments of the present application, Figure 6 As shown, step 320 includes steps 501 to 502:

[0106] Step 501: The storage location query request is a read request. Based on the read request, a target storage shard corresponding to the key-value information is determined in the constructed bidirectional redundant linked list. The key-value information is one of the historical key-value information in the target storage shard.

[0107] In some embodiments of the present application, the storage location query request sent by the client to the server is a read request, which indicates that the client is reading data information corresponding to key-value information through the server. It is understood that the client can first determine the target cache node corresponding to the key-value information of the data to be processed in a locally stored bidirectional redundant linked list. The client then sends a read request to the target cache node, which carries the key-value information of the data to be processed.

[0108] In some embodiments of the present application, a bidirectional redundant linked list is maintained in the client in the order of each cache node, and the bidirectional redundant linked list can also be understood as a range information list. For each cache node, the bidirectional redundant linked list records the label information of the starting storage shard in the cache node (the first shard label) and the label information of the next storage shard, which can be represented by a left-closed and right-open interval. It can be understood that when the client stores or queries the data to be processed (Key-Value data) through the server, the client first determines the target cache node where the target storage shard storing the key-value information of the data to be processed is located through the locally stored bidirectional redundant linked list, and then the client sends a storage location query request to the target cache node. All storage shards cached locally by each cache node are continuous and ordered. In this way, the target cache node can find the index information of the target storage shard by binary search, and return the target storage information to the client for the client to query or store the data to be processed.

[0109] In some embodiments of the present application, after the target cache node in the server receives a read request sent by the client, the target cache node determines the target storage shard corresponding to the key-value information in the constructed bidirectional redundant linked list, where the key-value information is one of the various historical key-value information in the target storage shard.

[0110] Step 502: The storage location query request is a write request. Based on the write request, when the key value information is one of the historical key value information in the constructed bidirectional redundant linked list, the target storage shard corresponding to the key value information is determined in the bidirectional redundant linked list. When the key value information is not one of the historical key value information in the constructed bidirectional redundant linked list, the target storage shard where the key value information should be stored is determined in the bidirectional redundant linked list, and the key value information is inserted into the target storage shard in the order of the historical key value information in the target storage shard.

[0111] In some embodiments of the present application, the storage location query request sent by the client to the server is a write request, which indicates that the client stores the data information corresponding to the key-value information in the server's storage node through the server. It is understood that when the key-value information is one of the historical key-value information in the bidirectional redundant linked list, the client can first determine the target cache node corresponding to the key-value information of the data to be processed in the locally stored bidirectional redundant linked list. Subsequently, the client sends a write request to the target cache node, which carries the key-value information of the information to be processed.

[0112] In the case where the key value information is not one of the various historical key value information in the constructed bidirectional redundant linked list, first, the client can determine the target cache node corresponding to the key value information of the data to be processed in the locally stored bidirectional redundant linked list. Specifically, the target cache node corresponding to the key value information can be determined by the information of the storage shards of each cache node recorded in the bidirectional redundant linked list. Subsequently, the client sends a write request to the target cache node, and the write request carries the key value information of the information to be processed. Finally, the target cache node determines the target storage shard where the key value information should be stored according to the order of the various historical key value information in each storage shard in the bidirectional redundant linked list, and inserts the key value information into the target storage shard according to the order of the various historical key value information in the target storage shard.

[0113] Here, if the target storage shard exceeds the preset number range of the target storage shard after inserting the key-value data of the data to be processed, the target storage node can split the part that exceeds the preset number range, and the part that exceeds the preset number range will form a new storage shard. Subsequent storage shards will be migrated to the corresponding cache node at one time.

[0114] In some embodiments of the present application, Figure 7 As shown, step 330 includes steps 601 to 602:

[0115] Step 601: The storage location query request is a read request. The first storage location information is sent to the client. The target storage node corresponding to the first storage location information receives the first data query request sent by the client. The first data query request carries the key value information of the data to be processed.

[0116] In some embodiments of the present application, if the storage location query request sent by the client to the server is a read request, the cache node determines the target storage shard corresponding to the key value information of the data to be processed in the constructed double-layer redundant linked list, and determines the location information (first storage location information) of the target storage node corresponding to the key value information through the tag information of the target storage shard. The target cache node of the target storage shard sends the first storage location information to the client, and the client can obtain the location information of the target storage node corresponding to the key value information of the data to be processed through the first storage location information. The location information of the target storage node can be the IP address information and port information of the target storage node.

[0117] In some embodiments of the present application, the client sends a first data query request to the target storage node based on the location information of the target storage node, wherein the first data query request carries the key value information of the data to be processed. After the target storage node in the server receives the first data query request sent by the client, it uses a preset algorithm to obtain the data information corresponding to the key value information of the data to be processed. It should be noted that the present application does not impose any restrictions on the method of finding the data information corresponding to the key value information by using binary search, sequential search, interpolation search, tree table search or sorting algorithm, etc., and the corresponding algorithm can be selected according to the actual situation.

[0118] In some embodiments of the present application, after obtaining the information data corresponding to the key-value information, the target storage node sends a first data query response to the client, and the first data query response carries the data information corresponding to the key-value information, completing the client's query processing of the data to be processed.

[0119] Step 602: The storage location query request is a write request. The first storage location information is sent to the client. The target storage node corresponding to the first storage location information receives the first data storage request sent by the client. The first data storage request carries the key value information and data information of the data to be processed.

[0120] In some embodiments of the present application, if the storage location query request sent by the client to the server is a write request, the cache node determines the target storage shard corresponding to the key value information of the data to be processed in the constructed double-layer redundant linked list, and determines the location information (first storage location information) of the target storage node corresponding to the key value information through the tag information of the target storage shard. The target cache node of the target storage shard sends the first storage location information to the client, and the client can obtain the location information of the target storage node corresponding to the key value information of the data to be processed through the first storage location information. The location information of the target storage node can be the IP address information and port information of the target storage node.

[0121] In some embodiments of the present application, according to the target storage node corresponding to the first storage location information, the key-value information of the data to be processed and the data information corresponding to the key-value information are stored in the target storage node, and a first data storage response is sent to the client. The first data storage response carries the storage result of the data to be processed, completing the client's storage processing of the data to be processed.

[0122] Here, if the storage location query request sent by the client to the server is a write request, and the key value information of the data to be processed is not recorded in the constructed double-layer redundant linked list. The client can first determine the target cache node where the key value information should be stored based on the locally stored bidirectional redundant linked list, and then the client sends a first data storage request to the target cache node. After receiving the first data storage request sent by the client, the target cache node determines the target storage shard where the key value data should be stored according to the order of each historical key value data in the constructed bidirectional redundant linked list, and inserts the key value information into the corresponding position of the target storage shard. Subsequently, the target cache node sends the location information of the target storage node of the target storage shard to the client, and the client sends the first data storage request to the target storage node. Finally, the target storage node stores the key value information and data information in the target storage node, completing the client's storage processing of the data to be processed.

[0123] In some embodiments of the present application, a data storage method is proposed, which is applied to a client, such as Figure 8 As shown, the method includes steps 710 to 720:

[0124] Step 710: Send a storage location query request to the server, where the storage location query request carries key value information of the data to be processed; the data to be processed includes at least one key value information.

[0125] In some embodiments of the present application, the client sends a storage location query request to the server, and the storage location query request carries key value information of the data to be processed.

[0126] Here, the client can first determine the target cache node corresponding to the key value information of the data to be processed in the locally stored bidirectional redundant linked list. The client then sends a storage location query request to the target cache node. The storage location query request can be divided into a read request and a write request.

[0127] Step 720: Receive first storage location information sent by the server in response to the storage location query request, and perform query or storage processing on the data to be processed based on the first storage location information.

[0128] In some embodiments of the present application, after the client sends a storage location query request to the target cache node, the target cache node determines the location information (first storage location information) of the target storage node corresponding to the key-value data of the data to be processed based on the key-value data of the data to be processed, and sends the first storage location information to the client. The client queries or performs storage processing on the data to be processed based on the first storage location information.

[0129] In some embodiments of the present application, Figure 9 As shown, step 720 includes steps 721 to 722:

[0130] Step 721: The storage location query request is a read request. Based on the first storage location information, a first data query request is sent to the server, and the first data query request carries the key value information of the data to be processed; a first data query response sent by the server in response to the first data query request is received, and the first data query response carries the data information corresponding to the key value information, thereby completing the query processing of the data to be processed.

[0131] In some embodiments of the present application, when the storage location query request is a read request, the client first obtains the location information of the target storage node through the first storage location information, and then the client sends a first data query request to the target storage node. After the target storage node receives the first data query request sent by the client, the target storage node obtains the data information corresponding to the key-value information carried in the first data query request and sends a first data query response to the client, the first data query response carrying the data information corresponding to the key-value information.

[0132] In some embodiments of the present application, after receiving the first data query request sent by the client, the target storage node in the server uses a preset algorithm to obtain the data information corresponding to the key-value information of the data to be processed. It should be noted that the present application does not impose any restrictions on the method of using a binary search, sequential search, interpolation search, tree table search, or sorting algorithm to obtain the data information corresponding to the key-value information of the data to be processed, and the corresponding algorithm can be selected according to the actual situation.

[0133] Step 722: The storage location query request is a write request. Based on the first storage location information, a first data storage request is sent to the server. The first data storage request carries the key value information and data information of the data to be processed. The server receives a first data storage response sent in response to the first data query request. The first data storage response carries the storage result of the data to be processed, thereby completing the storage processing of the data to be processed.

[0134] In some embodiments of the present application, when the storage location query request is a write request, the client can first obtain the location information of the target storage node through the first storage location information, and then the client sends a first data storage request to the target storage node. After the target storage node receives the first data storage request sent by the client, the target storage node stores the key information and corresponding data information carried by the first data storage request in the target storage node, and sends a first data storage response to the client, and the first data storage response carries the storage result of the data to be processed. It can be understood that if the target storage node stores the key information and data information of the data to be processed in the target storage node, the first data storage response carries information that the data storage is successful. Similarly, if the target storage node does not store the key information and data information of the data to be processed in the target storage node, the first data storage response carries information that the data storage failed.

[0135] In some embodiments of the present application, priority judgment is performed on the key value information of the data to be processed according to preset priority rules; if the key value information of the data to be processed is a first priority key value, the second storage location information of the data to be processed is obtained through a preset algorithm, and the data to be processed is queried or stored based on the second storage location information; if the key value information of the data to be processed is a first priority key value, the step of sending a storage location query request to the server is executed; the priority of the first priority key value is higher than the second priority key value.

[0136] In some embodiments of the present application, the client determines the priority of the key value information of the data to be processed according to a preset priority rule. For example, the client may specify a preset priority tag. If the key value information of the data to be processed is less than the preset priority tag, the key value information of the data to be processed is a first priority key value. If the key value information of the data to be processed is greater than the preset priority tag, the key value information of the data to be processed is a second priority key value. The first priority key value has a higher priority than the second priority key value.

[0137] In some embodiments of the present application, if the key value information of the data to be processed is a first-priority key value, the client obtains the second storage location information of the data to be processed using a preset algorithm. For example, the client may obtain the second storage location information of the data to be processed using a hash algorithm, where the second location information is the location information of the target storage node corresponding to the key value information.

[0138] When the client performs query processing on the data to be processed, the client sends a second data query request to the target storage node based on the location information of the target storage node, wherein the second data query request carries the key value information of the data to be processed; the client receives the second data query response sent by the server, wherein the second data query response carries the data information corresponding to the key value information, thereby completing the client's query processing on the data to be processed.

[0139] When the client performs storage processing on the data to be processed, the client sends a second data storage request to the target storage node based on the location information of the target storage node, wherein the second data query request carries the key value information and data information of the data to be processed; the client receives the second data storage response sent by the server, wherein the second data storage response carries the storage result of the data to be processed, thereby completing the client's storage processing of the data to be processed.

[0140] It should be noted that when the client performs query processing on the data to be processed, the client sends a second data query request to the target storage node based on the location information of the target storage node, wherein the second data query request carries the key value information of the data to be processed. After the target storage node in the server receives the second data query request sent by the client, it uses a preset algorithm to obtain the data information corresponding to the key value information of the data to be processed. It should be noted that the key value information of the data to be processed is searched for the data information corresponding to the key value information through binary search, sequential search, interpolation search, tree table search or sorting algorithm, etc. After obtaining the information data corresponding to the key value information, the target storage node sends a second data query response to the client, and the second data query response carries the data information corresponding to the key value information, thereby completing the client's query processing on the data to be processed.

[0141] It should be noted that when the client performs storage processing on the data to be processed, the client sends a second data storage request to the target storage node. After the target storage node receives the second data storage request sent by the client, the target storage node stores the key information and corresponding data information carried in the second data storage request in the target storage node, and sends a second data storage response to the client, and the second data storage response carries the storage result of the data to be processed. It can be understood that if the target storage node stores the key information and data information of the data to be processed in the target storage node, the second data storage response carries information of successful data storage. Similarly, if the target storage node does not store the key information and data information of the data to be processed in the target storage node, the second data storage response carries information of failed data storage. In this way, the client completes the storage processing of the data to be processed.

[0142] In some embodiments of the present application, when determining the location of a storage node (target storage node) where a storage shard (target storage shard) of key-value data (data to be processed) is located, a cache node (also known as a multi-layer cache system) can determine the location of the storage shard by combining a hash algorithm (Hash) with a storage shard index query. For example, for tag information (tag) with a length less than a fixed length, the cache node defaults to a first-priority tag, and a simple hash method can be used to quickly calculate the first-priority tag to obtain the storage location of the storage shard.

[0143] It should be noted that the storage location of a storage shard is determined using a hash algorithm and a random selection algorithm. Here, the storage location refers to the location information of the storage node corresponding to the storage shard. For storage shards whose storage location is determined using a hash algorithm, the storage location of the storage shard is also calculated using the hash algorithm during query. For storage shards whose storage location is determined using a random selection algorithm, the storage location of the storage shard is determined through a combination of cache node (multi-layer cache system) index query.

[0144] For example, a cache node (also known as a multi-layer cache system) can specify a preset priority tag length. For tags (tag information) with a length less than this fixed length, the multi-layer cache system will default to a first-priority tag (first-priority key value). Otherwise, the multi-layer cache system will default to a second-priority tag (second-priority key value). For first-priority tags, the multi-layer cache system uses a simple hashing method to quickly calculate the storage location of the data information associated with the first-priority tag. For second-priority tags, the multi-layer cache system randomly selects a storage node as the storage location for the data to be processed using a preset algorithm. Each storage node has a sequence number, and the sequence number (also known as an identifier) ​​of each storage node can be a continuous range of numbers starting from 0. First, the target cache node converts the tag information (also known as a hash tag) of the target cache node to obtain the corresponding target data tag (number). Subsequently, the target cache node performs a modulo calculation using the target data tag (number) and the maximum sequence number identifier of each storage node. The resulting numerical result is the storage location of the target storage shard (target storage node information).

[0145] It should be noted that when the client (user) uses the first priority tag as a prefix to query, and the cluster is not in a balanced state. The client does not need to go through the index cache system (also called a cache node or a multi-layer cache system), and can obtain the storage location of the storage shard through the hash algorithm. The rebalancing state here refers to the data migration that occurs when individual storage nodes fail and data is redistributed. If the cluster is in a rebalancing state, it is necessary to obtain the location information of the storage shard through the cache node. In this way, when the client (user) uses the first priority tag, it does not need to go through the index cache system, which will greatly speed up the response progress. When the prefix used by the client (user) does not fall within the range of the first priority tag, that is, when the client uses the second priority tag, it can be queried through the index cache system. Here, the priority of the first priority tag is higher than the second priority tag.

[0146] In addition, the cache node can establish a corresponding first-priority storage shard based on the client's preset first-priority tag, and store the key-value information corresponding to the first-priority tag in the first-priority storage shard, wherein the preset first-priority tag is pre-set by the client. The index cache system (also known as a cache node or a multi-layer cache system) can also allow the user (client) to actively specify a portion of the first-priority tags. For the specified first-priority tags, the index cache system (cache node) actively establishes the corresponding storage shard in advance, or splits the existing storage shard into new storage shards to store these first-priority tags.

[0147] It should be noted that regarding the calculation method for first-priority tags, the client (user) uses a hash algorithm to obtain the location information of the storage node corresponding to the storage shard (partitioned storage node). If the hash algorithm cardinality is different when adding storage nodes, the calculation results may be inconsistent, and migration may be necessary. If the scale of storage nodes changes, the cache node will determine whether the storage location corresponding to the first-priority tag needs to be migrated. If the storage location corresponding to the first-priority tag needs to be migrated, the cache node will select a portion of storage shards with the second-priority tag (second-priority key value) from the old cache node according to a preset ratio for migration, thereby achieving capacity balance across all cache nodes. In other words, when the cluster scale changes and storage nodes join or leave, the cluster will enter a rebalancing state, during which the index cache system will recalculate the location of the first-priority tag to determine whether it needs to be migrated. After the migration of the storage shards with the first-priority tag is completed, the remaining capacity of each cache node will be considered. The index cache system will select a batch of storage shards with the second-priority tag from the old cache node according to the ratio for migration, ultimately achieving capacity balance across all cache nodes and storage nodes.

[0148] In some embodiments of the present application, the storage location of the storage shard is determined by combining a hybrid hash algorithm (hash algorithm) and an index query. In this way, the storage location is determined by partial calculation and partial query, achieving a better balance in scalability and performance.

[0149] In some embodiments of the present application, a data storage method is proposed, such as Figure 10 As shown, it includes steps 801 to 809:

[0150] Step 801: According to the preset priority rules, the client performs a priority judgment on the key value information of the data to be processed; if the key value information of the data to be processed is a first priority key value, execute steps 802 to 804; if the key value information of the data to be processed is a second priority key value, execute steps 805 to 809, wherein the priority of the first priority key value is higher than that of the second priority key value.

[0151] Step 802: The client obtains second storage location information of the data to be processed through a preset algorithm.

[0152] Step 803: Based on the second storage location information, the client sends a second data query request to the server, or the client sends a second data storage request to the server.

[0153] Step 804: The server sends the second data query response to the client to complete the query processing of the data to be processed, or the server sends the second data storage response to the client to complete the storage processing of the data to be processed.

[0154] Step 805: The client sends a storage location query request to the server, where the storage location query request carries key value information of the data to be processed.

[0155] Step 806: Based on the storage location query request, the server determines the target storage shard corresponding to the key value information in the constructed bidirectional redundant linked list.

[0156] Step 807: The server determines the first storage location information of the data to be processed based on the tag information of the target storage shard, and sends the first storage location information to the client.

[0157] Step 808: Based on the first storage location information, the client sends a first data query request to the server, or the client sends a first data storage request to the server.

[0158] Step 809: The server sends the first data query response to the client to complete the query processing of the data to be processed, or the server sends the first data storage response to the client to complete the storage processing of the data to be processed.

[0159] In some embodiments of the present application, a data storage system (also called a distributed Key-Value storage system) is proposed, and the data storage system (also called a distributed Key-Value storage system) includes a client and a server. Among them, the server includes a management node, at least one cache node and at least one storage node. It should be noted that the client is used to send a request to query (read) or store (write) Key-Value data (data to be processed) to the server; the management node is used to manage all cache nodes and storage nodes (single storage node), and the management node is also used to divide the capacity range (range) for each cache node. The management node is also used to provide the client and the cache node with relevant information of the storage node (IP address and access port information of the storage node or single storage node, etc.); the storage node is a storage engine for stand-alone Key-Value data, which is used to store Key-Value data; the cache node is used to cache the index information of the Key-Value data storage shard, and the index information includes the label information and location information of the storage shard (where Key-Value data is stored), so that the client can determine the storage location information corresponding to the key value information (Key value) when querying (reading) or storing (writing) Key-Value data.

[0160] It should be noted that different cache nodes can cache storage shards with different ranges of tag information. The capacity range of cache nodes can be managed by the management node. The index information of cache nodes is organized using a multi-layer cache system. The cache of cache nodes is not persistent. In other words, the key-value information stored by the cache node is updated as the data is updated. When the cache node is enabled, the cache node connects to the network by accessing the interface of the management node. The cache node then communicates with the management node and storage nodes to build the corresponding cache information.

[0161] It's important to note that a multi-tiered cache system, also known as an index cache system or cache node, rapidly indexes storage shard index information using a two-tiered structure: range information (also known as capacity ranges or ranges) and storage shards. As you can see, each range is an ordered list of index information for multiple storage shards, each with multiple key-value pairs. The ranges of cache nodes are divided by the management node, which divides the index information for all storage shards into ranges and allocates multiple consecutive copies of the storage shards to each cache node, using two replicas for redundancy. Different cache nodes may have overlapping ranges (with the same storage shards) to redundancy, ensuring index query availability in the event of a single point of failure. For redundant ranges, the first position is prioritized when no single point of failure exists. The first position here refers to the first position (the storage node of the first replica) in the order of the storage nodes, from front to back.

[0162] Here, when key-value information (Key-Value data) is written into the storage system (data storage system or distributed Key-Value storage system), the cache node (multi-layer cache system) stores the Key-Value data in different storage shards according to the range and order of each key-value information (Key value). Each storage shard is organized in the form of a bidirectional redundant chain, and all key-value data (Key values) are organized in a continuous and ordered linked list (bidirectional redundant linked list) according to the order of each key-value information (Key lexicographic order), and the index information (label information, location information and key value information) of the adjacent storage shards of each storage shard is redundant.

[0163] In other words, key-value data sets are organized into storage shards (shards). Each storage shard stores a specified range of key-value data entries. All storage shards are serially linked together in lexicographic (key-value) order via a bidirectional redundant linked list. The key-value data within each storage shard is stored in lexicographic order, and the linked list organizes all storage shards into an ordered whole.

[0164] Figure 11 A schematic diagram of the structure of a cache node provided in an embodiment of the present application is shown as follows: Figure 11As shown, the cache nodes include cache node 1, cache node 2 and cache node 3. Among them, cache node 1 stores storage shard aa (label information), storage shard bb and storage shard cc, cache node 2 stores storage shard bb, storage shard cc and storage shard dd, and cache node 3 stores storage shard dd and storage shard aa. The cache node can be understood as a multi-level storage shard (shard) redundant cache system, which stores different storage shard indexes in the memory of different cache nodes according to a certain range. The cache node stores each key-value information in each storage shard in the key value order (Key order), and each storage shard storing the key-value information is organized in the form of a bidirectional redundant chain, organizing all Key-Value data in a continuous ordered linked list in the Key dictionary order, and redundancies the position information and index information of each storage shard with the adjacent storage shard on its linked list. As shown Figure 11 As shown, cache node 2 redundantly stores information about storage shards bb and cc in cache node 1. Cache node 3 redundantly stores information about storage shard dd in cache node 2 and storage shard aa in cache node 1. This way, if a cache node becomes corrupted, the cache system can find the corresponding redundant copy by querying the shard link order, allowing data recovery. This improves both the storage efficiency of key-value data and the accuracy of querying or storing key-value data.

[0165] Figure 12 A schematic diagram of the structure of another cache node provided in an embodiment of the present application is shown in FIG. Figure 12 As shown, cache node 1 stores key-value information aa to key-value information bb, and key-value information db to key-value information hb; cache node 2 stores key-value information hc to key-value information lb, and key-value information mn to key-value information zz; cache node 3 stores key-value information bc to key-value information da, and key-value information lc to key-value information mm. Each storage shard in cache node 1, cache node 2, and cache node 3 organizes the locally ordered and overall disordered Key-Value data in the form of a bidirectional redundant linked list to accelerate access to lexicographically ordered data. Each key-value information (Key value) is divided into multiple storage shards according to the order of the key-value information, and all storage shards are linked in the form of a bidirectional redundant chain according to the order of the key-value information, and each storage shard is redundantly stored. In this way, when the client (user) queries the Key-Value data, it can obtain the key value information of the Key-Value data in sequence according to the link order of each storage shard, and restore the corresponding data (index information, key value information or data information, etc.) from the redundant copy of the corresponding storage shard when the cache node or storage node is damaged.

[0166] In some embodiments of the present application, the data storage method and data storage system proposed in the present application have the following beneficial effects:

[0167] 1. The data storage system proposed in this application includes a management node, at least one cache node, and at least one storage node. The multi-level cache system (also known as an indexed redundant cache system, indexed redundant system, indexed query system, or cache node) redundantly caches key-value information that is not located in each storage shard in different cache nodes. Specifically, a bidirectional redundant linked list represents each key-value information in each storage shard, linking them in the form of a bidirectional redundant chain. This achieves the goal of scalability and fast indexing of the data storage system.

[0168] 2. The data storage method proposed in the present application is that the cache node organizes the locally ordered and overall disordered Key-Value data (data to be processed) in the form of a bidirectional redundant chain. The cache node stores each key-value information in each storage shard in the order of each key-value information, and links all the storage shards in the order of each key-value information using a bidirectional redundant chain, and redundancies each key-value information in each storage shard. In this way, when a storage node or a cache node fails, the data can be recovered from the redundant copy of the storage node, thereby improving the accuracy of data storage or data query.

[0169] 3. The data storage method proposed in this application determines the storage location of the storage shard by combining a hybrid hash algorithm (hash algorithm) and an index query. In this way, the storage location of the storage shard is determined by partial calculation and partial query, achieving a relatively good balance in scalability and performance.

[0170] In this way, the data storage method and data storage system (which can also be understood as a distributed Key-Value storage system) proposed in the embodiments of the present application have the characteristics of good scalability and persistence, and have relatively good efficiency in query or storage (writing). They are very friendly to the storage or query processing of lexicographical data and filtering according to the key (key value information) name prefix, and are suitable for large-scale, persistent Key-Value data storage or query scenarios.

[0171] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps; or steps in different embodiments may be combined to form a new technical solution.

[0172] Based on the above embodiments, the present application provides a server, such as Figure 13 As shown, the server 1000 includes a management node 1001, at least one cache node 1002 and at least one storage node 1003, wherein:

[0173] The cache node is used to receive a storage location query request sent by a client, where the storage location query request carries key value information of the data to be processed; the data to be processed includes at least one key value information; based on the storage location query request, a target storage shard corresponding to the key value information is determined in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list represents that each key value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes label information corresponding to the target storage shard; based on the label information of the target storage shard, first storage location information of the data to be processed is determined, and the first storage location information is sent to the client for the client to query or store the data to be processed.

[0174] The cache node is further configured to construct the bidirectional redundant linked list based on each historical key value information in the historical data to be processed.

[0175] The management node is used to divide the capacity range for at least one cache node according to each piece of historical key value information in the historical data to be processed.

[0176] The cache node is further used to store each historical key value information in each storage shard in each cache node based on the capacity range divided by each cache node and in the order of each historical key value information in the historical data to be processed; determine the label information of each storage shard based on each historical key value information of the historical data to be processed; link each historical key value information in each storage shard in the form of a bidirectional redundant chain, and construct the bidirectional redundant linked list with the index information of each storage shard, and send the constructed bidirectional redundant linked list to the client, wherein the index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs.

[0177] The cache node is further used to link each of the historical key value information in each of the storage shards in the form of a bidirectional redundant chain according to the order of each of the key value information; for each of the storage shards in each of the cache nodes, based on the label information of the storage shard and the location information of the cache node to which the storage shard belongs, link each of the storage shards in different cache nodes in the form of a bidirectional redundant chain, and construct the bidirectional redundant linked list together with the index information of each of the storage shards.

[0178] The cache node is further configured to obtain the starting key value information in the current storage shard for the current storage shard.

[0179] and the last ending key value information in the last storage slice; obtain the starting key value information in the current storage slice and the longest common prefix of the last ending key value information; determine the starting key value information in the current storage slice based on the longest common prefix and the preset number of label bits.

[0180] Determine the shortest different prefix of the current storage shard, where the shortest different prefix is ​​the label information of the current storage shard; continue to obtain the starting key value information of the next storage shard and the ending key value information of the current storage shard, thereby determining the label information of the next storage shard, until the label information of all the storage shards is obtained, and the label information of each of the storage shards is obtained.

[0181] The cache node is further configured to determine the target storage segment through a preset algorithm based on the tag information of the target storage segment.

[0182] Target storage node information corresponding to the storage shard, where the target storage node information represents location information of the target storage node;

[0183] Based on the target storage node information, first storage location information of the data to be processed is obtained.

[0184] The cache node is further configured to convert the label information of the target storage slice into

[0185] Based on the largest serial number identifier in each of the storage nodes and the target digital label, 5 determines the target storage node information.

[0186] The cache node is further configured to, during the process of storing the current historical key-value information in the current storage shard, determine whether each of the historical key-value information in the current storage shard exceeds a preset quantity range; if the each of the historical key-value information in the current storage shard does not exceed the preset quantity range, store the current historical key-value information in the current storage shard;

[0187] If the historical key value information in the current storage shard exceeds the preset number range, based on the 0 tag information of the current storage shard, the part of the current storage shard that exceeds the preset number range is used as a new storage shard and migrated.

[0188] to the next cache node; and each of the storage slices after the current storage slice is migrated in sequence, so that each of the cache nodes reaches a capacity balance.

[0189] The cache node is further configured to determine, based on the read request, a target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list, where the key-value information is one of the historical key-value information in the target storage shard; or

[0190] The storage location query request is a write request; based on the storage request, determining whether the key value information is one of the historical key value information in each storage shard in the constructed bidirectional redundant linked list;

[0191] If the key value information is each historical key value information in each storage slice in the bidirectional redundant linked list

[0192] One of them, determining the target storage shard corresponding to the key value information;

[0193] 0 If the key value information is not the historical key value information in each storage slice in the bidirectional redundant linked list

[0194] If one of the historical key-value information is obtained, the key-value information is inserted into the corresponding target storage shard in the order of each historical key-value information.

[0195] The storage location query request is a read request, and the cache node is further configured to send the first storage location information to the client.

[0196] 5 The storage node is configured to receive a first data query request sent by the client. The first data query request

[0197] Carrying the key value information of the data to be processed; sending a first data query response to the client, the first data query response carrying the data information corresponding to the key value information, and completing the query processing of the data to be processed by the client.

[0198] The storage location query request is a write request, and the cache node is further configured to send the first storage location information 0 to the client.

[0199] The storage node is further used to receive a first data storage request sent by the client, the first data storage request carries the key value information and the data information of the data to be processed; and send a first data storage response to the client, the first data storage response carries the storage result of the data to be processed, to complete the client's storage processing of the data to be processed.

[0200] 5 Based on the above embodiment, the present application embodiment provides a client 2000, such as Figure 14 As shown, the client 2000

[0201] It includes a sending unit 2001 and a receiving unit 2002, wherein:

[0202] The sending unit is used to send a storage location query request to the server, where the storage location query request carries key value information of the data to be processed; the data to be processed includes at least one key value information.

[0203] The receiving unit is configured to receive first storage location information sent by the server in response to the storage location query request, and perform query or storage processing on the data to be processed based on the first storage location information.

[0204] The storage location query request is a read request. The sending unit is further configured to send a first data query request to the server based on the first storage location information. The first data query request carries the key value information of the data to be processed.

[0205] The receiving unit is further configured to receive a first data query response sent by the server in response to the first data query request, wherein the first data query response carries the data information corresponding to the key value information, thereby completing query processing of the data to be processed.

[0206] The storage location query request is a write request, and the sending unit is further used to send a first data storage request to the server based on the first storage location information, where the first data storage request carries the key value information and the data information of the data to be processed.

[0207] The receiving unit is further configured to receive a first data storage response sent by the server in response to the first data query request, wherein the first data storage response carries the storage result of the data to be processed, thereby completing the storage processing of the data to be processed.

[0208] The client also includes a priority determination unit and a location information determination unit, wherein:

[0209] The priority judgment unit is used to perform priority judgment on the key value information of the data to be processed according to a preset priority rule.

[0210] If the key value information of the data to be processed is a first priority key value, the location information determination unit is used to obtain the second storage location information of the data to be processed through a preset algorithm, and perform query or storage processing on the data to be processed based on the second storage location information.

[0211] If the key value information of the data to be processed is a first priority key value, the sending unit is further used to execute the step of sending a storage location query request to the server; the priority of the first priority key value is higher than the second priority key value.

[0212] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0213] It should be noted that in the embodiments of this application Figure 13 The server shown and Figure 14 The client's module division shown is schematic and represents only one logical functional division. Actual implementations may employ different division methods. Furthermore, the functional units in the various embodiments of this application may be integrated into a single processing unit, physically exist separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in hardware or software functional units. A combination of software and hardware may also be employed.

[0214] It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0215] Based on the method of the above embodiment, an embodiment of the present application provides a data storage device, including a memory and a processor; wherein the memory is used to store executable instructions; the processor is used to execute the executable instructions stored in the memory to implement the data storage method described in the embodiment of the present application.

[0216] Based on the method of the above embodiment, an embodiment of the present application provides a data storage device including a server and a client, wherein:

[0217] like Figure 15 As shown, the server includes a first memory 1102 and a first processor 1101, wherein the first memory 1102 is used to store executable instructions; the first processor 1101 is used to execute the executable instructions stored in the first memory 1102, thereby implementing the data storage method on the server side of the embodiment of the present application.

[0218] like Figure 16 As shown, the client includes a second memory 2102 and a second processor 2101, wherein the second memory 2102 is used to store executable instructions; the second processor 2101 is used to execute the executable instructions stored in the second memory 2102, thereby implementing the data storage method on the client side of the embodiment of the present application.

[0219] An embodiment of the present application further provides a storage medium, which is applied to a server side. The storage medium stores executable instructions, which, when executed, cause the first processor to execute the data storage method on the server side of the embodiment of the present application.

[0220] An embodiment of the present application further provides a storage medium, which is applied to a client. The storage medium stores executable instructions, which, when executed, cause the second processor to execute the data storage method on the client side of the embodiment of the present application.

[0221] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0222] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0223] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0224] The term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, object A and / or object B can mean: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0225] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0227] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed across multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of this embodiment.

[0228] In addition, all functional modules in the embodiments of the present application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the above-mentioned integrated modules can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0229] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0230] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0231] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0232] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0233] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0234] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data storage method, characterized in that: Applied to a server, the server including a management node and at least one cache node, the method includes: Receive a storage location query request sent by a client, wherein the storage location query request carries key value information of data to be processed; the data to be processed includes at least one key value information; Based on the storage location query request, determining a target storage shard corresponding to the key-value information in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list represents that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes label information corresponding to the target storage shard; Determine first storage location information of the data to be processed according to the tag information of the target storage shard, and send the first storage location information to the client for the client to query or store the data to be processed; Wherein, before determining the target storage shard corresponding to the key-value information in the constructed bidirectional redundant linked list based on the storage location query request, the method further includes: Dividing a capacity range for the at least one cache node by the management node according to each historical key value information in the historical data to be processed; Based on the capacity range divided by each cache node, each historical key-value information is stored in each storage shard in each cache node in the order of each historical key-value information in the historical data to be processed; Determining label information of each storage shard according to each of the historical key-value information of the historical data to be processed; The historical key value information in each of the storage shards is linked in the form of a bidirectional redundant chain, and together with the index information of each of the storage shards, a bidirectional redundant linked list is constructed, and the constructed bidirectional redundant linked list is sent to the client, where the index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs.

2. The method according to claim 1, characterized in that The step of linking the historical key value information in each storage shard in the form of a bidirectional redundant chain and constructing the bidirectional redundant linked list with the index information of each storage shard includes: For each of the historical key-value information in each of the storage shards, linking each of the key-value information in each of the storage shards in the form of a bidirectional redundant chain according to the order of each of the key-value information; For each of the storage shards in each of the cache nodes, based on the label information of the storage shard and the location information of the cache node to which the storage shard belongs, the storage shards in different cache nodes are linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list is constructed together with the index information of each of the storage shards.

3. The method according to claim 1, characterized in that The determining the label information of each storage shard according to each historical key value information of the historical data to be processed includes: For the current storage shard, obtain the starting key value information in the current storage shard and the previous ending key value information in the previous storage shard; Obtain the longest common prefix of the starting key value information and the previous ending key value information in the current storage shard; Determine the shortest different prefix of the current storage shard based on the longest common prefix and the preset number of label bits, where the shortest different prefix is ​​the label information of the current storage shard; Continue to obtain the starting key value information of the next storage slice and the ending key value information of the current storage slice, so as to determine the tag information of the next storage slice, until the tag information of all the storage slices is obtained, and obtain the tag information of each storage slice.

4. The method according to claim 1, wherein The server includes at least one storage node; The determining, according to the tag information of the target storage shard, the first storage location information of the to-be-processed data includes: According to the label information of the target storage shard, determine the target storage node information corresponding to the target storage shard by a preset algorithm, wherein the target storage node information represents the location information of the target storage node; Based on the target storage node information, obtaining first storage location information of the data to be processed; The determining, according to the label information of the target storage shard, target storage node information corresponding to the target storage shard by a preset algorithm includes: For the target storage slice, convert the label information of the target storage slice to obtain a corresponding target digital label; The target storage node information is determined based on the maximum sequence number identifier in each of the storage nodes and the target digital label.

5. The method according to claim 1, wherein The method further comprises: In the process of storing the current historical key-value information in the current storage shard, determining whether each of the historical key-value information in the current storage shard exceeds a preset quantity range; If the number of historical key-value information in the current storage shard does not exceed a preset range, the current historical key-value information is stored in the current storage shard; If the number of historical key-value information in the current storage shard exceeds a preset range, the portion of the current storage shard that exceeds the preset range is treated as a new storage shard and migrated to the next cache node based on the tag information of the current storage shard; The storage shards subsequent to the current storage shard are migrated in sequence, so that the capacity of each cache node reaches a balance.

6. The method according to any one of claims 1 to 5, characterized in that The step of determining, based on the storage location query request, a target storage shard corresponding to the key value information in a constructed bidirectional redundant linked list includes: When the storage location query request is a read request, determining, based on the read request, a target storage shard corresponding to the key value information in each corresponding storage shard in the constructed bidirectional redundant linked list, where the key value information is one of each historical key value information in the target storage shard; or, When the storage location query request is a write request, based on the write request, if the key value information is one of the historical key value information in the constructed bidirectional redundant linked list, determining a target storage shard corresponding to the key value information in the bidirectional redundant linked list; In the case that the key value information is not one of the historical key value information in the constructed bidirectional redundant linked list, the target storage shard in which the key value information should be stored is determined in each corresponding storage shard in the bidirectional redundant linked list, and the key value information is inserted into the target storage shard in the order of the historical key value information in the target storage shard.

7. The method according to any one of claims 1 to 5, characterized in that The data to be processed includes at least one piece of data information, and the key value information corresponds to the data information in a one-to-one manner; The sending of the first storage location information to the client, so that the client can query or store the data to be processed, includes: When the storage location query request is a read request, the first storage location information is sent to the client, and a first data query request sent by the client is received through a target storage node corresponding to the first storage location information, where the first data query request carries the key value information of the data to be processed; Sending a first data query response to the client according to the target storage node corresponding to the first storage location information, wherein the first data query response carries the data information corresponding to the key value information, thereby completing the query processing of the to-be-processed data by the client; or, When the storage location query request is a write request, the first storage location information is sent to the client, and a first data storage request sent by the client is received through a target storage node corresponding to the first storage location information, where the first data storage request carries the key value information and the data information of the data to be processed; According to the target storage node corresponding to the first storage location information, the key-value information and the data information corresponding to the key-value information are stored in the target storage node, and a first data storage response is sent to the client. The first data storage response carries the storage result of the data to be processed, thereby completing the client's storage processing of the data to be processed.

8. A data storage method, characterized in that: Applied to a client, the method includes: A storage location query request is sent to the server, the storage location query request carries key value information of the data to be processed, and is used to determine the target storage shard corresponding to the key value information in the constructed bidirectional redundant linked list; the data to be processed includes at least one key value information; the bidirectional redundant linked list is obtained by the management node of the server connecting the various historical key value information in each storage shard in the form of a bidirectional redundant chain, and constructing it with the index information of each storage shard; the index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs; the label information is determined based on the various historical key value information in the historical data to be processed; the various historical key value information is stored in the order of the various historical key value information by the management node based on the capacity range of the various cache nodes of the server; the capacity range is obtained by dividing at least one cache node of the server according to the various historical key value information; Receive first storage location information sent by the server in response to the storage location query request, and query or store the data to be processed based on the first storage location information.

9. The method according to claim 8, characterized in that The querying or storage processing of the to-be-processed data based on the first storage location information includes: The storage location query request is a read request, and based on the first storage location information, a first data query request is sent to the server, where the first data query request carries the key value information of the data to be processed; receiving a first data query response sent by the server in response to the first data query request, wherein the first data query response carries data information corresponding to the key value information, and completing query processing of the data to be processed; or The storage location query request is a write request, and based on the first storage location information, a first data storage request is sent to the server, where the first data storage request carries the key value information and the data information of the data to be processed; A first data storage response is received from the server in response to the first data query request, wherein the first data storage response carries the storage result of the data to be processed, and storage processing of the data to be processed is completed.

10. The method according to claim 9, characterized in that Before sending the storage location query request to the server, the method further includes: According to a preset priority rule, priority determination is performed on the key value information of the data to be processed; If the key value information of the data to be processed is a first priority key value, obtaining second storage location information of the data to be processed through a preset algorithm, and performing query or storage processing on the data to be processed based on the second storage location information; If the key value information of the data to be processed is a second priority key value, the step of sending a storage location query request to the server is executed; the priority of the first priority key value is higher than the second priority key value.

11. A server, characterized in that: include: A management node, at least one cache node, and at least one storage node; wherein, The cache node is configured to receive a storage location query request sent by a client, wherein the storage location query request carries key value information of data to be processed; the data to be processed includes at least one key value information; The cache node is further configured to determine, based on the storage location query request, a target storage shard corresponding to each key-value information in a constructed bidirectional redundant linked list; the constructed bidirectional redundant linked list indicates that each key-value information in each storage shard is linked in the form of a bidirectional redundant chain, and the bidirectional redundant linked list includes label information corresponding to the target storage shard; The cache node is further configured to determine storage location information of the data to be processed based on the tag information of the target storage shard, and send the storage location information to the client for the client to query or store the data to be processed; The management node is used to divide the capacity range of the at least one cache node according to each historical key value information in the historical data to be processed, before determining the target storage shard corresponding to the key value information in the constructed bidirectional redundant linked list based on the storage location query request, and based on the capacity range divided by each cache node, store each historical key value information in each storage shard in each cache node in the order of each historical key value information in the historical data to be processed, and determine the label information of each storage shard based on each historical key value information of the historical data to be processed, and link each historical key value information in each storage shard in the form of a bidirectional redundant chain, and construct the bidirectional redundant linked list with the index information of each storage shard, and send the constructed bidirectional redundant linked list to the client; the index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs.

12. A client, characterized in that: include: A sending unit and a receiving unit; wherein, The sending unit is used to send a storage location query request to the server, wherein the storage location query request carries key value information of the data to be processed and is used to determine the target storage shard corresponding to the key value information in the constructed bidirectional redundant linked list; the data to be processed includes at least one key value information; the bidirectional redundant linked list is obtained by the management node of the server connecting the various historical key value information in each storage shard in the form of a bidirectional redundant chain, and constructing it with the index information of each storage shard; the index information includes: the label information of the storage shard and the location information of the cache node to which the storage shard belongs; the label information is determined based on the various historical key value information in the historical data to be processed; the various historical key value information is stored in each storage shard in the order of the various historical key value information by the management node based on the capacity range of each cache node of the server; the capacity range is obtained by dividing at least one cache node of the server according to each historical key value information; The receiving unit is configured to receive the storage location information sent by the server in response to the storage location query request, and perform query or storage processing on the data to be processed based on the storage location information.

13. A data storage device, characterized in that include: a memory for storing executable instructions; The processor is configured to, when executing the executable instructions stored in the memory, implement the data storage method according to any one of claims 1 to 7; or implement the data storage method according to any one of claims 8 to 10.

Images