Data storage method and device, storage medium and electronic equipment
By configuring proxy nodes and Raft groups in distributed clusters, the problems of limited storage capacity and poor redundant disaster capabilities are solved, storage capacity expansion and data consistency are achieved, and the system's redundant disaster capabilities are improved.
Patent Information
- Application Number
- CN202510342371.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-07-11
AI Technical Summary
In the prior art, stand-alone storage relies on a stand-alone computer, resulting in limited storage capacity and poor redundant disaster capability.
By configuring proxy nodes with data consistency functions in a distributed cluster, using Raft groups to determine target shards and target proxy nodes, data storage to target storage nodes, expand storage capacity and improve redundant disaster capabilities.
It expands storage capacity, improves the system's redundant disaster capability, ensures the consistency of data within the shard, and is suitable for a variety of storage scenarios.
Smart Images
Figure CN120295564A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing, and in particular, to a data storage method, apparatus, storage medium, and electronic device. Background Art
[0002] Data storage refers to saving data in physical or logical storage media for future use. Data storage can include various forms such as file systems, databases, data warehouses, and data lakes.
[0003] Currently, related technologies usually adopt single-machine storage, that is, a storage method in which data is stored on a single physical server or a single virtual machine.
[0004] However, in related technologies, single-machine storage usually relies on an independent computer to provide data storage services. Limited by single-machine hardware resources, the storage capacity is limited and the disaster recovery ability is poor. Summary of the Invention
[0005] In view of this, this application provides a data storage method, apparatus, storage medium, and electronic device, mainly aiming to improve the technical problems in related technologies at present, where single-machine storage usually relies on an independent computer to provide data storage services, and limited by single-machine hardware resources, the storage capacity is limited and the disaster recovery ability is poor.
[0006] In a first aspect, this application provides a data storage method, including:
[0007] Obtain a shard composed of storage units in a distributed cluster, where the storage unit includes a storage node and a proxy node corresponding to the storage node with a data consistency function;
[0008] Determine a target shard corresponding to a data storage request according to the proxy node information corresponding to the proxy node and a routing rule;
[0009] Use a Raft group composed of proxy nodes in the target shard to determine a target proxy node in the target shard;
[0010] Store the data corresponding to the data storage request into the storage node in the target shard through the target proxy node.
[0011] In a second aspect, this application provides a data storage apparatus, including:
[0012] An acquisition module, configured to obtain a shard composed of storage units in a distributed cluster, where the storage unit includes a storage node and a proxy node corresponding to the storage node with a data consistency function;
[0013] A determination module, configured to determine a target shard corresponding to a data storage request according to the proxy node information corresponding to the proxy node and a routing rule;
[0014] A determination module, configured to determine a target proxy node in the target shard by using a Raft group composed of proxy nodes in the target shard;
[0015] A storage module, configured to store the data corresponding to the data storage request into a storage node in the target shard through the target proxy node.
[0016] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data storage method described in the first aspect is implemented.
[0017] In a fourth aspect, the present application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the computer program, the data storage method described in the first aspect is implemented.
[0018] In a fifth aspect, the present application provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the data storage method described in the first aspect is implemented.
[0019] By means of the above technical solution, a data storage method, device, storage medium, and electronic device provided by the present application first obtain shards composed of storage units in a distributed cluster, where the storage units include storage nodes and proxy nodes corresponding to the storage nodes and having a data consistency function; then determine a target shard corresponding to a data storage request according to the proxy node information corresponding to the proxy node and a routing rule; then determine a target proxy node in the target shard by using a Raft group composed of proxy nodes in the target shard; and finally store the data corresponding to the data storage request into the storage nodes in the target shard through the target proxy node. Compared with the current existing technologies, the present application configures proxy nodes with a data consistency function for storage nodes, uses the proxy nodes and stand-alone storage nodes to form storage units, and then uses multiple storage units to form shards in a distributed cluster, expanding the storage capacity. Then, based on the proxy node information corresponding to the proxy node and a routing rule, a target shard for processing a data storage request is determined, and a target proxy node in the target shard is determined according to the Raft group composed of proxy nodes. Finally, the data corresponding to the data storage request is stored into each storage node in the target shard by using the target proxy node, so that the data stored in multiple storage units in the target shard is consistent, improving the disaster tolerance ability of the storage units.
[0020] The above description is only an overview of the technical solution of the present application. In order to be able to more clearly understand the technical means of the present application, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically exemplified below. Brief Description of the Drawings
[0021] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0023] Figure 1 It shows a schematic flow chart of a data storage method provided by an embodiment of the present application;
[0024] Figure 2 It shows a schematic flow chart of an example provided by an embodiment of the present application;
[0025] Figure 3 It shows a schematic structural diagram of a data storage device provided by an embodiment of the present application. Detailed Description of the Embodiments
[0026] The embodiments of the present application will be described in more detail below with reference to the drawings. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.
[0027] In order to improve the technical problems in the current related technologies that single-machine storage usually relies on an independent computer to provide data storage services, limited by the single-machine hardware resources, resulting in limited storage capacity and poor disaster tolerance. This embodiment provides a data storage method, as Figure 1 shown, the method includes:
[0028] Step 101, obtain shards composed of storage units in a distributed cluster, where the storage units include storage nodes and proxy nodes corresponding to the storage nodes with data consistency functions.
[0029] In some embodiments, the storage node can be a stand-alone storage node, responsible for providing storage space and read / write interfaces for upper-layer applications or services, and providing read / write capabilities. The proxy node can be located between the client and the backend storage node, acting as a traffic entry point, responsible for receiving application requests from the outside and reasonably distributing each request to different storage nodes for processing. It can dynamically adjust the task allocation strategy according to the real-time load situation to optimize resource utilization. Exemplarily, one proxy node can correspond to one storage node, and the corresponding storage node can be reconfigured according to the load conditions of each storage node.
[0030] Specifically, the proxy node can configure the data consistency function. For example, by implementing consistency protocols such as Paxos and Raft to reach a consensus, it can ensure data synchronization and consistency maintenance among all storage nodes within the shard. Exemplarily, taking Raft as an example, the proxy node can be used to implement functions such as Raft, request routing, and node detection.
[0031] Correspondingly, in order to ensure the high availability and fault tolerance of the distributed system, corresponding proxy nodes can be configured for each of the multiple storage nodes. A storage node and its corresponding proxy node form a storage unit, and then multiple storage units form a shard. Multiple shards jointly form a distributed cluster. Each shard can work independently, and the data between shards is independent. In this way, multiple shards are extended based on the storage unit, and the data can be dispersed on multiple shards, improving the problem of capacity limitation of the stand-alone storage node.
[0032] Step 102: Determine the target shard corresponding to the data storage request according to the proxy node information and routing rules corresponding to the proxy node.
[0033] In some embodiments, each proxy node within each shard can maintain all the proxy node information. The proxy node information can include a hash identifier (hashID), a node address, and the storage node information currently corresponding to each proxy node.
[0034] In some embodiments, the routing rules corresponding to each proxy node can be used to ensure that requests are correctly sent to the appropriate nodes, thereby improving the performance, availability, and scalability of the system. Exemplarily, each data storage request first passes through the proxy node and is randomly assigned to a certain proxy node. In order to make the data distribution more uniform and easy to migrate, the proxy node may adopt the consistent hashing algorithm to determine which storage node should receive a specific key-value pair, minimizing the data movement range when adding or deleting nodes, thereby improving the flexibility and efficiency of the system. In this way, each data storage request can be assigned to the corresponding target shard for processing, reducing the capacity limitation problem caused by a single storage node.
[0035] Step 103: Determine the target proxy node in the target shard by using the Raft group composed of proxy nodes in the target shard.
[0036] In some embodiments, multiple proxy nodes in the target shard can form a Raft group. Multiple storage units can implement the leader election and log replication functions of Raft through the Raft group composed of proxy nodes, determine the leader node and follower nodes, so that the data stored in multiple storage units is consistent, the data within each shard is completely consistent, and the backup function of the cluster can be provided, improving the disaster tolerance ability of the system.
[0037] Step 104: Store the data corresponding to the data storage request in the target storage node corresponding to the target proxy node.
[0038] In some embodiments, after determining the target shard for processing the data storage request, the data storage request can be first forwarded to the target storage node in the target shard through the target proxy node in the target shard, requesting the target storage node to store the data corresponding to the data storage request, and then the data storage request is sent to other proxy nodes in the target shard through the target proxy node to store the data in other storage nodes, thereby realizing data storage through sharding, expanding the storage capacity, ensuring the data consistency within each shard, improving the disaster tolerance ability of the storage unit, and being applicable to various storage scenarios.
[0039] Compared with the current existing technologies, in this embodiment, proxy nodes with data consistency functions are configured for the storage nodes, the proxy nodes and single-machine storage nodes are used to form storage units, and then multiple storage units are used to form shards in the distributed cluster to expand the storage capacity. Then, based on the proxy node information and routing rules corresponding to the proxy nodes, the target shard for processing the data storage request is determined, and the target proxy node in the target shard is determined according to the Raft group composed of proxy nodes. Finally, the data corresponding to the data storage request is stored in each storage node in the target shard by using the target proxy node, so that the data stored in multiple storage units in the target shard is consistent, improving the disaster tolerance ability of the storage unit.
[0040] Further optionally, step 102 may specifically include: determining the hash identifier corresponding to the data storage request according to the request identifier corresponding to the data storage request or the global identifier generated by the proxy node; comparing the hash identifier with the hash identifier in the proxy node information based on the routing rule to determine the shard identifier of the target shard.
[0041] In some embodiments, the data storage request may include a write request. If the write request contains a request identifier, such as a unique key or a unique ID, the request identifier can be used to perform a hash calculation to obtain a hash identifier. If the data storage request does not contain a request identifier, the proxy node can generate a global ID, and then use this information to perform a hash calculation to obtain a hash ID. Then, according to the hash ID and the routing rule, the corresponding shard ID can be found to determine the target shard.
[0042] Optionally, step 103 may specifically include: using the Raft leader election ability of the Raft group to determine the leader node of the proxy nodes in the target shard as the target proxy node.
[0043] In some embodiments, within the target shard, multiple proxy nodes can form one Raft group. The Raft leader election ability of the Raft group can be used to provide a unique leader node for the current target shard as the target proxy node, avoiding the problem that the cluster cannot be written or the writing is disordered due to the generation of 0 or 2 or more leader nodes.
[0044] Optionally, using the Raft leader election ability of the Raft group to determine the leader node of the proxy nodes in the target shard as the target proxy node may specifically include: using the proxy nodes within the Raft group to periodically detect other proxy nodes within the Raft group to obtain the detection result of the Raft group; according to the detection result of the Raft group, determine the leader node of the target shard as the target proxy node.
[0045] Optionally, each proxy node within each shard can periodically detect other proxy nodes within the shard to detect the load conditions of other proxy nodes and their corresponding storage nodes, as well as abnormal conditions such as storage failures. The leader node of the Raft group can be determined or updated according to the detection results of the Raft groups within each shard. Correspondingly, if an abnormal node is detected, the abnormal node can be removed for automatic repair to ensure the stability of the system.
[0046] Optionally, step 104 may specifically include: sending the data storage request to the target storage node through the target proxy node corresponding to the target storage node; using the target storage node to store the data corresponding to the data storage request.
[0047] In some embodiments, a proxy node of a certain shard in a distributed cluster may first receive a write request. After determining the target shard, it may check whether the shard currently receiving the data storage request is the target shard. If the current shard is the target shard and the current node is the target proxy node, the request (along with the global ID if necessary) may be forwarded to the storage node corresponding to the target proxy node for data storage. If the current shard is not the target shard, the request may be forwarded to the target proxy node corresponding to the target shard, and the request may be sent to its corresponding single-machine storage node through the target proxy node, thus completing the data writing of the master node.
[0048] Optionally, the method of this embodiment may specifically further include: sending data storage requests to other proxy nodes in the Raft group except the target proxy node through the target proxy node, so as to store the data corresponding to the data storage requests to the storage nodes corresponding to other proxy nodes.
[0049] In some embodiments, for the replication of write requests, the log replication function provided by Raft may be used to ensure the consistency of the data of all nodes in the current shard. Specifically, the target storage node may synchronize the original request content to the proxy nodes of all slave nodes under Raft through Raft's log synchronization, that is, other proxy nodes in the Raft group except the target proxy node. After receiving the request data, other proxy nodes may parse and restore the original write command. Correspondingly, after the write command is parsed by the proxy nodes, other proxy nodes may respectively request their corresponding storage nodes, so that the storage nodes corresponding to other proxy nodes complete data storage after executing the write command. In this way, the original command is written to the slave nodes, and the consistency of the master-slave node data can be ensured through such a replication method, realizing the data consistency of each storage unit in each shard.
[0050] Optionally, the method of this embodiment may specifically further include: determining the target shard corresponding to the data read request according to the proxy node information and the routing rule; randomly selecting a target proxy node from the target shard corresponding to the data read request; and using the storage node corresponding to the target proxy node to return the data corresponding to the data read request.
[0051] In some embodiments, for a data read request (read request), if the request contains a unique key or a unique ID, the same routing rule as that for a write request can be used to find the corresponding target shard. If the shard where the current proxy node receiving the read request is located is the target shard, the request can be forwarded to the corresponding storage node through the current proxy node. The storage node returns the data read according to the read request, and if parsing and assembly are required, it is completed by the current proxy node and finally returned to the requester. If the shard where the current proxy node receiving the read request is located is not the target shard, a target proxy node is randomly determined from the target shards, and the randomly selected target proxy node requests the corresponding storage node to return the data corresponding to the read request, improving the efficiency of data reading.
[0052] Optionally, the method of this embodiment may specifically further include: obtaining a snapshot and incremental logs of a target storage unit in a Raft group at a target time; creating a latest storage unit according to the snapshot and incremental logs, and adding the proxy nodes in the latest storage unit as slave nodes to the Raft group.
[0053] In some embodiments, the snapshot function of the proxy node Raft and the snapshot function of the storage node can be used to record a snapshot of a certain storage unit based on a certain time point, obtain a snapshot of the target storage unit in the Raft group at the target time, and back up the snapshot to a newly created storage unit. Correspondingly, the proxy nodes in the newly created storage unit can be used as slave nodes to join the Raft group corresponding to the current shard, and then the incremental log replication function of Raft is used for full synchronization. After all incremental logs are synchronized, the node can be connected to the real traffic as a normal node, improving the flexibility of node access and ensuring data consistency among nodes within the shard.
[0054] As a possible implementation, as Figure 2 shown, corresponding proxy nodes can be created for each storage node, and a storage unit is formed by combining a storage node and a proxy node. Multiple storage units contribute to shards in a distributed cluster, such as shard 1, shard 2, shard 3, etc. in the figure. The proxy nodes in a shard can form 1 Raft group to ensure data consistency among storage units within the shard, and the data between shards is independent.
[0055] In this way, in this embodiment, a proxy node and a single-machine storage node are used to form a storage unit, and multiple storage units are combined into a Raft group. The Raft replication function is used to ensure the data consistency of multiple storage units. At the same time, in view of the capacity limitation of a single machine, a solution of establishing a distributed cluster on top of sharding is proposed. The data is scattered on multiple shards to form a complete data storage solution. Multiple shards are expanded, and each shard is an independent Raft group. Each Raft group works independently, enabling horizontal splitting of data. In this way, the number of reasonable Raft groups can be set according to the cluster scale, thereby avoiding the problem of the capacity limitation of a single machine.
[0056] In addition, without modifying the code of the single-machine storage, only the proxy node needs to be added. The proxy node can be implemented in a more familiar language. The two are completely decoupled, which is suitable for scenarios where the code of the storage unit is relatively complex and the modification cost is very high. It shields the modification of the underlying storage unit, is simple and easy to implement, and saves the implementation cost of data synchronization.
[0057] Compared with the current existing technologies, in this embodiment, the master node of the target shard can be determined according to the detection result of the Raft group as the target proxy node. The data corresponding to the data storage request is stored in the target storage node through the target proxy node, and then based on the Raft log, the data storage request is synchronized to other proxy nodes, ensuring the data consistency of multiple storage units, improving the disaster tolerance ability of the system. Also, the target shard corresponding to the data read request can be determined according to the proxy node information and routing rules, and the target proxy node is randomly selected from the target shards corresponding to the data read request to process the data read request, improving the data read efficiency. And a new storage unit is created according to the snapshot and incremental log of the storage unit, improving the flexibility of node access and ensuring the data consistency of the nodes within the shard.
[0058] Further, as Figure 1 a specific implementation of the method shown, this embodiment provides a data storage device, as Figure 3 shown. The device includes: an acquisition module 31, a determination module 32, and a storage module 33.
[0059] The acquisition module 31 is configured to acquire the shards formed by the storage units in the distributed cluster. The storage unit includes a storage node and a proxy node corresponding to the storage node with a data consistency function.
[0060] The determination module 32 is configured to determine the target shard corresponding to the data storage request according to the proxy node information and routing rules corresponding to the shard.
[0061] The determination module 32 is configured to use the Raft group composed of the proxy nodes in the target shard to determine the target proxy node in the target shard.
[0062] A storage module 33, configured to store data corresponding to a data storage request in a target storage node corresponding to a target proxy node.
[0063] In some examples of this embodiment, the determination module 32 is specifically configured to use the Raft leader election ability of the Raft group to determine the leader node of the proxy nodes in the target shard as the target proxy node.
[0064] In some examples of this embodiment, the determination module 32 is specifically configured to periodically detect other proxy nodes in the Raft group by using the proxy nodes in the Raft group to obtain the detection result of the Raft group; determine the leader node of the target shard as the target proxy node according to the detection result of the Raft group.
[0065] In some examples of this embodiment, the storage module 33 is specifically configured to send a data storage request to the target storage node through the target proxy node corresponding to the target storage node; use the target storage node to store the data corresponding to the data storage request.
[0066] In some examples of this embodiment, the storage module 33 is specifically further configured to send a data storage request to other proxy nodes in the Raft group except the target proxy node through the target proxy node, so as to store the data corresponding to the data storage request in the storage nodes corresponding to the other proxy nodes.
[0067] In some examples of this embodiment, the determination module 32 is specifically configured to determine the hash identifier corresponding to the data storage request according to the request identifier corresponding to the data storage request or the global identifier generated by the proxy node; based on the routing rule, compare the hash identifier with the hash identifier in the proxy node information to determine the shard identifier of the target shard.
[0068] In some examples of this embodiment, the determination module 32 is specifically configured to determine the target shard corresponding to the data read request according to the proxy node information and the routing rule; randomly select a target proxy node from the target shards corresponding to the data read request; use the storage node corresponding to the target proxy node to return the data corresponding to the data read request.
[0069] In some examples of this embodiment, the acquisition module 31 is specifically further configured to acquire the snapshot and incremental log of the target storage unit in the Raft group at the target moment; create the latest storage unit according to the snapshot and incremental log and join the Raft group as a follower node.
[0070] It should be noted that for other corresponding descriptions of each functional unit involved in a data storage device provided in this embodiment, reference can be made to the corresponding description in Figure 1 and will not be elaborated here.
[0071] Based on the method as described above Figure 1 shown, correspondingly, this embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method as described above Figure 1 shown.
[0072] Based on the method as described above Figure 1 shown, correspondingly, this embodiment also provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method as described above Figure 1 shown.
[0073] Based on such an understanding, the technical solution of this application can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method of the implementation scenario of this application.
[0074] Based on the method as described above Figure 1 shown, and Figure 3 the virtual device embodiment shown, in order to achieve the above object, this embodiment of the application also provides an electronic device, and the device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the method as described above Figure 1 shown.
[0075] Optionally, the above-mentioned physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and optionally the user interface may further include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.
[0076] Those skilled in the art can understand that the above-mentioned physical device structure provided by this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine some components, or have different component arrangements.
[0077] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware and software resources of the above-mentioned physical device, and supports the operation of an information processing program and other software and / or programs. The network communication module is used to implement communication between the components inside the storage medium, and communication between other hardware and software in the information processing physical device.
[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or can also be implemented by hardware. By applying the technical solution of this embodiment, this embodiment can determine the master node of the target shard as the target proxy node according to the detection result of the Raft group, store the data corresponding to the data storage request into the target storage node through the target proxy node, and then synchronize the data storage request to other proxy nodes based on the Raft log, ensuring the data consistency of multiple storage units, improving the redundancy and disaster tolerance capabilities of the system. It can also determine the target shard corresponding to the data read request according to the proxy node information and the routing rule, randomly select a target proxy node from the target shards corresponding to the data read request to process the data read request, improving the efficiency of data reading, and create a new storage unit according to the snapshot and incremental log of the storage unit, improving the flexibility of node access and ensuring the data consistency of nodes within the shard.
[0079] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.
[0080] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments described herein, but will conform to the broadest scope consistent with the principles and novel features claimed herein.
Claims
1. A data storage method, characterized in that, Including: Obtain shards composed of storage units in a distributed cluster, where the storage units include storage nodes and proxy nodes corresponding to the storage nodes with data consistency functions; Determine a target shard corresponding to a data storage request according to the proxy node information corresponding to the proxy node and a routing rule; Use a Raft group composed of proxy nodes in the target shard to determine a target proxy node in the target shard; Store the data corresponding to the data storage request into the storage node in the target shard through the target proxy node.
2. The method according to claim 1, wherein The using a Raft group composed of proxy nodes in the target shard to determine a target proxy node in the target shard includes: Use the Raft leader election ability of the Raft group to determine the leader node of the proxy nodes in the target shard as the target proxy node.
3. The method according to claim 2, wherein The using the Raft leader election ability of the Raft group to determine the leader node of the proxy nodes in the target shard as the target proxy node includes: Use the proxy nodes in the Raft group to periodically detect other proxy nodes in the Raft group to obtain the detection result of the Raft group; Determine the leader node of the target shard as the target proxy node according to the detection result of the Raft group.
4. The method according to claim 1, characterized in that The storing the data corresponding to the data storage request into the storage node in the target shard through the target proxy node includes: Send the data storage request to the target storage node through the target proxy node corresponding to the target storage node; Use the target storage node to store the data corresponding to the data storage request.
5. The method according to claim 1, characterized in that, The storing the data corresponding to the data storage request into the storage node in the target shard through the target proxy node further includes: Send the data storage request to other proxy nodes in the Raft group except the target proxy node through the target proxy node to store the data corresponding to the data storage request into the storage nodes corresponding to the other proxy nodes.
6. The method according to claim 1, wherein The determining a target shard corresponding to a data storage request according to the proxy node information corresponding to the proxy node and a routing rule includes: Determine a hash identifier corresponding to the data storage request according to the request identifier corresponding to the data storage request or the global identifier generated by the proxy node; Based on the routing rule, compare the hash identifier with the hash identifiers in the proxy node information to determine the shard identifier of the target shard.
7. The method according to claim 1, characterized in that The method further includes: Determine a target shard corresponding to a data read request according to the proxy node information and the routing rule; Randomly select a target proxy node from the target shard corresponding to the data read request; Use the storage node corresponding to the target proxy node to return the data corresponding to the data read request.
8. The method according to claim 1, wherein The method further includes: Obtain a snapshot and incremental logs of a target storage unit in the Raft group at a target time; Create a latest storage unit according to the snapshot and incremental logs, and use the proxy nodes in the latest storage unit as follower nodes to join the Raft group.
9. A data storage device, characterized in that, Including: An acquisition module, configured to acquire shards composed of storage units in a distributed cluster, where the storage units include storage nodes and corresponding proxy nodes with data consistency functions for the storage nodes; A determination module, configured to determine a target shard corresponding to a data storage request according to proxy node information corresponding to the proxy nodes and routing rules; A determination module, configured to determine a target proxy node in the target shard by using a Raft group composed of proxy nodes in the target shard; A storage module, configured to store data corresponding to the data storage request into a storage node in the target shard through the target proxy node.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 8.
11. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.
12. A computer program product having a computer program stored thereon, characterized in that, When the computer program product is executed by a processor, it implements the method according to any one of claims 1 to 8.