Data write request processing method, device and distributed data storage system

Abstract

The invention discloses a data writing request processing method, device and distributed data storage system. Wherein, the method writes data to the storage node based on the data write request sent by the caller, wherein the data write request includes a main request and at least one sub-request, and the method includes: for a sub-request, calling the access interface to select the data to be written from the global variable The storage node address of the storage node that enters the data, and sends a sub-request to the storage node, wherein the selected storage node address is saved in the context of the sub-request; the selected storage node address is extracted from the context of the sub-request, and saved to the storage node In the context of the node; extract the address of the storage node from the context of the storage node, establish a connection with the storage node, and write the data to the corresponding storage node, which can meet the customization requirements of the caller and write data to the specified storage node, realizing Customized load balancing solves the single-point limitation of KV storage.

Description

technical field [0001] The invention relates to the technical field of computer networks, in particular to a data write request processing method, device and distributed data storage system. Background technique [0002] In a distributed data storage system, a distributed component is generally used to forward the data write request sent by the caller to the storage engine, wherein the distributed component is a reverse proxy server, specifically, an Nginx server. For KV storage, Nginx's built-in load balancing strategy, whether it is round-robin, ip_hash, or url_hash, cannot meet the demand for load balancing based on key. For example, the distributed component uses 1Primary- Take the storage mode of 1Replica Shard as an example, which inevitably involves two operations of writing storage, and these two writing operations need to be two completely different nodes, but the built-in load balancing strategy of Nginx cannot meet such highly customized requirements , the fundam...

AI Technical Summary

Problems solved by technology

For KV storage, Nginx's built-in load balancing strategy, whether it is round-robin, ip_hash, or url_hash, cannot meet the demand for load balancing based on key. For example, the distributed component uses 1Primary- Take the storage mode of 1Replica Shard as an example, which inevitably involves two operations of writing storage, and these two writing operations need to be two completely different nodes, but the built-in load balancing strategy of Nginx cannot meet such highly customized requirements , the fundamental reason is that Nginx's load balancing strategy completely shields details such as the address of the storage node from the outer layer, and it is impossible to obtain the address of a storage node through any built-in function when writing the http module, that is to say, these details It is transparent to developers, thus limiting the implementation of customized load balancing strategies

Images