Distributed storage system data processing method and device

Abstract

The embodiment of the invention provides a data processing method and device for a distributed storage system, the distributed storage system comprises a plurality of nodes, and the method comprises the following steps: when the consistency mode of the distributed storage system is a global causal consistency mode, processing data according to local operation serial numbers locally generated by the nodes; and when the consistency mode of the distributed storage system is an on-demand local linear consistency mode, if the node is a node of a non-specified service domain, processing data according to a local operation sequence number locally generated by the node, and if the node is a node of a specified service domain, processing data according to a local operation sequence number locally generated by the node. And if yes, processing the data according to the local operation serial number generated by the specified node. According to the distributed storage system provided by the embodiment of the invention, multiple consistency modes are designed, different consistency modes can be selected according to different storage requirements, the global resource overhead of the distributed storage system can be reduced, and the performance of the distributed storage system is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer storage, and in particular to a distributed storage system data processing method, a distributed storage system data processing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of information technology, the amount of data has increased exponentially and will continue to grow in the future. Therefore, traditional storage systems can no longer meet the needs of modern applications. In order to cope with the needs of storing and managing massive data, distributed storage systems have emerged.

[0003] Distributed storage systems store data in multiple independent devices (nodes) to share the storage load and use location servers to locate storage information. This not only improves the system's reliability, availability, and access efficiency, but also makes it easy to expand.

[0004] Distributed storage systems should provide users with necessary consistency guarantees, ensure the correctness of user data, and provide observability of system operation timing. Therefore, distributed storage systems design consistency models to achieve consistency. However, the current consistency models of distributed storage systems cannot meet different storage requirements. Summary of the invention

[0005] In view of the above problems, a distributed storage system data processing method and device are proposed to overcome the above problems or at least partially solve the above problems. The specific technical solution is as follows:

[0006] In a first aspect of the present invention, a distributed storage system data processing method is provided, wherein the distributed storage system includes a plurality of nodes, and the method includes:

[0007] When the consistency mode of the distributed storage system is a global causal consistency mode, processing data according to a local operation sequence number generated locally by the node;

[0008] When the consistency mode of the distributed storage system is the on-demand local linear consistency mode, if the node is a node of a non-specified business domain, the data is processed according to the local operation sequence number generated locally by the node; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

[0009] In one embodiment of the present invention, the nodes each maintain a local operation sequence number; and processing data according to the local operation sequence number generated locally by the node includes:

[0010] Generate a first local operation sequence number according to the local operation sequence number;

[0011] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0012] The first write request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain a second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0013] receiving a response message sent by the first target node and returned in response to the first write request; the response message includes a second local operation sequence number;

[0014] The local operation sequence number is updated according to the second local operation sequence number.

[0015] In one embodiment of the present invention, processing data according to a local operation sequence number generated locally by the node includes:

[0016] Generate a third local operation sequence number according to the local operation sequence number;

[0017] Generate a first read request; the first read request includes the third local operation sequence number and a first write data item corresponding to the first local operation sequence number;

[0018] The first read request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain a fourth local operation sequence number, and obtain a first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item;

[0019] receiving a response message sent by the first target node and returned in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value;

[0020] The local operation sequence number is updated according to the fourth local operation sequence number.

[0021] In one embodiment of the present invention, the node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; and processing data according to the local operation sequence number generated by the designated node includes:

[0022] Generate a second write request; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value;

[0023] Sending the second write request to a second target node corresponding to the designated business domain; the second target node is used to obtain a first local operation sequence number generated according to the local operation sequence number from the designated node, and storing the second write data and the unique identifier according to the first local operation sequence number;

[0024] Receive a response message returned by the second target node in response to the second write request.

[0025] In one embodiment of the present invention, the node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; and processing data according to the local operation sequence number generated by the designated node includes:

[0026] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0027] Sending the second read request to a second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0028] Receive a response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0029] In an embodiment of the present invention, the nodes of the designated service domain are determined by using a distributed hash mechanism.

[0030] In one embodiment of the present invention, the local operation sequence number and the partial operation sequence number are monotonically increasing.

[0031] The embodiment of the present invention further discloses a distributed storage system data processing device, wherein the distributed storage system includes a plurality of nodes, and the device includes:

[0032] A first data processing module, configured to process data according to a local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is a global causal consistency mode;

[0033] The second data processing module is used to process data according to the local operation sequence number generated locally by the node if the node is a node of a non-specified business domain when the consistency mode of the distributed storage system is an on-demand local linear consistency mode; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

[0034] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned distributed storage system data processing methods.

[0035] In another aspect of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned distributed storage system data processing methods.

[0036] Compared with the related art, the embodiments of the present invention have at least the following advantages:

[0037] In an embodiment of the present invention, a distributed storage system may include multiple consistency modes. When the consistency mode of the distributed storage system is a global causal consistency mode, data is processed according to a local operation sequence number generated locally at the node. When the consistency mode of the distributed storage system is an on-demand local linear consistency mode, if the node is a node of a non-specified business domain, data may be processed according to the local operation sequence number generated locally. If the node is a node of a specified business domain, data may be processed according to a local operation sequence number generated by the specified node. The distributed storage system of an embodiment of the present invention is designed with multiple consistency modes, and different consistency modes may be selected according to different storage requirements, which may reduce the global resource overhead of the distributed storage system and improve the performance of the distributed storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0039] Figure 1 A schematic diagram of a global view of an operation event of a distributed storage system provided in an embodiment of the present invention;

[0040] Figure 2 A flowchart of a distributed storage system data processing method provided in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the relationship between two modes in a distributed storage system provided in an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of implementing a global causal consistency mode provided in an embodiment of the present invention;

[0043] Figure 5 A structural block diagram of a distributed storage system data processing device provided in an embodiment of the present invention;

[0044] Figure 6 The present invention is a block diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0046] In specific implementations, a reliable storage system needs to provide users with data consistency guarantees. Consistency guarantees mean that users can read the correct data value after successfully writing data into the storage system, and the data is correct and consistent no matter when it is read later.

[0047] In order to achieve a higher degree of reliability, the storage system also needs to ensure that the operation events (events) of the storage system observed by all users are the same, so the storage system needs to implement a consistency model. For example: after the user successfully writes A=1, B=2, A=3, B=4 in the storage system, the user will read the latest A=3, B=4; at the same time, during the entire writing process and after the writing is completed, each user will observe that the storage system processes the write sequence of A=1, B=2, A=3, B=4 in sequence. The consistency model can include multiple consistency models, such as linear consistency model, sequential consistency model, eventual consistency model and causal consistency model. Among them, the consistency model in the above example is a linear consistency model. The linear consistency model is the highest level consistency model that can be achieved by engineering implementation. The linear consistency model ensures that the data read by the user at any time is accurate and valid, and it is also the basis for the storage system to expand multi-version support, backup / recovery, snapshot / rollback features.

[0048] In the early days, storage systems were single-copy storage systems. A single-copy storage system means that the data in the storage system is stored in only one copy without redundant backup. The storage system can use the local physical clock or self-increasing sequence number to sort data operations, which can easily provide linear consistency guarantees.

[0049] At present, most storage systems are distributed storage systems. Since each node of a distributed storage system may be deployed on a different physical host, in order to ensure the linear consistency requirement, it is necessary to consider sorting all operation events in the entire distributed storage system. Figure 1, is a schematic diagram of a global view of an operation event of a distributed storage system provided in an embodiment of the present invention. Assume that the distributed storage system includes a replica service node P1 and a replica service node P2, and the replica service node P1 executes write:x=4, read:y=2, and the replica service node P2 executes read:x=0, write:y=2, and read:x=4. Then, according to the timing of the operation events, the global view recorded for the distributed storage system is: read:x=0→write:x=4→write:y=2→read:y=2→read:x=4.

[0050] Due to the cross-node nature of distributed storage systems, there is no longer a unique physical clock to determine the timing of operation events like in single-copy storage systems. If you still rely on physical clock sorting, consider that the single request processing time of a high-concurrency system is measured in microseconds, and the high-precision physical clock mechanism with microsecond accuracy is expensive.

[0051] Therefore, in order to achieve linear consistency, distributed storage systems often need to use complex logical clocks or hybrid clock algorithms to synchronize timing. The algorithm implementation is relatively complex and difficult to implement, such as Amazon's Dynamo; or set up a centralized timing coordination service in the system, which can easily become a system performance bottleneck, such as TiDB using TSO in PD. As a result, distributed storage systems have to pay a high price to accurately and efficiently achieve linear consistency.

[0052] Since the cost of implementing linear consistency in distributed storage systems is so high, some storage systems simply abandon the design of linear consistency models and instead pursue eventual consistency models, such as Netflix's Eureka and Alibaba's Nacos. However, these storage systems are distributed registration centers, which have higher requirements for availability and lower requirements for data real-time performance. In addition, eventual consistency is a weak consistency model and is difficult to apply to distributed storage systems.

[0053] In response to the above problems, an embodiment of the present invention provides a method for data processing in a distributed storage system, which selects a global causal consistency model to provide users with effective data consistency guarantees. Furthermore, the business domain scope can be delineated when the business needs it. In the distributed storage system, an on-demand local linear consistency model can be selected for the specified business domain to provide more effective data consistency guarantees for users of the specified business domain. In this way, the overall design and implementation difficulty of the consistency model of the distributed storage system in the embodiment of the present invention is much lower than that of the global linear consistency model.

[0054] Reference Figure 2 , is a flow chart of the steps of a distributed storage system data processing method provided in an embodiment of the present invention, such as Figure 1As shown, the distributed storage system includes multiple nodes, and the method may specifically include the following steps:

[0055] Step 201: When the consistency model of the distributed storage system is a global causal consistency model, data is processed according to a local operation sequence number generated locally by the node;

[0056] Step 202: When the consistency mode of the distributed storage system is the on-demand local linear consistency mode, if the node is a node of a non-specified business domain, data is processed according to a local operation sequence number generated locally by the node; if the node is a node of a specified business domain, data is processed according to a local operation sequence number generated by the specified node.

[0057] In a specific implementation, the distributed storage system includes multiple nodes, such as a client and a replica service node. Of course, it may also include other nodes. For ease of description, the following mainly uses the client and the replica service node as an example.

[0058] In an embodiment of the present invention, the operation order of nodes and the execution order of corresponding nodes are used to establish a causal relationship of data. For example, the operation order of the user-side client and the execution order of the replica service node can be used to establish a causal relationship of data to maintain causal consistency of data for users.

[0059] In a distributed storage system, a business domain refers to a logical area divided to implement specific business logic or functions. Each business domain usually contains a set of related data and operations, which have consistency requirements within the business domain. For different business domains, the distributed storage system of the embodiment of the present invention provides two consistency modes:

[0060] Mode 1: The default global causal consistency mode. The interactive client and the replica service node exchange the operation sequence number (local operation sequence number) sn in real time during the communication process to arrange the data reading and writing order and achieve the causal consistency of the associated data.

[0061] Mode 2: On-demand local linear consistency mode for specific businesses, providing a business domain isolation mechanism and formulating linear consistency constraints for data within the business domain. Business domains other than the specified business domain can be used as default domains, using the default causal consistency; different business domains are also supported only by causal consistency.

[0062] Reference Figure 3, is a schematic diagram of the relationship between two modes in a distributed storage system provided in an embodiment of the present invention. The data in the global default domain of the distributed storage system is in a causal consistency relationship, and mode 1, that is, the global causal consistency mode, can be used to process the data; it supports on-demand specification of business domains (local business domains), and the data in the specified business domains are subject to linear consistency constraints. Mode 2, that is, on-demand local linear consistency mode, can be used to process data, such as user business data and other data.

[0063] It should be noted that the embodiments of the present invention may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0064] In the above-mentioned distributed storage system data processing method, the distributed storage system may include multiple consistency modes. When the consistency mode of the distributed storage system is the global causal consistency mode, data is processed according to the local operation sequence number generated locally at the node. When the consistency mode of the distributed storage system is the on-demand local linear consistency mode, if the node is a node of a non-specified business domain, data can be processed according to the local operation sequence number generated locally. If the node is a node of a specified business domain, data can be processed according to the local operation sequence number generated by the specified node. The distributed storage system of the embodiment of the present invention is designed with multiple consistency modes, and different consistency modes can be selected according to different storage requirements, which can reduce the global resource overhead of the distributed storage system and improve the performance of the distributed storage system.

[0065] In one embodiment of the present invention, the nodes each maintain a local operation sequence number; and processing data according to the local operation sequence number generated locally by the node includes:

[0066] Generate a first local operation sequence number according to the local operation sequence number;

[0067] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0068] The first write request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain a second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0069] receiving a response message sent by the first target node and returned in response to the first write request; the response message includes a second local operation sequence number;

[0070] The local operation sequence number is updated according to the second local operation sequence number.

[0071] Reference Figure 4 , is a schematic diagram of implementing a global causal consistency mode provided in an embodiment of the present invention. Specifically, the nodes of the distributed storage system may include clients and replica service nodes. For clients and replica service nodes, implementing the default global causal consistency mode may include:

[0072] 1) Global causal consistency operation sequence number sn: Client Cm and replica service node Pm maintain their own local operation sequence number sn, where client Cm is recorded as Cm:sn and replica service node Pm is recorded as Pm:sn; the initial value of the local operation sequence number sn is 0, and the valid sn starts from 1; 2) Since sn usually increases monotonically in chronological order, it can represent the order of occurrence of operation events (such as write or read operation events): the larger the sn, the later the event occurs.

[0073] In some specific examples, the process of a write operation event in a global causal consistency mode may include:

[0074] 1) When the business end B1 needs to write data A=x, it calls the client (for example, client C1), and client C1 obtains C1:sn=C1:sn+1 and sends a write request write(A=x, sn=C1:sn) to the replica service node P1; where A represents the write data item and x represents the write data value.

[0075] 2) After receiving the write request write(A=x, sn=C1:sn), the replica service node P1 updates the local P1:sn=max(P1:sn, C1:sn)+1 and persists the data required by the write request (key=A, ts=P1:sn)=x;

[0076] 3) The replica service node P1 returns a success message to the client C1 along with P1:sn, i.e., a response message resp(succ, P1:sn);

[0077] 4) After receiving resp(succ, P1:sn) returned by the replica service node P1, the client C1 updates the local operation sequence number C1:sn=max(C1:sn, P1:sn)+1.

[0078] In one embodiment of the present invention, processing data according to the local operation sequence number generated locally by the node may include:

[0079] Generate a third local operation sequence number according to the local operation sequence number.

[0080] Generate a first read request; the first read request includes the third local operation sequence number and the first write data item corresponding to the first local operation sequence number.

[0081] Send the first read request to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain a fourth local operation sequence number, and obtain the first write data value corresponding to the maximum local operation sequence number of the first write data item.

[0082] Receive a response message returned by the first target node for the first read request; the response message includes the fourth local operation sequence number and the first write data value.

[0083] Update the local operation sequence number according to the fourth local operation sequence number.

[0084] In some specific examples, the process of the read operation event of global causal consistency may include:

[0085] 1) The service end B2 needs to read data A, calls the client C2, the client C2 obtains C2:sn = C2:sn + 1, and sends a read request read(A, sn = C2:sn) to the replica service node P1.

[0086] 2) After receiving read(A, sn = C2:sn), the replica service node P1 updates the local P1:sn = max(P1:sn, C2:sn) + 1, and queries the data value y corresponding to the closest tuple (key = A, ts < P1:sn); max() represents taking the maximum value.

[0087] 3) The replica service node P1 returns a response message resp(A = y, P1:sn).

[0088] 3) After receiving resp(A = y, P1:sn) returned by the replica service node P1, the client C2 updates the local operation sequence number C2:sn = max(C2:sn, P1:sn) + 1.

[0089] In an embodiment of the present invention, the node corresponding to the specified service domain includes the unique identifier corresponding to the specified service domain; the specified node maintains a local operation sequence number; processing data according to the local operation sequence number generated by the specified node includes:

[0090] Generate a second write request; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value;

[0091] Sending the second write request to a second target node corresponding to the designated business domain; the second target node is used to obtain a first local operation sequence number generated according to the local operation sequence number from the designated node, and storing the second write data and the unique identifier according to the first local operation sequence number;

[0092] Receive a response message returned by the second target node in response to the second write request.

[0093] In a specific implementation, for clients and replica service nodes, an on-demand local linearizability mode is implemented. The specific steps may include:

[0094] 1) Local linear consistency business domain unique name domain: 1.1) The distributed storage system provides a business domain unique name domain mechanism to distinguish different business domains; 1.2) The business domain unique name / unique identifier domain can be specified by the business end and is globally unique, that is, it is unique in the entire distributed storage system; 1.3) All domains are evenly distributed to each replica service node through a distributed hash mechanism; 1.4) The replica service node assigned to the domain, in addition to providing business data read and write services, is also responsible for the domain's linear consistency operation sequence number csn (i.e., local operation sequence number) generation service; 1.5) csn is monotonically increasing and can represent the order in which write operations occur: the smaller the csn, the earlier the write time, and the larger the csn, the later the write time.

[0095] The purpose of the distributed hash mechanism is to achieve load balancing and uniform data distribution of the distributed storage system, and to ensure that the amount of data and the load on each node of the distributed storage system are roughly the same. Of course, in addition to the distributed hash mechanism, the embodiment of the present invention can also use methods such as randomness and user selection to determine the node corresponding to the business domain, and the embodiment of the present invention does not need to be limited to this.

[0096] In some specific examples, the process of a write operation event in on-demand local linearizability mode may include:

[0097] 1) The business end B3 specifies the unique name of the business domain domain=un, and calls the client C3 to write a series of data, such as A=x1, B=y1, C=z1;

[0098] 2) Client C3 encapsulates each write operation into several write requests such as cwrite(domain=un, A=x1), cwrite(domain=un, B=y1), cwrite(domain=un, B=z1) and sends them to replica service nodes P3, P4, P5 responsible for different data respectively;

[0099] 3) The replica service node P3 obtains the monotonically increasing serial number csn of the unique identifier domain=un of the designated service domain from the SPm (designated node) responsible for domain=un; as an optimization, a batch of csn can be obtained at one time; in addition, the node that obtains csn can also be a client.

[0100] 3) The replica service node P3 uses csn as the sequence number for this write request, persists the data, and returns a successful response message to the client C3;

[0101] 4) The replica service node P3 caches the largest csn sequence number of domain=un;

[0102] 5) The replica service nodes P4 and P5 perform the same operations as the replica service node P3 to implement the write operation event, which will not be repeated here.

[0103] In one embodiment of the present invention, the node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; and processing data according to the local operation sequence number generated by the designated node includes:

[0104] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0105] Sending the second read request to a second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0106] Receive a response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0107] In some specific examples, the process of a read operation event in the on-demand local linearizability mode may include:

[0108] 1) The business end B4 specifies the unique name of the business domain domain=un, and calls the client C4 to read a series of data, such as A, B, C;

[0109] 2) Client C4 encapsulates each read operation into several read requests such as cread(domain=un, A), cread(domain=un, B), cread(domain=un, C) and sends them to replica service nodes P3, P4, P5 responsible for different data respectively;

[0110] 3) The replica service node P3 knows the local maximum operation sequence number csn of domain=un, and uses the tuple (A, csn) to query the last write record of data A, and the one with the largest csn is the latest;

[0111] 4) The replica service node P3 returns the read data to the client C4;

[0112] 5) The replica service nodes P4 and P5 perform the same operations as the replica service node P3 to implement the write operation event, which will not be repeated here.

[0113] The causal consistency data processing method in the distributed storage system provided in the embodiment of the present invention aims to reasonably apply the correlation of user business data to ensure the causal consistency of user business data. The distributed storage system in the embodiment of the present invention does not have a centralized timing coordination service, and the overall design implementation difficulty is also reduced compared to the linear consistency model. At the same time, from the user's perspective, the distributed consistency guarantee provided by the distributed storage system is not damaged, ensuring the user's experience of the distributed storage system.

[0114] In order to enable those skilled in the art to better understand the embodiments of the present invention, a specific embodiment is used for illustration below. In a specific implementation, the distributed storage system may be a distributed KV storage system.

[0115] Embodiment 1: A distributed KV storage system combining global causal consistency with on-demand local linear consistency, a distributed KV storage system in a global causal consistency mode, with a Key-Value pair as the latest data read and write access unit. Among them, Key is the unique key or name of the data (data item / write data item), which is the basis for data writing and query; Value is the data value / write data value.

[0116] The distributed KV storage system according to the embodiment of the present invention adopts the global causal consistency model by default:

[0117] 1) The client maintains the local operation sequence number Ci:sn, the initial value is 0, and the effective value starts from 1;

[0118] 2) sn increases monotonically in chronological order, and therefore represents the order in which events occur: the larger the sn, the later the event occurs.

[0119] 3) When the client writes data with key = A and value = x, it first executes Ci:sn = Ci:sn + 1, and then sends a write request put(key = A, value = x, sn = Ci:sn) to the replica service node;

[0120] 4) Which replica service node is responsible for key = A is assigned by the system through the distributed hash mechanism;

[0121] 5) After the replica service node receives put(key = A, value = x, sn = Ci:sn), it first updates its own Pi:sn = max(Pi:sn, Ci:sn) + 1;

[0122] 6) After the replica service node persists (A, Pi:sn) = x locally, it returns resp(succ, key = A, Pi:sn) to the client;

[0123] 7) The client receives resp(succ, key = A, Pi:sn) and updates the local Ci:sn = max(Ci:sn, Pi:sn) + 1.

[0124] 8) When the client reads data with key = A, it first executes Ci:sn = Ci:sn + 1 and sends a read request get(key = A, sn = Ci:sn) to the replica service node;

[0125] 9) After the replica service node receives get(key = A, sn = Ci:sn), it first updates its own Pi:sn = max(Pi:sn, Ci:sn) + 1;

[0126] 10) The replica service node queries the latest data value = x that satisfies key = A and sn < Pi:sn and returns resp(succ, key = A, value = x, Pi:sn) to the client;

[0127] 11) The client receives resp and updates the local Ci:sn = max(Ci:sn, Pi:sn) + 1.

[0128] Based on the distributed KV storage system of the embodiments of the present invention, it also supports dividing business domains through domains and supports the linear consistency model within the specified business domain:

[0129] 1) The service end specifies the unique name of the business domain domain = un and calls the client to write data to the business domain, such as A = x1;

[0130] 2) The client generates a write request cput(domain = un, key = A, value = x1) and sends it to the replica service node responsible for key = A;

[0131] 3) Note: The key = A in domain = un is not the same key as the key = A in the global scope, and the consistency models are also different;

[0132] 4) Which replica service node is responsible for key = A is allocated by the distributed storage system through the distributed hashing mechanism;

[0133] 5) After the replica service node P1 receives the cput request, it first checks whether the csn for domain = un has been pre-applied locally. If not, it batches and applies for a batch of csn from another replica service node corresponding to domain = un;

[0134] 6) The replica service node P1 persists domain = un, key = A, value = x1, sn = csn, records the latest csn for domain = un in memory, and returns success to the client;

[0135] 7) When the service end reads data, it also needs to specify domain = un and call the client to initiate a read request;

[0136] 8) The client generates a read request cget(domain = un, key = A) and sends it to the replica service node responsible for key = A;

[0137] 9) After the replica service node P1 receives the cget request, it first obtains the latest csn for domain = un locally, and then queries the latest value that satisfies domain = un, key = A, and sn < csn, and returns it to the client.

[0138] Embodiment 2: The linear consistency model of distributed block storage with LUN as the business domain unit. Among them, LUN (Logical Unit Number) is used to identify and manage the logical storage units divided by physical storage devices (such as disk arrays).

[0139] The distributed storage system of the embodiment of the present invention can also support the linear consistency model with LUN as the unit, and the specific implementation steps are as follows:

[0140] 1) The service end specifies LUN = un and calls the client to write data to LUN, such as LUN = un, lba = m, data = x;

[0141] 2) The client generates a write request write(LUN = un, lba = m, data = x) and sends it to the replica service node responsible for LUN = un, lba = 0;

[0142] 3) After the replica service node P1 receives a write request, it first checks whether the csn with LUN = un has been pre-applied locally. If not, it batch-applies for a batch of csns from another replica service node corresponding to LUN = un;

[0143] 4) The replica service node P1 persists LUN = un, lba = m, data = x, and sn = csn, records the latest csn of LUN = un in memory, and returns a successful response message to the client;

[0144] 5) When the service end reads data, it also needs to specify LUN = un and call the client to initiate a read request;

[0145] 6) The client generates a read request read(LUN = un, lba = m) and sends it to the corresponding replica service node;

[0146] 7) After the replica service node P1 receives a read request, it first obtains the latest csn of LUN = un locally, and then queries the latest value that satisfies LUN = un, lba = m, and sn < csn, and returns it to the client.

[0147] The method for processing data in the distributed storage system provided by the embodiments of the present invention lifts the global linear consistency constraint, and by default uses a more relaxed causal consistency model in most scenarios (default domain), providing effective data consistency guarantees for users. At the same time, the decentralized design avoids the single-point bottleneck brought by the centralized timing coordination service. In addition, when the business requires it, the business domain scope is delimited by the unique business domain name (specifying the business domain), and local linear consistency is executed as needed. Among them, the data in the same business domain is linearly consistent, and unrelated business domains can run in parallel independently. The overall design of the consistency of the distributed system in the embodiments of the present invention is less difficult to implement than the global linear consistency model, improving the performance of the distributed storage system in the embodiments of the present invention.

[0148] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0149] Refer to Figure 5 , which is a structural block diagram of a data processing device for a distributed storage system provided in the embodiments of the present invention. As Figure 5 shown, the distributed storage system includes multiple nodes, and this device may specifically include the following modules:

[0150] A first data processing module 501, configured to process data according to a local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is a global causal consistency mode;

[0151] The second data processing module 502 is used to process data according to the local operation sequence number generated locally by the node if the node is a node of a non-specified business domain when the consistency mode of the distributed storage system is an on-demand local linear consistency mode; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

[0152] In one embodiment of the present invention, each of the nodes maintains a local operation sequence number; the first data processing module 501 is used to:

[0153] Generate a first local operation sequence number according to the local operation sequence number;

[0154] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0155] The first write request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain a second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0156] receiving a response message sent by the first target node and returned in response to the first write request; the response message includes a second local operation sequence number;

[0157] The local operation sequence number is updated according to the second local operation sequence number.

[0158] In one embodiment of the present invention, the first data processing module 501 is used to:

[0159] Generate a third local operation sequence number according to the local operation sequence number;

[0160] Generate a first read request; the first read request includes the third local operation sequence number and a first write data item corresponding to the first local operation sequence number;

[0161] The first read request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain a fourth local operation sequence number, and obtain a first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item;

[0162] receiving a response message sent by the first target node and returned in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value;

[0163] The local operation sequence number is updated according to the fourth local operation sequence number.

[0164] In an embodiment of the present invention, the node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; the second data processing module 502:

[0165] Generate a second write request; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value;

[0166] Sending the second write request to a second target node corresponding to the designated business domain; the second target node is used to obtain a first local operation sequence number generated according to the local operation sequence number from the designated node, and storing the second write data and the unique identifier according to the first local operation sequence number;

[0167] Receive a response message returned by the second target node in response to the second write request.

[0168] In an embodiment of the present invention, the node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; the second data processing module 502:

[0169] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0170] Sending the second read request to a second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0171] Receive a response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0172] In an embodiment of the present invention, the nodes of the designated service domain are determined by using a distributed hash mechanism.

[0173] In one embodiment of the present invention, the local operation sequence number and the partial operation sequence number are monotonically increasing.

[0174] In an embodiment of the present invention, a distributed storage system may include multiple consistency modes. When the consistency mode of the distributed storage system is a global causal consistency mode, data is processed according to a local operation sequence number generated locally at the node. When the consistency mode of the distributed storage system is an on-demand local linear consistency mode, if the node is a node of a non-specified business domain, data may be processed according to the local operation sequence number generated locally. If the node is a node of a specified business domain, data may be processed according to a local operation sequence number generated by the specified node. The distributed storage system of an embodiment of the present invention is designed with multiple consistency modes, and different consistency modes may be selected according to different storage requirements, which may reduce the global resource overhead of the distributed storage system and improve the performance of the distributed storage system.

[0175] As for the above-mentioned device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0176] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0177] Memory 603, used for storing computer programs;

[0178] The processor 601 is used to implement the distributed storage system data processing method described in any of the above embodiments when executing the program stored in the memory 603:

[0179] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0180] The communication interface is used for communication between the above terminal and other devices.

[0181] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0182] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0183] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the distributed storage system data processing method described in any of the above embodiments.

[0184] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the distributed storage system data processing method described in any one of the above embodiments.

[0185] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk SSD), etc.

[0186] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0187] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0188] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A distributed storage system data processing method, characterized in that: The distributed storage system includes a plurality of nodes, and the method includes: When the consistency mode of the distributed storage system is a global causal consistency mode, processing data according to a local operation sequence number generated locally by the node; When the consistency mode of the distributed storage system is the on-demand local linear consistency mode, if the node is a node of a non-specified business domain, the data is processed according to the local operation sequence number generated locally by the node; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

2. The method according to claim 1, characterized in that Each of the nodes maintains a local operation sequence number; Processing data according to a local operation sequence number generated locally by the node includes: Generate a first local operation sequence number according to the local operation sequence number; Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value; The first write request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain a second local operation sequence number, and store the first write data according to the second local operation sequence number; receiving a response message sent by the first target node and returned in response to the first write request; the response message includes a second local operation sequence number; The local operation sequence number is updated according to the second local operation sequence number.

3. The method according to claim 2, characterized in that Processing data according to a local operation sequence number generated locally by the node includes: Generate a third local operation sequence number according to the local operation sequence number; Generate a first read request; the first read request includes the third local operation sequence number and a first write data item corresponding to the first local operation sequence number; The first read request is sent to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain a fourth local operation sequence number, and obtain a first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item; receiving a response message sent by the first target node and returned in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value; The local operation sequence number is updated according to the fourth local operation sequence number.

4. The method according to claim 1, characterized in that: The node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; and processing data according to the local operation sequence number generated by the designated node includes: Generate a second write request; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value; Sending the second write request to a second target node corresponding to the designated business domain; the second target node is used to obtain a first local operation sequence number generated according to the local operation sequence number from the designated node, and storing the second write data and the unique identifier according to the first local operation sequence number; Receive a response message returned by the second target node in response to the second write request.

5. The method according to claim 3, characterized in that: The node corresponding to the designated service domain includes a unique identifier corresponding to the designated service domain; the designated node maintains a local operation sequence number; and processing data according to the local operation sequence number generated by the designated node includes: Generate a second read request; the second read request includes the unique identifier and the second write data item; Sending the second read request to a second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier; Receive a response message returned by the second target node in response to the second read request; the response message includes the second write data value.

6. The method according to claim 1, characterized in that The nodes of the designated business domain are determined by using a distributed hashing mechanism.

7. The method according to claim 1, characterized in that The local operation sequence number and the partial operation sequence number are monotonically increasing.

8. A distributed storage system data processing device, characterized in that: The distributed storage system includes a plurality of nodes, and the device includes: A first data processing module, configured to process data according to a local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is a global causal consistency mode; The second data processing module is used to process data according to the local operation sequence number generated locally by the node if the node is a node of a non-specified business domain when the consistency mode of the distributed storage system is an on-demand local linear consistency mode; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images