Distributed storage system data processing method and apparatus

Abstract

This invention provides a data processing method and apparatus for a distributed storage system. The distributed storage system includes multiple nodes. The method includes: when the consistency mode of the distributed storage system is a global causal consistency mode, processing data according to locally generated local operation sequence numbers of the nodes; when the consistency mode of the distributed storage system is an on-demand local linear consistency mode, if the node is a node outside a specified business domain, processing data according to locally generated local operation sequence numbers of the nodes; if the node is a node within a specified business domain, processing data according to locally generated local operation sequence numbers of the specified nodes. This invention's distributed storage system designs multiple consistency modes, allowing selection of different consistency modes based on different storage needs, reducing the global resource overhead of the distributed storage system and improving its performance.

Description

Technical Field

[0001] The present invention relates to the field of computer storage, and in particular to a data processing method for a distributed storage system, a data processing device for a distributed storage system, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of information technology, the amount of data is growing exponentially and will continue to grow in the future. Therefore, traditional storage systems can no longer meet the needs of modern applications. In order to cope with the needs of storing and managing massive amounts of data, distributed storage systems have emerged.

[0003] Distributed storage systems distribute data across multiple independent devices (nodes), sharing the storage load and using location servers to locate stored information. This not only improves the system's reliability, availability, and access efficiency but also makes it easy to expand.

[0004] Distributed storage systems should provide necessary consistency guarantees to users, ensuring the correctness of user data and providing observability of system operation timing. Therefore, distributed storage systems are designed with consistency models to achieve consistency. However, current consistency models for distributed storage systems cannot meet diverse storage needs. Summary of the Invention

[0005] In view of the above problems, a data processing method and apparatus for a distributed storage system are proposed to overcome or at least partially solve the above problems. The specific technical solution is as follows:

[0006] In a first aspect of the present invention, a data processing method for a distributed storage system is provided, the distributed storage system comprising multiple nodes, the method comprising:

[0007] When the consistency mode of the distributed storage system is global causal consistency mode, data is processed according to the local operation sequence number generated by the node.

[0008] When the consistency mode of the distributed storage system is on-demand local linear consistency mode, if the node is a node outside the specified business domain, the data is processed according to the local operation sequence number generated locally by the node; if the node is a node within the specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

[0009] In one embodiment of the present invention, each node maintains a local operation sequence number; processing data according to the local operation sequence number generated by the node includes:

[0010] A first local operation sequence number is generated based on the local operation sequence number;

[0011] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0012] The first write request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain the second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0013] Receive a response message sent by the first target node in response to the first write request; the response message includes a second local operation sequence number;

[0014] Update the local operation sequence number according to the second local operation sequence number.

[0015] In one embodiment of the present invention, processing data according to the local operation sequence number generated locally by the node includes:

[0016] A third local operation sequence number is generated based on the aforementioned local operation sequence number;

[0017] Generate a first read request; the first read request includes the third local operation sequence number and the first write data item corresponding to the first local operation sequence number;

[0018] The first read request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain the fourth local operation sequence number, and obtain the first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item;

[0019] Receive a response message sent by the first target node in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value;

[0020] Update the local operation sequence number according to the fourth local operation sequence number.

[0021] In one embodiment of the present invention, the node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; data is processed according to the local operation sequence number generated by the specified node, including:

[0022] A second write request is generated; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value.

[0023] The second write request is sent to the second target node corresponding to the specified business domain; the second target node is used to obtain a first local operation sequence number generated based on the local operation sequence number from the specified node, and store the second write data and the unique identifier based on the first local operation sequence number;

[0024] Receive the response message returned by the second target node in response to the second write request.

[0025] In one embodiment of the present invention, the node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; data is processed according to the local operation sequence number generated by the specified node, including:

[0026] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0027] The second read request is sent to the second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0028] Receive the response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0029] In one embodiment of the present invention, the node of the specified business domain is determined using a distributed hash mechanism.

[0030] In one embodiment of the present invention, the local operation sequence number and the local operation sequence number are monotonically increasing.

[0031] This invention also discloses a data processing apparatus for a distributed storage system, wherein the distributed storage system includes multiple nodes, and the apparatus includes:

[0032] The first data processing module is used to process data according to the local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is the global causal consistency mode.

[0033] The second data processing module is used to process data according to the local operation sequence number generated locally by the node if the node is not a node of a specified business domain, and to process data according to the local operation sequence number generated by the specified node if the node is a node of a specified business domain.

[0034] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform any of the distributed storage system data processing methods described above.

[0035] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the distributed storage system data processing methods described above.

[0036] Compared with related technologies, the embodiments of the present invention have at least the following advantages:

[0037] In this embodiment of the invention, the distributed storage system may include multiple consistency modes. When the consistency mode of the distributed storage system is global causal consistency, data is processed according to the locally generated local operation sequence number of the node. When the consistency mode of the distributed storage system is on-demand local linear consistency, if the node is not a node of a specified business domain, data can be processed according to the locally generated local operation sequence number; if the node is a node of a specified business domain, data can be processed according to the local operation sequence number generated by the specified node. The distributed storage system of this embodiment of the invention is designed with multiple consistency modes, and different consistency modes can be selected according to different storage needs, which can reduce the global resource overhead of the distributed storage system and improve the performance of the distributed storage system. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0039] Figure 1 This is a schematic diagram of a global view of operation events in a distributed storage system provided in an embodiment of the present invention;

[0040] Figure 2 This is a flowchart illustrating the steps of a data processing method for a distributed storage system provided in an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram illustrating the relationship between two modes in a distributed storage system provided in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the implementation of a global causal consistency pattern provided in an embodiment of the present invention;

[0043] Figure 5 This is a structural block diagram of a distributed storage system data processing device provided in an embodiment of the present invention;

[0044] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0046] In practical implementation, a reliable storage system needs to provide users with data consistency guarantees. Consistency guarantees mean that after a user successfully writes data to the storage system, they can read the correct data value, and the data will be correct and consistent no matter when it is read afterward.

[0047] To achieve a higher level of reliability, storage systems also need to ensure that the timing of all operational events (events) observed by all users is the same. Therefore, storage systems need to implement a consistency model. For example, after a user successfully writes A=1, B=2, A=3, B=4 to the storage system sequentially, the user will read the latest A=3, B=4. Throughout the writing process and after completion, each user will observe that the storage system processes the writes in the order of A=1, B=2, A=3, B=4. Consistency models can include various types, such as linear consistency, sequential consistency, eventual consistency, and causal consistency. The consistency model in the example above is a linear consistency model. The linear consistency model is the highest level of consistency model achievable in engineering implementation. It ensures that the data a user reads is accurate and valid at all times, and it is also the foundation for storage systems to support multi-versioning, backup / recovery, and snapshot / rollback features.

[0048] Early storage systems were single-replica storage systems. A single-replica storage system means that the data in the storage system is stored in only one copy, without redundant backups. The storage system can use the local physical clock or auto-incrementing sequence number to sort data operations, and it is easy to provide linear consistency guarantees.

[0049] Currently, most storage systems are distributed storage systems. Because each node in a distributed storage system may be deployed on different physical hosts, ensuring linear consistency requires prioritizing all operational events across the entire distributed storage system. (See reference...) Figure 1This is a schematic diagram of a global view of operation events in a distributed storage system provided in an embodiment of the present invention. Assuming that the distributed storage system includes replica service node P1 and replica service node P2, replica service node P1 executes write:x=4, read:y=2, and replica service node P2 executes read:x=0, write:y=2, read:x=4. Then, according to the sequence of operation events, the global view of the distributed storage system records is: read:x=0→write:x=4→write:y=2→read:y=2→read:x=4.

[0050] Due to the cross-node nature of distributed storage systems, they no longer have a unique physical clock to determine the timing of operation events, unlike single-replica storage systems. If physical clock sequencing is still relied upon, consider that the processing time of a single request in a high-concurrency system is measured in microseconds, and a high-precision physical clock mechanism with microsecond accuracy is extremely expensive.

[0051] Therefore, to achieve linear consistency, distributed storage systems often need to rely on complex logical clocks or hybrid clock algorithms for time synchronization. These algorithms are complex to implement and difficult to engineer, such as Amazon's Dynamo. Alternatively, a centralized time-series coordination service can be established within the system, which can easily become a performance bottleneck, as seen in TiDB's use of TSO in PD. This results in distributed storage systems incurring significant costs to achieve accurate and efficient linear consistency.

[0052] Because achieving linear consistency in distributed storage systems is so costly, some storage systems have simply abandoned the linear consistency model altogether, opting instead for an eventual consistency model, such as Netflix's Eureka and Alibaba's Nacos. However, these storage systems are distributed registries, with higher requirements for availability and lower requirements for data real-time performance. Furthermore, eventual consistency is a weak consistency model, making it difficult to apply to distributed storage systems.

[0053] To address the aforementioned issues, this invention provides a method for data processing in a distributed storage system. It employs a global causal consistency model to provide effective data consistency guarantees for users. Furthermore, when business needs dictate a business domain, the distributed storage system can select an on-demand local linear consistency model for a specified business domain, providing even more effective data consistency guarantees for users within that domain. Thus, the overall design and implementation difficulty of the consistency model in the distributed storage system of this invention is far lower than that of the global linear consistency model.

[0054] Reference Figure 2 The above is a flowchart of the steps of a data processing method for a distributed storage system provided in an embodiment of the present invention, as follows: Figure 1As shown, the distributed storage system includes multiple nodes, and the method may specifically include the following steps:

[0055] Step 201: When the consistency model of the distributed storage system is a global causal consistency model, data is processed according to the local operation sequence number generated locally by the node.

[0056] Step 202: When the consistency mode of the distributed storage system is on-demand local linear consistency mode, if the node is a node of a non-specified business domain, the data is processed according to the local operation sequence number generated locally by the node; if the node is a node of a specified business domain, the data is processed according to the local operation sequence number generated by the specified node.

[0057] In a practical implementation, a distributed storage system includes multiple nodes, such as client and replica service nodes. Of course, it can also include other nodes. For ease of explanation, the following mainly uses client and replica service nodes as examples.

[0058] In this embodiment of the invention, the order of operation of nodes and the execution order of corresponding nodes are used to establish the causal relationship of data. For example, the order of operation of the user-side client and the execution order of the replica service node can be used to establish the causal relationship of data, so as to maintain the causal consistency guarantee of data for the user.

[0059] In a distributed storage system, a business domain (domain) refers to a logical area divided to implement specific business logic or functions. Each business domain typically contains a set of related data and operations, which must maintain consistency within the business domain. For different business domains, the distributed storage system of this invention provides two consistency modes:

[0060] Mode 1: The default global causal consistency mode. This mode utilizes the interaction between clients and replica service nodes to exchange operation sequence numbers (local operation sequence numbers) sn in real time during communication, arranging the data read / write order to achieve causal consistency of related data.

[0061] Mode 2: On-demand local linear consistency mode for specific business needs, providing a business domain isolation mechanism and defining linear consistency constraints for data within a business domain. Business domains outside the specified business domain can be used as default domains, employing default causal consistency; different business domains are also supported solely by causal consistency.

[0062] Reference Figure 3This is a schematic diagram illustrating the relationship between two modes in a distributed storage system provided in this embodiment of the invention. The data in the global default domain of the distributed storage system has a causal consistency relationship, and mode 1, i.e., global causal consistency mode, can be used to process the data. On-demand specification of business domains (local business domains) is supported, and linear consistency constraints are implemented on the data in the specified business domains. Mode 2, i.e., on-demand local linear consistency mode, can be used to process the data, such as user business data and other data.

[0063] It should be noted that the embodiments of the present invention may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0064] In the aforementioned data processing method for a distributed storage system, the distributed storage system can include multiple consistency modes. When the consistency mode of the distributed storage system is global causal consistency, data is processed according to the locally generated local operation sequence number of the node. When the consistency mode of the distributed storage system is on-demand local linear consistency, if the node is not a node of a specified business domain, data can be processed according to the locally generated local operation sequence number; if the node is a node of a specified business domain, data can be processed according to the local operation sequence number generated by the specified node. The distributed storage system of this embodiment of the invention designs multiple consistency modes, allowing different consistency modes to be selected according to different storage needs, which can reduce the global resource overhead of the distributed storage system and improve its performance.

[0065] In one embodiment of the present invention, each node maintains a local operation sequence number; processing data according to the local operation sequence number generated by the node includes:

[0066] A first local operation sequence number is generated based on the local operation sequence number;

[0067] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0068] The first write request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain the second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0069] Receive a response message sent by the first target node in response to the first write request; the response message includes a second local operation sequence number;

[0070] Update the local operation sequence number according to the second local operation sequence number.

[0071] Reference Figure 4 This is a schematic diagram illustrating the implementation of a global causal consistency mode provided in an embodiment of the present invention. Specifically, the nodes of the distributed storage system may include client and replica service nodes. For the client and replica service nodes, a default global causal consistency mode is implemented, and the specific steps may include:

[0072] 1) Global causal consistency operation sequence number sn: The client Cm and the replica service node Pm maintain their respective local operation sequence number sn, where the client Cm is denoted as Cm:sn and the replica service node Pm is denoted as Pm:sn; the initial value of the local operation sequence number sn is 0, and the valid sn starts from 1; 2) Since sn usually increases monotonically in chronological order, it can represent the order in which operation events (such as write or read operation events) occur: the larger the sn, the later the event occurs.

[0073] In some specific examples, the write operation event process in the global causal consistency mode can include:

[0074] 1) When business terminal B1 needs to write data A=x, it calls the client (e.g., client C1). Client C1 obtains C1:sn=C1:sn+1 and sends a write request write(A=x, sn=C1:sn) to replica service node P1; where A represents the data item to be written and x represents the data value to be written.

[0075] 2) After receiving the write request write(A=x, sn=C1:sn), the replica service node P1 updates its local P1:sn=max(P1:sn, C1:sn)+1 and persists the data (key=A, ts=P1:sn)=x requested by the write request.

[0076] 3) When replica service node P1 returns success to client C1, it also includes P1:sn, i.e., the response message resp(succ, P1:sn);

[0077] 4) After receiving the resp(succ, P1:sn) returned by the replica service node P1, the client C1 updates the local operation sequence number C1:sn = max(C1:sn, P1:sn) + 1.

[0078] In one embodiment of the present invention, processing data based on a locally generated local operation sequence number of the node may include:

[0079] Generate a third local operation sequence number according to the local operation sequence number.

[0080] Generate a first read request; the first read request includes the third local operation sequence number and the first write data item corresponding to the first local operation sequence number.

[0081] Send the first read request to a first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain a fourth local operation sequence number, and obtain the first write data value corresponding to the maximum local operation sequence number of the first write data item.

[0082] Receive a response message returned by the first target node for the first read request; the response message includes the fourth local operation sequence number and the first write data value.

[0083] Update the local operation sequence number according to the fourth local operation sequence number.

[0084] In some specific examples, the process of the read operation event of global causal consistency may include:

[0085] 1) Service end B2 needs to read data A, call client C2, client C2 obtains C2:sn = C2:sn + 1, and send a read request read(A, sn = C2:sn) to replica service node P1.

[0086] 2) After receiving read(A, sn = C2:sn), replica service node P1 updates local P1:sn = max(P1:sn, C2:sn) + 1, and queries the data value y corresponding to the closest tuple (key = A, ts < P1:sn); max() represents taking the maximum value.

[0087] 3) Replica service node P1 returns a response message resp(A = y, P1:sn).

[0088] 3) After receiving resp(A = y, P1:sn) returned by replica service node P1, client C2 updates the local operation sequence number C2:sn = max(C2:sn, P1:sn) + 1.

[0089] In an embodiment of the present invention, the node corresponding to the specified service domain includes the unique identifier corresponding to the specified service domain; the specified node maintains a local operation sequence number; processing data according to the local operation sequence number generated by the specified node includes:

[0090] A second write request is generated; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value.

[0091] The second write request is sent to the second target node corresponding to the specified business domain; the second target node is used to obtain a first local operation sequence number generated based on the local operation sequence number from the specified node, and store the second write data and the unique identifier based on the first local operation sequence number;

[0092] Receive the response message returned by the second target node in response to the second write request.

[0093] In the specific implementation, for both the client and the replica service node, an on-demand local linear consistency mode is implemented. Specific steps may include:

[0094] 1) Local linear consistency business domain unique name: 1.1) The distributed storage system provides a business domain unique name mechanism to distinguish different business domains; 1.2) The business domain unique name / unique identifier domain can be specified by the business end, and is globally unique, that is, unique throughout the entire distributed storage system; 1.3) All domains are evenly distributed to each replica service node through a distributed hash mechanism; 1.4) The replica service node allocated with a domain, in addition to providing business data read and write services, is also responsible for generating the domain's linear consistency operation sequence number (csn) (i.e., local operation sequence number); 1.5) The csn is monotonically increasing and can represent the order in which write operations occur: the smaller the csn, the earlier the write time, and the larger the csn, the later the write time.

[0095] The purpose of the distributed hash mechanism is to achieve load balancing and uniform data distribution in the distributed storage system, ensuring that the amount of data and the load on each node of the distributed storage system are approximately the same. Of course, in addition to the distributed hash mechanism, embodiments of the present invention may also use methods such as random selection or user-selected methods to determine the node corresponding to the business domain; the present invention does not impose any restrictions on this.

[0096] In some specific examples, the write operation event process in the on-demand local linear consistency mode may include:

[0097] 1) Business terminal B3 specifies the unique name of the business domain as domain=un, and calls client C3 to write a series of data, such as A=x1, B=y1, C=z1;

[0098] 2) Client C3 encapsulates each write operation into several write requests such as cwrite(domain=un, A=x1), cwrite(domain=un, B=y1), cwrite(domain=un, B=z1) and sends them to the replica service nodes P3, P4, and P5, which are responsible for different data, respectively.

[0099] 3) The replica service node P3 obtains the monotonically increasing sequence number (csn) of the unique identifier domain=un from the SPm (designated node) responsible for domain=un. As an optimization, a batch of csn can be obtained at once. Alternatively, the node that obtains the csn can also be a client.

[0100] 3) The replica service node P3 uses the CSN as the sequence number for this write request, persists the data, and returns a success response message to the client C3;

[0101] 4) The P3 cache domain of the replica service node is the largest CSN sequence number;

[0102] 5) Replica service nodes P4 and P5 perform the same operations as replica service node P3 to implement write operation events, which will not be elaborated here.

[0103] In one embodiment of the present invention, the node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; data is processed according to the local operation sequence number generated by the specified node, including:

[0104] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0105] The second read request is sent to the second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0106] Receive the response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0107] In some specific examples, the procedure for read operation events in the on-demand local linear consistency mode may include:

[0108] 1) The business terminal B4 specifies the unique name of the business domain as domain=un, and calls the client C4 to read a series of data, such as A, B, C;

[0109] 2) Client C4 encapsulates each read operation into several read requests such as cread(domain=un, A), cread(domain=un, B), cread(domain=un, C) and sends them to the replica service nodes P3, P4, and P5, which are responsible for different data, respectively.

[0110] 3) The replica service node P3 knows the maximum operation sequence number csn of domain=un locally. Use the tuple (A, csn) to query the last write record of data A, and take the record with the largest csn as the latest.

[0111] 4) The replica service node P3 returns the data it has read to the client C4;

[0112] 5) Replica service nodes P4 and P5 perform the same operations as replica service node P3 to implement write operation events, which will not be elaborated here.

[0113] The causal consistency data processing method in a distributed storage system provided in this invention aims to rationally apply the correlation of user business data to ensure its causal consistency. The distributed storage system in this invention does not have a centralized time-series coordination service, and its overall design and implementation difficulty is reduced compared to the linear consistency model. Furthermore, from the user's perspective, the distributed consistency guarantee provided by the distributed storage system is not compromised, ensuring a positive user experience.

[0114] To enable those skilled in the art to better understand the embodiments of the present invention, specific examples are described below. In a specific implementation, the distributed storage system can be a distributed key-value storage system.

[0115] Example 1: A distributed key-value storage system combining global causal consistency and on-demand local linear consistency. This distributed key-value storage system uses key-value pairs as the unit for accessing the latest data. The key is the unique key or name of the data (data item / write data item), serving as the basis for data writing and querying; the value is the data value / write data value.

[0116] The distributed key-value storage system based on this invention adopts a global causal consistency model by default.

[0117] 1) The client maintains a local operation sequence number Ci:sn, with an initial value of 0 and valid values ​​starting from 1;

[0118] 2) sn increases monotonically in chronological order, thus representing the order in which events occur: the larger the sn, the later the event occurs.

[0119] 3) When the client writes data with key = A and value = x, it first executes Ci:sn = Ci:sn + 1, and then sends a write request put(key = A, value = x, sn = Ci:sn) to the replica service node;

[0120] 4) Which replica service node is responsible for key = A is assigned by the system through a distributed hash mechanism;

[0121] 5) After the replica service node receives put(key = A, value = x, sn = Ci:sn), it first updates its own Pi:sn = max(Pi:sn, Ci:sn) + 1;

[0122] 6) After the replica service node persists (A, Pi:sn) = x locally, it returns resp(succ, key = A, Pi:sn) to the client;

[0123] 7) The client receives resp(succ, key = A, Pi:sn) and updates the local Ci:sn = max(Ci:sn, Pi:sn) + 1.

[0124] 8) When the client reads data with key = A, it first executes Ci:sn = Ci:sn + 1 and sends a read request get(key = A, sn = Ci:sn) to the replica service node;

[0125] 9) After the replica service node receives get(key = A, sn = Ci:sn), it first updates its own Pi:sn = max(Pi:sn, Ci:sn) + 1;

[0126] 10) The replica service node queries the latest data value = x that satisfies key = A and sn < Pi:sn and returns resp(succ, key = A, value = x, Pi:sn) to the client;

[0127] 11) The client receives resp and updates the local Ci:sn = max(Ci:sn, Pi:sn) + 1.

[0128] The distributed KV storage system based on the embodiment of the present invention also supports dividing business domains through domains and supports a linear consistency model within a specified business domain:

[0129] 1) The service end specifies the unique name of the business domain domain = un and calls the client to write data to the business domain, such as A = x1;

[0130] 2) The client generates a write request cput(domain = un, key = A, value = x1) and sends it to the replica service node responsible for key = A;

[0131] 3) Note: The key = A in domain = un is not the same key as the key = A in the global scope, and the consistency models are also different;

[0132] 4) Which replica service node is responsible for key = A is assigned by the distributed storage system through the distributed hash mechanism;

[0133] 5) After receiving the cput request, replica service node P1 first checks whether it has pre-applied for the csn of domain = un locally. If not, it batches and applies for a batch of csn from another replica service node corresponding to domain = un;

[0134] 6) Replica service node P1 persists domain = un, key = A, value = x1, sn = csn, records the latest csn of domain = un in memory, and returns success to the client;

[0135] 7) When the business side reads data, it also needs to specify domain = un and call the client to initiate a read request;

[0136] 8) The client generates a read request cget(domain = un, key = A) and sends it to the replica service node responsible for key = A;

[0137] 9) After receiving the cget request, replica service node P1 first obtains the latest csn of local domain = un, then queries the latest value that satisfies domain = un, key = A, sn < csn, and returns it to the client.

[0138] Embodiment 2: The linear consistency model of distributed block storage with LUN as the business domain unit. Among them, LUN (Logical Unit Number) is used to identify and manage the logical storage units divided by physical storage devices (such as disk arrays).

[0139] The distributed storage system of the embodiment of the present invention can also support the linear consistency model with LUN as the unit. The specific implementation steps are as follows:

[0140] 1) The business side specifies LUN = un and calls the client to write data to LUN, such as LUN = un, lba = m, data = x;

[0141] 2) The client generates a write request write(LUN = un, lba = m, data = x) and sends it to the replica service node responsible for LUN = un, lba = 0;

[0142] 3) After the replica service node P1 receives a write request, it first checks whether the csn with LUN = un has been pre-applied locally. If not, it batch-applies for a batch of csns from another replica service node corresponding to LUN = un;

[0143] 4) The replica service node P1 persists LUN = un, lba = m, data = x, sn = csn, records the latest csn of LUN = un in memory, and returns a successful response message to the client;

[0144] 5) When the business side reads data, it also needs to specify LUN = un and call the client to initiate a read request;

[0145] 6) The client generates a read request read(LUN = un, lba = m) and sends it to the corresponding replica service node;

[0146] 7) After the replica service node P1 receives the read request, it first obtains the latest csn of LUN = un locally, and then queries the latest value that satisfies LUN = un, lba = m, and sn < csn, and returns it to the client.

[0147] The method for processing data in the distributed storage system provided by the embodiments of the present invention lifts the global linear consistency constraint, and by default uses a more relaxed causal consistency model in most scenarios (default domain), providing effective data consistency guarantee for users. At the same time, the decentralized design avoids the single-point bottleneck brought by the centralized timing coordination service. In addition, when the business requires, the scope of the business domain is defined by the unique business domain name (specifying the business domain), and local linear consistency is executed as needed. Among them, the data in the same business domain is linearly consistent, and unrelated business domains can run in parallel independently. The overall design of the consistency of the distributed system in the embodiments of the present invention is less difficult to implement than the global linear consistency model, improving the performance of the distributed storage system in the embodiments of the present invention.

[0148] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0149] Refer to Figure 5 , which is the structural block diagram of a data processing device in the distributed storage system provided in the embodiments of the present invention. As Figure 5 shown, the distributed storage system includes multiple nodes, and this device may specifically include the following modules:

[0150] The first data processing module 501 is used to process data according to the local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is the global causal consistency mode.

[0151] The second data processing module 502 is used to process data according to the local operation sequence number generated locally by the node if the node is not a node of a specified business domain, and to process data according to the local operation sequence number generated by the specified node if the node is a node of a specified business domain.

[0152] In one embodiment of the present invention, each node maintains its own local operation sequence number; the first data processing module 501 is used for:

[0153] A first local operation sequence number is generated based on the local operation sequence number;

[0154] Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value;

[0155] The first write request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain the second local operation sequence number, and store the first write data according to the second local operation sequence number;

[0156] Receive a response message sent by the first target node in response to the first write request; the response message includes a second local operation sequence number;

[0157] Update the local operation sequence number according to the second local operation sequence number.

[0158] In one embodiment of the present invention, the first data processing module 501 is configured to:

[0159] A third local operation sequence number is generated based on the aforementioned local operation sequence number;

[0160] Generate a first read request; the first read request includes the third local operation sequence number and the first write data item corresponding to the first local operation sequence number;

[0161] The first read request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain the fourth local operation sequence number, and obtain the first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item;

[0162] Receive a response message sent by the first target node in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value;

[0163] Update the local operation sequence number according to the fourth local operation sequence number.

[0164] In one embodiment of the present invention, the node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; the second data processing module 502:

[0165] A second write request is generated; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value.

[0166] The second write request is sent to the second target node corresponding to the specified business domain; the second target node is used to obtain a first local operation sequence number generated based on the local operation sequence number from the specified node, and store the second write data and the unique identifier based on the first local operation sequence number;

[0167] Receive the response message returned by the second target node in response to the second write request.

[0168] In one embodiment of the present invention, the node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; the second data processing module 502:

[0169] Generate a second read request; the second read request includes the unique identifier and the second write data item;

[0170] The second read request is sent to the second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier;

[0171] Receive the response message returned by the second target node in response to the second read request; the response message includes the second write data value.

[0172] In one embodiment of the present invention, the node of the specified business domain is determined using a distributed hash mechanism.

[0173] In one embodiment of the present invention, the local operation sequence number and the local operation sequence number are monotonically increasing.

[0174] In this embodiment of the invention, the distributed storage system may include multiple consistency modes. When the consistency mode of the distributed storage system is global causal consistency, data is processed according to the locally generated local operation sequence number of the node. When the consistency mode of the distributed storage system is on-demand local linear consistency, if the node is not a node of a specified business domain, data can be processed according to the locally generated local operation sequence number; if the node is a node of a specified business domain, data can be processed according to the local operation sequence number generated by the specified node. The distributed storage system of this embodiment of the invention is designed with multiple consistency modes, and different consistency modes can be selected according to different storage needs, which can reduce the global resource overhead of the distributed storage system and improve the performance of the distributed storage system.

[0175] The above-described apparatus embodiments are basically similar to the method embodiments, so they are described in a relatively simple manner. For relevant details, please refer to the description of the method embodiments.

[0176] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0177] Memory 603 is used to store computer programs;

[0178] When the processor 601 executes the program stored in the memory 603, it implements the data processing method of the distributed storage system described in any of the above embodiments:

[0179] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0180] The communication interface is used for communication between the aforementioned terminal and other devices.

[0181] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0182] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0183] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the distributed storage system data processing methods described in the above embodiments.

[0184] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the distributed storage system data processing methods described in the above embodiments.

[0185] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk SSD), etc.

[0186] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0187] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0188] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A data processing method for a distributed storage system, characterized in that, The distributed storage system includes multiple nodes, and the method includes: When the consistency mode of the distributed storage system is global causal consistency mode, data is processed according to the local operation sequence number generated by the node. When the consistency mode of the distributed storage system is on-demand local linear consistency mode, if the node is a node outside the specified business domain, the data is processed according to the local operation sequence number generated locally by the node; if the node is a node within the specified business domain, the data is processed according to the local operation sequence number generated by the specified node. Each node maintains its own local operation sequence number; the data processing based on the locally generated local operation sequence number includes: Generate a first local operation sequence number based on the local operation sequence number; Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value; The first write request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain the second local operation sequence number, and store the first write data according to the second local operation sequence number; Receive a response message sent by the first target node in response to the first write request; the response message includes a second local operation sequence number; Update the local operation sequence number according to the second local operation sequence number.

2. The method according to claim 1, characterized in that, The process of processing data based on the local operation sequence number generated locally by the node includes: A third local operation sequence number is generated based on the aforementioned local operation sequence number; Generate a first read request; the first read request includes the third local operation sequence number and the first write data item corresponding to the first local operation sequence number; The first read request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the third local operation sequence number to obtain the fourth local operation sequence number, and obtain the first write data value corresponding to the maximum local operation sequence number corresponding to the first write data item; Receive a response message sent by the first target node in response to the first read request; the response message includes a fourth local operation sequence number and the first write data value; Update the local operation sequence number according to the fourth local operation sequence number.

3. The method according to claim 1, characterized in that, The node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; data is processed according to the local operation sequence number generated by the specified node, including: Generate a second write request; the second write request includes the unique identifier and the second write data; the second write data includes a second write data item and a second write data value; The second write request is sent to the second target node corresponding to the specified business domain; the second target node is used to obtain a first local operation sequence number generated based on the local operation sequence number from the specified node, and store the second write data and the unique identifier based on the first local operation sequence number; Receive the response message returned by the second target node in response to the second write request.

4. The method according to claim 2, characterized in that, The node corresponding to the specified business domain includes a unique identifier corresponding to the specified business domain; the specified node maintains a local operation sequence number; data is processed according to the local operation sequence number generated by the specified node, including: Generate a second read request; the second read request includes the unique identifier and the second write data item; The second read request is sent to the second target node; the second target node is used to obtain the second write data item and the second write data value corresponding to the maximum local operation sequence number corresponding to the unique identifier; Receive the response message returned by the second target node in response to the second read request; the response message includes the second write data value.

5. The method according to claim 1, characterized in that, The nodes of the specified business domain are determined using a distributed hash mechanism.

6. The method according to claim 1, characterized in that, The local operation sequence number and the local operation sequence number are monotonically increasing.

7. A data processing device for a distributed storage system, characterized in that, The distributed storage system includes multiple nodes, and the device includes: The first data processing module is used to process data according to the local operation sequence number generated locally by the node when the consistency mode of the distributed storage system is the global causal consistency mode. The second data processing module is used to process data according to the local operation sequence number generated locally by the node if the node is not a node of a specified business domain, and to process data according to the local operation sequence number generated by the specified node if the node is a node of a specified business domain. Each node maintains its own local operation sequence number; the first data processing module is used for: Generate a first local operation sequence number based on the local operation sequence number; Generate a first write request; the first write request includes the first local operation sequence number and the first write data corresponding to the first local operation sequence number; the first write data includes a first write data item and a first write data value; The first write request is sent to the first target node; the first target node is used to update the local operation sequence number of the first target node according to the first local operation sequence number to obtain the second local operation sequence number, and store the first write data according to the second local operation sequence number; Receive a response message sent by the first target node in response to the first write request; the response message includes a second local operation sequence number; Update the local operation sequence number according to the second local operation sequence number.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images