Data read / write methods, devices, electronic equipment, and storage media in storage systems

Abstract

This application discloses a data read / write method, apparatus, electronic device, and computer-readable storage medium in a storage system. The storage system includes a client, a master node, and multiple storage nodes. The client and each storage node are connected to the master node. The method is applied to the master node and includes: determining target storage nodes based on read / write control information sent by the client and generating data read / write requests corresponding to each target storage node; if it is a data read request, sending each data read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client; if it is a data write request, sending each data write request to the corresponding target storage node so that the target storage node obtains the second target data corresponding to the data write request from the client and saves it; receiving read / write completion messages from each target storage node and sending them back to the client. This solution can effectively improve the data read / write efficiency in the storage system.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data reading and writing method in a storage system, as well as a data reading and writing device, electronic device, and computer-readable storage medium in a storage system. Background Technology

[0002] Currently, the performance bottleneck of storage systems mainly lies in the network layer. To ensure data reliability, storage systems often employ redundancy strategies, with common technologies including Erasure Coding and replication. This means that when a customer writes data once, the backend storage system often needs to forward and write the data multiple times, which undoubtedly increases the burden on the network interface card (NIC) many times over.

[0003] In related technologies, storage systems based on erasure coding calculate the primary node and storage device to which data belongs after the client sends business data once. The primary node, upon receiving the entire data block via network transmission, uses its internal distribution algorithm and erasure coding calculations to determine the storage backup node to which each data block belongs, and then sends each data block to the corresponding storage node via the network interface card (NIC). This clearly leads to significant bandwidth waste: for the overall business data, the amount of data sent by the storage system via the NIC is divided into the initial business data sent to the primary node and the data blocks forwarded to each storage node, which obviously exceeds the amount of business data itself, resulting in wasted NIC resources.

[0004] Therefore, how to effectively solve the network card bottleneck problem in data read and write scenarios, reduce the latency consumption caused by network card forwarding, and improve data read and write efficiency is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a data read / write method in a storage system, which can effectively solve the network card bottleneck problem in data read / write scenarios, reduce the latency consumption caused by network card forwarding, and improve data read / write efficiency. Another purpose of this application is to provide a data read / write device, electronic device, computer-readable storage medium, and computer program product in a storage system, all of which have the above-mentioned beneficial effects.

[0006] In a first aspect, this application provides a data read / write method in a storage system, the storage system including a client, a master node, and multiple storage nodes, wherein the client and each of the storage nodes are connected to the master node, and the method is applied to the master node, including:

[0007] Receive read / write control information sent by the client, determine the target storage node based on the read / write control information, and generate data read / write requests corresponding to each target storage node;

[0008] When the data read / write request is a data read request, each data read request is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0009] When the data read / write request is a data write request, each data write request is sent to the corresponding target storage node so that the target storage node can obtain the second target data corresponding to the data write request from the client and save it.

[0010] The system receives read / write completion messages from each of the target storage nodes and sends these messages back to the client to complete the data read / write operation.

[0011] Optionally, when the data read / write request is the data read request, the method further includes:

[0012] When device anomaly information is detected, the data version logs of each of the storage nodes are obtained;

[0013] Determine the authoritative version log based on the various data version logs;

[0014] Generate a data rollback request based on the data version information in the authoritative version log;

[0015] The data rollback request is sent to each of the storage nodes so that each of the storage nodes can roll back the data according to the data version information.

[0016] Optionally, the authoritative version log is determined based on each of the aforementioned data version logs, including:

[0017] Determine the data version on the corresponding storage node based on the data version logs described above;

[0018] Determine the latest data version based on each of the aforementioned data versions;

[0019] The data version log to which the latest data version belongs is taken as the authoritative version log.

[0020] Optionally, when the data read / write request is a data read request, each data read request is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, including:

[0021] When the data read / write request is a data read request, each data read request is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

[0022] Optionally, each of the data read requests is sent to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, including:

[0023] Each data read request is sent to the corresponding target storage node via the transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client via the remote direct data access protocol.

[0024] Sending each of the aforementioned data write requests to the corresponding target storage node, so that the target storage node obtains and saves the second target data corresponding to the data write request from the client, including:

[0025] Each data write request is sent to the corresponding target storage node through the transmission control protocol, so that the target storage node obtains and saves the second target data corresponding to the data write request from the client through the remote direct data access protocol.

[0026] Accordingly, receiving read / write control information sent by the client includes: receiving the read / write control information sent by the client through the transmission control protocol;

[0027] Sending the read / write completion message to the client includes: sending the read / write completion message to the client via the transmission control protocol.

[0028] Optionally, the data read / write method in the storage system further includes:

[0029] When network anomalies are detected, the remote direct data access protocol is switched to the transmission control protocol.

[0030] Optionally, determining the target storage node based on the read / write control information and generating data read / write requests corresponding to each target storage node includes:

[0031] The target read / write object and its metadata information are determined based on the read / write control information.

[0032] The metadata information is calculated using a preset distribution algorithm to determine the target storage nodes and generate data read / write requests corresponding to each target storage node.

[0033] Secondly, this application also discloses a data read / write device in a storage system, the storage system including a client, a master node, and multiple storage nodes, wherein the client and each of the storage nodes are connected to the master node, and the device is applied to the master node, including:

[0034] The determination module is used to receive read / write control information sent by the client, determine the target storage node according to the read / write control information, and generate data read / write requests corresponding to each target storage node;

[0035] The reading module is used to send each data read request to the corresponding target storage node when the data read / write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0036] The writing module is used to send each of the data write requests to the corresponding target storage node when the data read / write request is a data write request, so that the target storage node can obtain the second target data corresponding to the data write request from the client and save it;

[0037] The feedback module is used to receive read / write completion messages from each of the target storage nodes and send the read / write completion messages back to the client to complete the data read / write operation.

[0038] Thirdly, this application also discloses an electronic device, comprising:

[0039] Memory, used to store computer programs;

[0040] A processor, configured to implement the steps of a data read / write method in any of the storage systems described above when executing the computer program.

[0041] Fourthly, this application also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data read / write method in any of the storage systems described above.

[0042] Fifthly, the present invention also discloses a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of any of the data read / write methods in a storage system as described above.

[0043] This application provides a data read / write method in a storage system. The storage system includes a client, a master node, and multiple storage nodes. The client and each of the storage nodes are connected to the master node. The method is applied to the master node and includes: receiving read / write control information sent by the client; determining target storage nodes based on the read / write control information; and generating data read / write requests corresponding to each target storage node; when the data read / write request is a read request, sending each read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the read request to the client; when the data read / write request is a write request, sending each write request to the corresponding target storage node so that the target storage node obtains the second target data corresponding to the write request from the client and saves it; receiving read / write completion messages from each target storage node and sending the read / write completion messages back to the client to complete the data read / write operation.

[0044] By applying the technical solution provided in this application, in a business scenario where data read and write operations are implemented in a storage system based on a client, a master node, and multiple storage nodes, the master node can determine the target storage node based on the client's read and write control information and forward the generated data read and write requests to each target storage node. This allows the target storage node to directly interact with the client to perform data read and write operations based on the data read and write requests. In other words, the actual business data is transmitted through direct data interaction between the client and the storage nodes, only by forwarding the control information about read and write operations between the client and the storage nodes through the master node. This effectively avoids secondary forwarding of actual business data through the master node. Therefore, this technical solution can effectively solve the network card bottleneck problem in data read and write scenarios, reduce the latency consumption caused by network card forwarding, and further improve the data read and write efficiency in the storage system.

[0045] In one embodiment of this application, real-time device anomaly monitoring is performed in a data read scenario. Once device anomaly information is detected, the authoritative version log is immediately filtered from the data version logs of all storage nodes, and the data version information in the authoritative version log is used to control each storage node to perform data rollback. This effectively ensures that all storage nodes in the storage system roll back to the same data version, effectively ensuring data consistency between storage nodes in the storage system, thereby ensuring the normal operation and stability of the storage system.

[0046] The data read / write device, electronic device, computer-readable storage medium, and computer program product in the storage system provided in this application also have the above-mentioned technical effects, and will not be described in detail here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the prior art and the embodiments of this application, the accompanying drawings used in the description of the prior art and the embodiments of this application will be briefly introduced below. Of course, the accompanying drawings described below with respect to the embodiments of this application are only a part of the embodiments in this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and such other drawings also fall within the protection scope of this application.

[0048] Figure 1 A schematic diagram of the structure of a storage system provided in this application;

[0049] Figure 2 A flowchart illustrating a data read / write method in a storage system provided in this application;

[0050] Figure 3 A flowchart illustrating a data write business scenario in a storage system provided in this application;

[0051] Figure 4 A flowchart illustrating a data read service scenario in a storage system provided in this application;

[0052] Figure 5 A schematic diagram of the structure of a data read / write device in a storage system provided in this application;

[0053] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0054] The core of this application is to provide a data read / write method in a storage system. This data read / write method in a storage system can effectively solve the network card bottleneck problem in data read / write scenarios, reduce the latency consumption caused by network card forwarding, and improve data read / write efficiency. Another core of this application is to provide a data read / write device, electronic device, computer-readable storage medium, and computer program product in a storage system, all of which have the above-mentioned beneficial effects.

[0055] To provide a clearer and more complete description of the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0056] This application provides a data read / write method in a storage system.

[0057] First, please refer to Figure 1 , Figure 1 This is a schematic diagram of a storage system provided in this application. The storage system includes a client, a master node, and multiple storage nodes, all of which are connected to the master node. The data read / write method provided in this application embodiment is implemented based on this storage system.

[0058] For further information, please refer to [link / reference]. Figure 2 , Figure 2 This application provides a flowchart illustrating a data read / write method in a storage system, which is applied to... Figure 1 The master node shown may include S101 to S104 as follows.

[0059] S101: Receive read / write control information sent by the client, determine the target storage node based on the read / write control information, and generate data read / write requests corresponding to each target storage node.

[0060] This step aims to determine the target storage node and generate corresponding data read / write requests. The target storage node is the storage node (read data node / write data node) corresponding to this read / write operation, and may be any one or more of all storage nodes. Specifically, when the client receives a read / write operation from the upper-layer module, it can send the read / write control information corresponding to that operation to the master node. The master node can then determine the target storage node based on this control information and generate data read / write requests for each target storage node. It can be understood that if the read / write operation is a data read operation, the data read / write request is specifically a data read request; if the read / write operation is a data write operation, the data read / write request is specifically a data write request.

[0061] In one embodiment of this application, determining the target storage node based on read / write control information and generating data read / write requests corresponding to each target storage node may include:

[0062] The target read / write object and its metadata information are determined based on the read / write control information.

[0063] The system uses a pre-defined distribution algorithm to calculate the metadata information, determine the target storage nodes, and generate data read / write requests for each target storage node.

[0064] This application provides a method for determining a target storage node. Specifically, the read / write control information sent by the client specifies the target object to be read / written (i.e., the aforementioned target data object) and its corresponding metadata, such as the memory address and data length of the target data object. Therefore, the master node can use a pre-set distribution algorithm to calculate the metadata information of each target data object, obtain the target storage node corresponding to each target data object, and generate corresponding data read / write requests. In one possible implementation, the aforementioned pre-set distribution algorithm can specifically be a consistent hashing algorithm.

[0065] In one embodiment of this application, after receiving the read / write control information sent by the client, determining the target storage node based on the read / write control information, and generating data read / write requests corresponding to each target storage node, the method further includes: inputting the data read / write requests into a first-in-first-out queue for temporary storage.

[0066] It is understandable that in a storage system, the number of data read and write operations is not unique, and multiple data read and write operations may exist simultaneously. Based on this, in order to avoid data IO conflicts caused by multiple data read and write operations running at the same time, this application proposes a request paving method based on a first-in-first-out queue to effectively ensure the sequential execution of each data read and write operation and further ensure the stable operation of the storage system.

[0067] S102: When the data read / write request is a data read request, each data read request is sent to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client.

[0068] This step aims to implement data reading services. Specifically, when a data read / write request is a read request, each read request is sent to its corresponding target storage node. The target storage node then responds to the read request, locates the target data (the aforementioned first target data) on its local hard drive, and communicates directly with the client to send the first target data back to the client. It's important to note that the first target data is sent directly from the target storage node to the client; this process bypasses the master node's data forwarding, significantly improving data reading efficiency.

[0069] In one embodiment of this application, when a data read / write request is a data read request, sending each data read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client may include: when a data read / write request is a data read request, sending each data read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client so that the client can verify all the first target data.

[0070] The data read / write method provided in this application can further implement data verification functionality in data read business scenarios to effectively ensure the correctness and consistency of the read data. Specifically, during the data read process, each target storage node will send back the first target data to the client according to its received data read request. Thus, the client can decode and verify the first target data sent back by all target storage nodes, and send back all the first target data to the upper-layer module when the verification passes. In one possible implementation, the client can utilize open-source erasure coding algorithms to implement the data verification function, effectively ensuring the consistency of each first target data.

[0071] In one embodiment of this application, after the above-mentioned sending each data read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, the method further includes: when the client does not receive the first target data fed back by all target storage nodes within a preset time period, clearing the local pre-allocated memory space and resending the read / write control information to the master node; at this time, the read / write control information is specifically read control information.

[0072] Specifically, the client can pre-set the duration of the data reading operation during the data reading process, i.e., the aforementioned preset duration. If the client cannot obtain the first target data returned by all target storage nodes within the preset duration, the data reading is considered a failure. Therefore, the client can clear the local pre-allocated memory space and re-initiate the data reading operation. The local pre-allocated memory space is the memory space pre-allocated by the client for this data reading operation, used to temporarily store the first target data returned by each target storage node. When the first target data is not successfully obtained, it can be cleared and re-allocated. It should be noted that the specific value of the aforementioned preset duration does not affect the implementation of this technical solution; it can be set by technical personnel according to the actual situation, and this application does not impose any limitations on it.

[0073] S103: When the data read / write request is a data write request, each data write request is sent to the corresponding target storage node so that the target storage node can obtain the second target data corresponding to the data write request from the client and save it.

[0074] This step aims to implement data writing services. Specifically, when a data read / write request is a write request, each write request can be sent to its corresponding target storage node. The target storage node can then respond to the write request, directly communicating with the client to retrieve the target write data corresponding to the write request (i.e., the second target data mentioned above), and save it to its local hard drive. It should be noted that the second target data is directly retrieved by the client and sent to the target storage node; this process does not involve data forwarding through the master node, significantly improving data writing efficiency.

[0075] S104: Receive read / write completion messages from each target storage node and send read / write completion messages back to the client to complete the data read / write operation.

[0076] This step aims to provide feedback on data read / write completion messages. Specifically, after each target storage node completes the data read or write request it has received, it can directly send a read / write completion message (read completion message or write completion message) to the master node. The master node can then forward this message to the client to inform the client that the data read / write operation has been completed.

[0077] Based on this, in the above-mentioned client-based verification process of the first target data, the client can perform the verification operation on each first target data after receiving the read completion message from the master node, ensuring that the client performs the verification operation after receiving all the first target data, thereby ensuring the integrity of the data and the accuracy of the verification operation.

[0078] In one embodiment of this application, sending each data read request to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client may include: sending each data read request to the corresponding target storage node through a transmission control protocol so that the target storage node sends the first target data corresponding to the data read request to the client through a remote direct data access protocol;

[0079] Sending each data write request to the corresponding target storage node so that the target storage node can obtain and save the second target data corresponding to the data write request from the client can include: sending each data write request to the corresponding target storage node through the transmission control protocol so that the target storage node can obtain and save the second target data corresponding to the data write request from the client through the remote direct data access protocol.

[0080] Accordingly, receiving read / write control information sent by the client may include: receiving read / write control information sent by the client via the Transmission Control Protocol;

[0081] Sending a read / write completion message to the client can include sending a read / write completion message to the client via the Transmission Control Protocol.

[0082] This application provides a data communication method between a client, a master node, and various storage nodes. As described above, by forwarding control information (including read / write control information and read / write completion messages) between the client and storage nodes solely through the master node, and achieving the transmission of actual business data (including first target data and second target data) through direct data interaction between the client and storage nodes, the secondary forwarding of actual business data through the master node can be effectively avoided. Based on this, this application proposes a hybrid data communication scheme using TCP (Transmission Control Protocol) + RDMA (Remote Direct Memory Access), whereby the master node transmits read / write control information and read / write completion messages via TCP, and the client transmits first target data and second target data with each storage node via RDMA.

[0083] Understandably, the RDMA protocol does not require scheduling by a central controller (CPU), which can reduce its computational burden and decrease the overall input / output latency. This is highly compatible with the scheme where read and write data are transmitted directly between the client and each storage node without being forwarded by the master node.

[0084] Furthermore, the data read / write method in this storage system may also include: switching the remote direct data access protocol to the transmission control protocol when network anomaly information is detected.

[0085] The data read / write method provided in this application embodiment can further implement the communication protocol switching function in network anomaly scenarios. Specifically, when a network anomaly is detected in the storage system, the RDMA protocol can be directly switched to the TCP protocol. This means that the TCP protocol is used to transmit read / write control information, read / write completion messages, first target data, and second target data. It is understood that the RDMA protocol has advantages such as not consuming CPU and high speed, but it still has drawbacks in handling network failures: because it needs to use its own registered memory, it is easy to exceed the registered size when transmission fails; if the registered space is too large, it will lead to resource waste. Therefore, when a network anomaly is detected, it can be switched back to TCP transmission mode, which can utilize the efficient transmission speed of RDMA while avoiding the problems of RDMA's one-sided operation when handling network anomaly scenarios.

[0086] As can be seen, the data read / write method in the storage system provided in this application embodiment, in a business scenario where data read / write in the storage system is implemented based on a client, a master node, and multiple storage nodes, allows the master node to determine the target storage node based on the client's read / write control information and forward the generated data read / write requests to each target storage node. This enables the target storage node to directly interact with the client to perform data read / write operations based on the data read / write requests. In other words, the actual business data is transmitted through direct data interaction between the client and the storage nodes, only by forwarding the control information about read / write business between the client and the storage nodes through the master node. This effectively avoids secondary forwarding of the actual business data through the master node. Therefore, this technical solution can effectively solve the network card bottleneck problem in data read / write scenarios, reduce the latency consumption caused by network card forwarding, and further improve the data read / write efficiency in the storage system.

[0087] Based on the above embodiments:

[0088] In one embodiment of this application, when the data read / write request is a data read request, the data read / write method in the storage system may further include:

[0089] When abnormal device information is detected, the data version logs of each storage node are retrieved;

[0090] Determine the authoritative version log based on the version logs of each data source;

[0091] Generate a data rollback request based on the data version information in the authoritative version log;

[0092] Send data rollback requests to each storage node so that each storage node can roll back the data according to the data version information.

[0093] The data read / write method provided in this application embodiment can further implement data rollback functionality in the event of device malfunction. It is understood that when device malfunctions occur in the storage system (such as storage node hard drive failure, server crash, etc.), data read / write failures are likely to occur. Therefore, to effectively ensure data consistency among storage nodes within the storage system, all storage nodes can be controlled to perform data rollback operations, ensuring that all storage nodes roll back to the same data version.

[0094] During implementation, real-time device anomaly monitoring can be performed on the storage system during data read and write processes. Once device anomaly information is detected, the data version logs on each storage node are obtained. These logs record the data version information of the corresponding storage node. Thus, the authoritative version log can be determined by referring to the data version information of each node, and the data version information in the authoritative version log can be used to control all storage nodes to perform data rollback, that is, rollback to the data version recorded in the authoritative version log.

[0095] Determining the authoritative version log based on each data version log can include:

[0096] Determine the data version on the corresponding storage node based on the data version logs;

[0097] Determine the latest data version based on each data version;

[0098] The data version log to which the latest data version belongs is regarded as the authoritative version log.

[0099] This application provides a method for selecting and determining the authoritative version log from numerous data version logs, which can be implemented by referring to the latest version idol principle. In other words, the data version log with the latest data version can be selected from all data version logs as the authoritative version log, ensuring data consistency while guaranteeing that the stored data is up-to-date.

[0100] Of course, the above-mentioned method of filtering authoritative version logs is only one implementation method provided by the embodiments of this application, and is not the only one. Other methods can also be used, such as the minimum coverage repair principle and the principle of discarding conflict logs.

[0101] As can be seen, in this embodiment of the application, real-time device anomaly monitoring is performed in the data read scenario. Once device anomaly information is detected, the authoritative version log is immediately filtered from the data version logs of all storage nodes, and the data version information in the authoritative version log is used to control each storage node to perform data rollback. This effectively ensures that all storage nodes in the storage system roll back to the same data version, effectively ensuring data consistency between storage nodes in the storage system, and thus ensuring the normal operation and stability of the storage system.

[0102] Based on the above embodiments, this application provides another method for reading and writing data in a storage system.

[0103] First, please refer to Figure 3 , Figure 3The flowchart provided in this application illustrates a data write operation scenario in a storage system. The implementation process is as follows: First, the erasure coding calculation, which should have been performed on the master node, is moved to the client. The client sends read / write control information (such as the target data object name and data size) to the master node. The master node calculates the target storage node where the corresponding data should be stored using a consistent hashing algorithm and sends a data write request to it. Second, upon receiving the data write request from the master node, each target storage node directly retrieves the first target data from the client via the RDMA protocol and commits the transaction according to the order of the request sent by the master node, persisting the first target data and the data version log to the hard disk device. It is understandable that since each target storage node receives the data write request in the same order, its disk write and transaction commit order are also consistent, ensuring the consistency of the data write timing on the storage nodes. Finally, after the target storage node completes the transaction, it sends a success message to the master node, causing the master node to reply to the client with a write completion message after receiving all completion messages. Among them, each storage node may experience write failures due to hard drive failures or service outages. Therefore, when dealing with write failure scenarios, we can also select the authoritative version log by comparing the data version logs of each storage node, and then roll back the stored data on each storage node to the same version to maintain data consistency.

[0104] For further information, please refer to [link / reference]. Figure 4 , Figure 4 The flowchart provided in this application illustrates a data read operation scenario in a storage system. The implementation process is as follows: First, after the client sends a data read request to the master node, it allocates memory of the corresponding length locally to store the data to be read from each storage node (the second target data), and maintains metadata such as request information corresponding to this memory data. Simultaneously, the master node distributes the corresponding data read requests to each target storage node. Second, after each target storage node reads the corresponding data from its hard drive, it directly pushes it to the client via the RDMA protocol, without merging and decoding it at the master node before forwarding it to the client. It also sends and aggregates the completion message to the master node. Finally, after receiving completion messages from all target storage nodes, the master node considers the request complete and replies to the client. The client decodes and verifies the received data, and after successful verification, feeds back the final result to the upper-level module. If an exception occurs during the client's data read process, such as the request not returning for a long time or returning a read error, the client will clear the allocated memory space and metadata and resend the read request.

[0105] Finally, because the implementation of data read / write operations places high demands on the processing order of storage nodes, the TCP protocol can be used in message exchange to ensure order and reliability. Specifically, the overall read / write operation interaction can use a hybrid approach of TCP + RDMA one-sided operation, that is, using the TCP protocol to transmit relevant control information for data read / write operations, and using the RDMA protocol to transmit the actual read / write data. Furthermore, during data read / write operations, the network status of the storage system can be monitored in real time, and if a network anomaly is detected, the RDMA protocol will be switched back to the TCP protocol.

[0106] Furthermore, an internal queue can be used to avoid out-of-order issues caused by RDMA one-sided operations. Specifically, since read / write data and control messages are transmitted separately, and RDMA one-sided operations may be slower than TCP control message transmission or experience transmission failures, a situation may arise where the data portion is not yet ready when a write operation is executed. Simply discarding the current operation or waiting for the client to resend would lead to out-of-order issues. Therefore, all business operations can be stored in an internal queue, and the dequeue of this business operation is triggered by the callback of the completed RDMA one-sided operation. In this way, a single RDMA one-sided operation will not affect the execution order of this business operation and other business operations.

[0107] As can be seen, the data read / write method in the storage system provided in this application embodiment, in a business scenario where data read / write in the storage system is implemented based on a client, a master node, and multiple storage nodes, allows the master node to determine the target storage node based on the client's read / write control information and forward the generated data read / write requests to each target storage node. This enables the target storage node to directly interact with the client to perform data read / write operations based on the data read / write requests. In other words, the actual business data is transmitted through direct data interaction between the client and the storage nodes, only by forwarding the control information about read / write business between the client and the storage nodes through the master node. This effectively avoids secondary forwarding of the actual business data through the master node. Therefore, this technical solution can effectively solve the network card bottleneck problem in data read / write scenarios, reduce the latency consumption caused by network card forwarding, and further improve the data read / write efficiency in the storage system.

[0108] This application provides a data read / write device for a storage system.

[0109] Please refer to Figure 5 , Figure 5 This application provides a schematic diagram of the structure of a data read / write device in a storage system. The storage system includes a client, a master node, and multiple storage nodes. The client and each storage node are connected to the master node. The data read / write device in this storage system is applied to the master node and may include:

[0110] Module 1 is used to receive read / write control information sent by the client, determine the target storage node based on the read / write control information, and generate data read / write requests corresponding to each target storage node.

[0111] Read module 2 is used to send each data read request to the corresponding target storage node when the data read / write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0112] The write module 3 is used to send each data write request to the corresponding target storage node when the data read / write request is a data write request, so that the target storage node can obtain the second target data corresponding to the data write request from the client and save it;

[0113] Feedback module 4 is used to receive read / write completion messages from each target storage node and send read / write completion messages back to the client to complete the data read / write operation.

[0114] As can be seen, the data read / write device in the storage system provided in this application embodiment, in a business scenario where data read / write is implemented based on a client, a master node, and multiple storage nodes, allows the master node to determine the target storage node based on the client's read / write control information and forward the generated data read / write requests to each target storage node. This enables the target storage node to directly interact with the client to perform data read / write operations based on the data read / write requests. In other words, the actual business data is transmitted through direct data interaction between the client and the storage nodes, only by forwarding the control information about read / write services between the client and the storage nodes through the master node. This effectively avoids secondary forwarding of actual business data through the master node. Therefore, this technical solution can effectively solve the network card bottleneck problem in data read / write scenarios, reduce the latency consumption caused by network card forwarding, and further improve the data read / write efficiency in the storage system.

[0115] In one embodiment of this application, the data read / write device in the storage system may further include an exception rollback module, which is used to obtain the data version logs of each storage node when the data read / write request is a data read request and when device exception information is detected; determine the authoritative version log based on each data version log; generate a data rollback request based on the data version information in the authoritative version log; and send the data rollback request to each storage node so that each storage node performs data rollback according to the data version information.

[0116] In one embodiment of this application, the above-mentioned exception rollback module can be specifically used to determine the data version on the corresponding storage node based on each data version log; determine the latest data version based on each data version; and use the data version log to which the latest data version belongs as the authoritative version log.

[0117] In one embodiment of this application, the above-mentioned reading module 2 can be specifically used to send each data read request to the corresponding target storage node when the data read / write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

[0118] In one embodiment of this application, the read module 2 can be specifically used to send each data read request to the corresponding target storage node through the transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client through the remote direct data access protocol; the write module 3 can be specifically used to send each data write request to the corresponding target storage node through the transmission control protocol, so that the target storage node obtains the second target data corresponding to the data write request from the client through the remote direct data access protocol and saves it.

[0119] Accordingly, the aforementioned determining module 1 can be specifically used to receive read / write control information sent by the client via the transmission control protocol; the aforementioned feedback module 4 can be specifically used to send a read / write completion message back to the client via the transmission control protocol.

[0120] In one embodiment of this application, the data read / write device in the storage system may further include an anomaly switching module, used to switch the remote direct data access protocol to the transmission control protocol when network anomaly information is detected.

[0121] In one embodiment of this application, the determination module 1 can be specifically used to determine the target read / write object and the metadata information of the target read / write object based on the read / write control information; calculate each metadata information using a preset distribution algorithm to determine each target storage node, and generate data read / write requests corresponding to each target storage node.

[0122] For a description of the apparatus provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0123] This application provides an electronic device.

[0124] Please refer to Figure 6 , Figure 6 This application provides a schematic diagram of the structure of an electronic device, which may include:

[0125] Memory 11 is used to store computer programs;

[0126] The processor 10 is configured to execute computer programs and implement the steps of data read / write methods as described above in any of the storage systems.

[0127] like Figure 6 The diagram shows the structural composition of an electronic device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0128] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0129] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the data read and write method in the storage system.

[0130] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0131] Receive read / write control information sent by the client, determine the target storage node based on the read / write control information, and generate data read / write requests corresponding to each target storage node;

[0132] When a data read / write request is a data read request, each data read request is sent to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client.

[0133] When a data read / write request is a data write request, each data write request is sent to the corresponding target storage node so that the target storage node can obtain the second target data corresponding to the data write request from the client and save it.

[0134] It receives read / write completion messages from each target storage node and sends these messages back to the client to complete the data read / write operation.

[0135] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0136] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0137] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0138] Of course, it should be noted that, Figure 6 The structure shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than Figure 6 More or fewer components as shown, or combinations of certain components.

[0139] This application provides a computer-readable storage medium.

[0140] The computer-readable storage medium provided in this application embodiment stores a computer program, and when the computer program is executed by a processor, it can implement the steps of the data read and write method in any of the above storage systems.

[0141] The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a server or data center that integrates one or more available media. For example, it can be any medium that can store computer program code, such as magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0142] For a description of the computer-readable storage medium provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0143] This application provides a computer program product.

[0144] The computer program product provided in this application includes a computer program / instruction. When the computer program / instruction is executed by a processor, it can implement the steps of any of the data read / write methods in the storage system described above.

[0145] Specifically, in the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0146] The computer program product may include one or more computer programs / instructions, which, when loaded and executed on a computer, can generate all or part of the processes or functions described in the embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0147] For a description of the computer program products provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0149] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0151] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A data read / write method in a storage system, characterized by, The storage system comprises a client, a master node and a plurality of storage nodes, the client and each of the storage nodes are connected to the master node, the method is applied to the master node, and comprises: receiving read-write control information sent by the client, determining target storage nodes according to the read-write control information, and generating data read-write requests corresponding to each of the target storage nodes; when the data read-write request is a data read request, sending each of the data read requests to the corresponding target storage node, so that the target storage node sends first target data corresponding to the data read request to the client; wherein, when the client does not receive the first target data sent by the target storage node within a preset time length, the client clears a local pre-application memory space and re-sends read control information to the master node; the local pre-application memory space is used to store the first target data; when the data read-write request is a data write request, sending each of the data write requests to the corresponding target storage node, so that the target storage node obtains second target data corresponding to the data write request from the client and saves the second target data; receiving read-write completion messages fed back by each of the target storage nodes, and feeding back the read-write completion messages to the client to complete the data read-write operation; wherein, when the read-write completion message is a read completion message, the client performs consistency check on the first target data fed back by each of the target storage nodes by using an error correction code calculation algorithm; when the data read-write request is the data read request, the method further comprises: when device exception information is monitored, obtaining data version logs of each of the storage nodes; determining data versions on the corresponding storage nodes according to each of the data version logs; determining a latest data version according to each of the data versions; taking a data version log to which the latest data version belongs as an authoritative version log; generating a data rollback request according to data version information in the authoritative version log; and sending the data rollback request to each of the storage nodes, so that each of the storage nodes performs data rollback according to the data version information; sending each of the data read requests to the corresponding target storage node, so that the target storage node sends first target data corresponding to the data read request to the client, comprises: sending each of the data read requests to the corresponding target storage node through a transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client through a remote direct data access protocol; sending each of the data write requests to the corresponding target storage node, so that the target storage node obtains second target data corresponding to the data write request from the client and saves the second target data, comprises: sending each of the data write requests to the corresponding target storage node through the transmission control protocol, so that the target storage node obtains the second target data corresponding to the data write request from the client through the remote direct data access protocol and saves the second target data; and Receiving the read-write control information sent by the client, comprising: receiving the read-write control information sent by the client through the transmission control protocol; Feeding back the read-write completion message to the client, comprising: feeding back the read-write completion message to the client through the transmission control protocol; When network exception information is monitored, switching the remote direct data access protocol to the transmission control protocol.

2. The data read / write method in the storage system according to claim 1, wherein, According to the read-write control information, determining target storage nodes, and generating data read-write requests corresponding to each target storage node, comprising: According to the read-write control information, determining target read-write objects and metadata information of the target read-write objects; Using a preset distribution algorithm to calculate each metadata information, determining each target storage node, and generating data read-write requests corresponding to each target storage node.

3. A data read / write apparatus in a storage system, characterized by comprising: The storage system comprises a client, a master node and a plurality of storage nodes, the client and each storage node are connected to the master node, and the device is applied to the master node, comprising: A determination module for receiving read-write control information sent by the client, determining target storage nodes according to the read-write control information, and generating data read-write requests corresponding to each target storage node; A reading module for sending each data read request to the corresponding target storage node when the data read-write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client; wherein, when the client does not receive the first target data sent by the target storage node within a preset time length, the client clears the local pre-application memory space and re-sends the read control information to the master node; the local pre-application memory space is used to store the first target data; A writing module for sending each data write request to the corresponding target storage node when the data read-write request is a data write request, so that the target storage node obtains the second target data corresponding to the data write request from the client and saves it; A feedback module for receiving read-write completion messages fed back by each target storage node, and feeding back the read-write completion messages to the client to complete the data read-write operation; wherein, when the read-write completion message is a read completion message, the client uses an erasure code calculation algorithm to perform consistency check on the first target data fed back by each target storage node; An exception rollback module for, when the data read-write request is the data read request, obtaining data version logs of each storage node when device exception information is monitored; determining data versions on the corresponding storage nodes according to each data version log; determining the latest data version according to each data version; taking the data version log to which the latest data version belongs as an authoritative version log; generating a data rollback request according to the data version information in the authoritative version log; and sending the data rollback request to each storage node, so that each storage node performs data rollback according to the data version information. The reading module is specifically configured to send each data read request to a corresponding target storage node through a transmission control protocol (TCP), so that the target storage node sends first target data corresponding to the data read request to the client through a remote direct memory access (RDMA) protocol. The writing module is specifically configured to send each data write request to a corresponding target storage node through the TCP, so that the target storage node acquires second target data corresponding to the data write request from the client through the RDMA protocol and saves the second target data. The determining module is specifically configured to receive the read-write control information sent by the client through the TCP. The feedback module is specifically configured to feed back the read-write completion message to the client through the TCP. The device further comprises an abnormal switching module configured to switch the RDMA protocol to the TCP when network abnormal information is monitored.

4. An electronic device, comprising: The device further comprises a memory configured to store a computer program. The processor is configured to execute the computer program to implement the steps of the data read-write method in the storage system according to claim 1 or 2. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data read-write method in the storage system according to claim 1 or 2.

5. A computer readable storage medium, characterized in that, ​

Images