Data read-write method and device in storage system, electronic equipment and storage medium

Abstract

The invention discloses a data read-write method and device in a storage system, electronic equipment and a computer readable storage medium, the storage system comprises a client, a main node and a plurality of storage nodes, the client and each storage node are connected to the main node, and the method is applied to the main node. Comprising the following steps: determining target storage nodes according to read-write control information sent by a client, and generating data read-write requests corresponding to the target storage nodes; if the data reading requests are data reading requests, sending the data reading requests to corresponding target storage nodes, so that the target storage nodes send first target data corresponding to the data reading requests to the client; if the data writing requests are data writing requests, sending the data writing requests to corresponding target storage nodes, so that the target storage nodes obtain second target data corresponding to the data writing requests from the client and store the second target data; and receiving a read-write completion message fed back by each target storage node and feeding back the read-write completion message to the client. According to the scheme, the data read-write efficiency in the storage system can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a method for reading and writing data in a storage system, and also to a data reading and writing device, an electronic device, and a computer-readable storage medium in a storage system. Background Art

[0002] At present, the performance bottleneck of storage systems mainly lies in the network layer. In order to ensure the reliability of data, storage systems often adopt certain redundancy strategies, and commonly used technologies include Erasure Code and replication. This means that when customers write data once, the backend storage system often needs to forward and write data multiple times, which undoubtedly increases the burden on the network card exponentially.

[0003] In the related technology, the storage system based on erasure code technology will calculate the master node and storage device to which the business data belongs after the client sends it once. After the master node receives the entire block of business data through network transmission, it uses the internal distribution algorithm and erasure code calculation to obtain the storage backup node to which each data block belongs, and then sends each data block to the corresponding storage node through the network card. Obviously, this brings a large waste of bandwidth: for the overall business data, the amount of data sent by the storage system through the network card is divided into the business data sent to the master node for the first time and the data blocks forwarded to each storage node for the second time. This is obviously more than the amount of business data itself, and there is a waste of network card resources.

[0004] Therefore, how to effectively solve the network card bottleneck problem in data reading and writing scenarios, reduce the delay consumption caused by network card forwarding, and improve data reading and writing efficiency is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The purpose of the present application is to provide a data reading and writing method in a storage system, which can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by the network card forwarding, and improve the data reading and writing efficiency; another purpose of the present application is to provide a data reading and writing device, electronic device, computer-readable storage medium and computer program product in a storage system, all of which have the above-mentioned beneficial effects.

[0006] In a first aspect, the present application provides a method for reading and writing data in a storage system, wherein the storage system includes a client, a master node, and multiple storage nodes, wherein the client and each of the storage nodes are connected to the master node, and the method is applied to the master node, including:

[0007] Receive the read / write control information sent by the client, determine the target storage node according to the read / write control information, and generate a data read / write request corresponding to each target storage node;

[0008] When the data read / write request is a data read request, each of the data read requests is sent to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0009] When the data read / write request is a data write request, each of the data write requests is sent to a corresponding target storage node, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client;

[0010] Receive the read / write completion message fed back by each of the target storage nodes, and feed back the read / write completion message to the client to complete the data read / write operation.

[0011] Optionally, when the data read and write request is the data read request, the method further includes:

[0012] When device abnormal information is detected, the data version log of each storage node is obtained;

[0013] Determine the authoritative version log according to each of the data version logs;

[0014] Generate a data rollback request according to the data version information in the authoritative version log;

[0015] The data rollback request is sent to each of the storage nodes, so that each of the storage nodes performs data rollback according to the data version information.

[0016] Optionally, determining the authoritative version log according to each of the data version logs includes:

[0017] Determine the data version on the corresponding storage node according to each of the data version logs;

[0018] Determine the latest data version according to each of the data versions;

[0019] The data version log to which the latest data version belongs is used as the authoritative version log.

[0020] Optionally, when the data read / write request is a data read request, sending each of the data read requests to a corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, comprising:

[0021] When the data read and write request is the data read request, each of the data read requests is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

[0022] Optionally, sending each of the data read requests to a corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client includes:

[0023] Sending each of the data read requests to a corresponding target storage node through a transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client through a remote direct data access protocol;

[0024] Sending each of the data write requests to a corresponding target storage node so that the target storage node obtains and saves the second target data corresponding to the data write request from the client, including:

[0025] Sending each of the data write requests to a corresponding target storage node through the transmission control protocol, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client through the remote direct data access protocol;

[0026] Correspondingly, receiving the read-write control information sent by the client includes: receiving the read-write control information sent by the client through the transmission control protocol;

[0027] Feeding back the read / write completion message to the client includes: feeding back the read / write completion message to the client via the transmission control protocol.

[0028] Optionally, the data reading and writing method in the storage system further includes:

[0029] When network abnormal information is monitored, the remote direct data access protocol is switched to the transmission control protocol.

[0030] Optionally, determining a target storage node according to the read / write control information, and generating a data read / write request corresponding to each target storage node, includes:

[0031] Determine a target read / write object and metadata information of the target read / write object according to the read / write control information;

[0032] The metadata information is calculated using a preset distribution algorithm to determine the target storage nodes, and generate data read and write requests corresponding to the target storage nodes.

[0033] In a second aspect, the present application further discloses a data reading and writing device in a storage system, the storage system comprising a client, a master node and a plurality of storage nodes, the client and each of the storage nodes are connected to the master node, the device is applied to the master node, and comprises:

[0034] A determination module, configured to receive the read-write control information sent by the client, determine the target storage node according to the read-write control information, and generate a data read-write request corresponding to each of the target storage nodes;

[0035] a reading module, configured to, when the data read / write request is a data read request, send each of the data read requests to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0036] A writing module, used for sending each data writing request to a corresponding target storage node when the data read / write request is a data writing request, so that the target storage node obtains and saves the second target data corresponding to the data writing request from the client;

[0037] A feedback module is used to receive the read and write completion message fed back by each of the target storage nodes, and feed back the read and write completion message to the client to complete the data read and write operation.

[0038] In a third aspect, the present application further discloses an electronic device, comprising:

[0039] Memory for storing computer programs;

[0040] A processor is used to implement the steps of any of the above-mentioned methods for reading and writing data in a storage system when executing the computer program.

[0041] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data reading and writing method in any of the storage systems described above are implemented.

[0042] In a fifth aspect, the present invention further discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the data reading and writing method in any of the storage systems described above.

[0043] The present application provides a data reading and writing method in a storage system, the storage system comprising a client, a master node and a plurality of storage nodes, the client and each of the storage nodes being connected to the master node, the method being applied to the master node, comprising: receiving read-write control information sent by the client, determining a target storage node according to the read-write control information, and generating data reading and writing requests corresponding to each of the target storage nodes; when the data reading and writing request is a data read request, sending each of the data read requests to a corresponding target storage node, so that the target storage node sends first target data corresponding to the data read request to the client; when the data reading and writing request is a data write request, sending each of the data write requests to a corresponding target storage node, so that the target storage node obtains and saves second target data corresponding to the data write request from the client; receiving a read-write completion message fed back by each of the target storage nodes, and feeding back the read-write completion message to the client, so as to complete the data reading and writing operation.

[0044] By applying the technical solution provided in the present application, in a business scenario in which data reading and writing in a storage system is implemented based on a client, a master node and multiple storage nodes, the master node can determine the target storage node based on the read and write control information of the client, and forward the generated data read and write request to each target storage node, so that the target storage node can directly interact with the client to implement data read and write operations according to the data read and write request. In other words, the control information about the read and write business between the client and the storage node is forwarded only through the master node, and the transmission of actual business data is realized through direct data interaction between the client and the storage node, which effectively avoids the secondary forwarding of the actual business data through the master node. Therefore, the present technical solution can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by the network card forwarding, and further improve the data reading and writing efficiency in the storage system.

[0045] In one embodiment of the present application, real-time device abnormality monitoring is performed in a data reading scenario. Once device abnormality information is monitored, the authoritative version log is immediately filtered from the data version logs of all storage nodes, and the data version information in the authoritative version log is used to control each storage node to perform data rollback, so as to effectively ensure that all storage nodes in the storage system are rolled back to the same data version, effectively ensuring data consistency between storage nodes in the storage system, and thus ensuring the normal operation and stability of the storage system.

[0046] The data reading and writing device, electronic device, computer-readable storage medium and computer program product in the storage system provided by this application also have the above-mentioned technical effects, which will not be described in detail in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0048] Figure 1 A schematic diagram of the structure of a storage system provided by this application;

[0049] Figure 2 A schematic diagram of a flow chart of a method for reading and writing data in a storage system provided by the present application;

[0050] Figure 3 A schematic diagram of a process flow in a data writing business scenario in a storage system provided by this application;

[0051] Figure 4 A schematic diagram of a process flow in a data reading business scenario in a storage system provided by this application;

[0052] Figure 5 A schematic diagram of the structure of a data reading and writing device in a storage system provided by the present application;

[0053] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0054] The core of the present application is to provide a data reading and writing method in a storage system, which can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by the network card forwarding, and improve the data reading and writing efficiency; another core of the present application is to provide a data reading and writing device, electronic device, computer-readable storage medium and computer program product in a storage system, all of which have the above-mentioned beneficial effects.

[0055] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] An embodiment of the present application provides a method for reading and writing data in a storage system.

[0057] First, please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a storage system provided by the present application, the storage system includes a client, a master node and multiple storage nodes, the client and each storage node are connected to the master node. The data reading and writing method provided in the embodiment of the present application is implemented based on the storage system.

[0058] For further information, please refer to Figure 2 , Figure 2 A schematic diagram of a data reading and writing method in a storage system provided in the present application, wherein the data reading and writing method in the storage system is applied to Figure 1 The main node shown may include the following S101 to S104.

[0059] S101: receiving read / write control information sent by a client, determining a target storage node according to the read / write control information, and generating a data read / write request corresponding to each target storage node.

[0060] This step is intended to determine the target storage node and generate the corresponding data read and write request, wherein the target storage node is the storage node (read data node / write data node) corresponding to the current read and write service, which may be any one or more of all storage nodes. Specifically, when the client receives the read and write service sent by the upper-level module, it can send the read and write control information corresponding to the read and write service to the master node, whereby the master node can determine the target storage node based on the read and write control information, and generate data read and write requests corresponding to each target storage node. It can be understood that if the read and write service is a data read service, the data read and write request is specifically a data read request; if the read and write service is a data write service, the data read and write request is specifically a data write request.

[0061] In one embodiment of the present application, determining the target storage node according to the read / write control information and generating the data read / write request corresponding to each target storage node may include:

[0062] Determine the target read / write object and metadata information of the target read / write object according to the read / write control information;

[0063] The preset distribution algorithm is used to calculate the metadata information, determine the target storage nodes, and generate data read and write requests corresponding to the target storage nodes.

[0064] An embodiment of the present application provides a method for determining a target storage node. Specifically, the read-write control information sent by the client specifies the target object that needs to be read and written (i.e., the above-mentioned target data object) and its corresponding metadata, such as the memory address of the target data object, the data length, etc., so that the master node can use a pre-set distribution algorithm to calculate the metadata information of each target data object, obtain the target storage node corresponding to each of the target data objects, and generate the corresponding data read and write request. In a possible implementation, the above-mentioned preset distribution algorithm can specifically be a consistent hashing algorithm.

[0065] In one embodiment of the present application, after receiving the read-write control information sent by the client, determining the target storage node according to the read-write control information, and generating data read-write requests corresponding to each target storage node, it also includes: inputting the data read-write request into a first-in-first-out queue for temporary storage.

[0066] It is understandable that in a storage system, the number of data read and write services is not unique, and there may be multiple data read and write services at the same time. Based on this, in order to avoid data IO conflicts caused by multiple data read and write services being carried out at the same time, the present application proposes a request temporary storage method based on a first-in-first-out queue to effectively ensure the order of each data read and write service, and further ensure the stable operation of the storage system.

[0067] S102: When the data read / write request is a data read request, each data read request is sent to a corresponding target storage node, so that the target storage node sends first target data corresponding to the data read request to the client.

[0068] This step is intended to implement data reading services. Specifically, when the data read / write request is a data read request, each data read request can be sent to its corresponding target storage node, whereby the target storage node can respond to the data read request, find the target read data corresponding to the data read request on the local hard disk, that is, the first target data mentioned above, and directly communicate data with the client to realize the feedback of the first target data to the client. It should be noted that the first target data is sent directly from the target storage node to the client, and this process does not forward data via the master node, which greatly improves the data reading efficiency.

[0069] In one embodiment of the present application, when the data read and write requests are data read requests, each data read request is sent to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, which may include: when the data read and write requests are data read requests, each data read request is sent to the corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

[0070] The data reading and writing method provided in the embodiment of the present application can further realize the read data verification function in the data reading business scenario to effectively ensure the correctness and consistency of the read data. Specifically, during the data reading process, each target storage node will feedback the first target data to the client according to the data reading request received by each node. Thus, the client can decode and verify the first target data fed back by all target storage nodes, and feed back all the first target data to the upper module when the verification passes. In one possible implementation method, the client can use an open source erasure code calculation algorithm to implement the data verification function to effectively ensure the consistency of each first target data.

[0071] In one embodiment of the present application, when the data read and write request is a data read request, each data read request is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, and also includes: when the client does not receive the first target data fed back by all target storage nodes within a preset time period, clearing the local pre-applied memory space and resending the read and write control information to the master node; at this time, the read and write control information is specifically read control information.

[0072] Specifically, during the data reading process, the client can pre-set the duration of the data reading service, that is, the above-mentioned preset duration. If the client cannot obtain the first target data fed back by all target storage nodes within the preset duration, it can be considered that the data reading has failed, so the local pre-applied memory space can be cleared and the data reading operation can be re-initiated. Among them, the local pre-applied memory space is the memory space allocated by the client in advance according to the data reading service, which is used to temporarily store the first target data fed back by each target storage node. When the first target data cannot be successfully obtained, it can be cleared and re-applied. It should be noted that the specific value of the above-mentioned preset duration does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual conditions, and this application does not limit this.

[0073] S103: When the data read / write request is a data write request, each data write request is sent to a corresponding target storage node, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client.

[0074] This step is intended to implement data writing services. Specifically, when the data read and write request is a data write request, each data write request can be sent to its corresponding target storage node, whereby the target storage node can respond to the data write request, directly communicate data with the client, and pull the target write data corresponding to the data write request from the client, i.e. the above-mentioned second target data, and save it to the local hard disk. It should be noted that the second target data is directly pulled by the client and sent to the target storage node. This process does not forward data via the master node, which greatly improves the data writing efficiency.

[0075] S104: receiving the read / write completion message fed back by each target storage node, and feeding back the read / write completion message to the client to complete the data read / write operation.

[0076] This step aims to realize the feedback of data read and write completion messages. Specifically, after each target storage node has executed the data read request or data write request it has received, it can directly feedback the read and write completion message (read completion message or write completion message) to the master node, and then the master node can continue to forward it to the client to inform the client that the data read and write operation has been completed.

[0077] On this basis, in the above-mentioned verification process of the first target data based on the client, the client can perform the verification operation on each first target data after receiving the read completion message fed back by the master node, thereby ensuring that the client performs the verification operation after receiving all the first target data, thereby ensuring the integrity of the data and the accuracy of the verification operation.

[0078] In one embodiment of the present application, sending each data read request to a corresponding target storage node so that the target storage node sends first target data corresponding to the data read request to the client may include: sending each data read request to the corresponding target storage node through a transmission control protocol so that the target storage node sends the first target data corresponding to the data read request to the client through a remote direct data access protocol;

[0079] Sending each data write request to a corresponding target storage node so that the target storage node obtains and stores the second target data corresponding to the data write request from the client, may include: sending each data write request to the corresponding target storage node through a transmission control protocol so that the target storage node obtains and stores the second target data corresponding to the data write request from the client through a remote direct data access protocol;

[0080] Accordingly, receiving the read-write control information sent by the client may include: receiving the read-write control information sent by the client through a transmission control protocol;

[0081] Feeding back the read / write completion message to the client may include: feeding back the read / write completion message to the client via a transmission control protocol.

[0082] The embodiment of the present application provides a data communication method between a client, a master node, and each storage node. As described above, the control information about the read and write business between the client and the storage node is forwarded only through the master node (here including the read and write control information and the read and write completion message), and the actual business data (here including the first target data and the second target data) is transmitted through direct data interaction between the client and the storage node, which can effectively avoid the secondary forwarding of the actual business data through the master node. On this basis, the embodiment of the present application proposes a data communication scheme of TCP (Transmission Control Protocol) + RDMA (Remote Direct Memory Access), that is, the master node transmits the read and write control information and the read and write completion message through the TCP protocol, and the first target data and the second target data are transmitted between the client and each storage node through the RDMA protocol.

[0083] It is understandable that the RDMA protocol does not need to be scheduled by the central controller (CPU), which can reduce its computing pressure and reduce the overall input and output latency. This is highly consistent with the solution in which read and write data is transmitted directly between the client and each storage node without being forwarded by the master node.

[0084] Furthermore, the data reading and writing method in the storage system may also include: when abnormal network information is monitored, switching the remote direct data access protocol to the transmission control protocol.

[0085] The data reading and writing method provided in the embodiment of the present application can further realize the communication protocol switching function in the network abnormality scenario, that is, when the network abnormality problem is detected in the storage system, the RDMA protocol can be directly switched to the TCP protocol, that is, the TCP protocol is used to realize the transmission of read and write control information, read and write completion messages, first target data, and second target data. It can be understood that the RDMA protocol has the advantages of not consuming CPU and fast speed, but it still has disadvantages in dealing with network failures: because it needs to use its own registered memory, it is easy to exceed the registered size when the transmission fails; if the registered space is too large, it will lead to waste of resources. Therefore, when a network abnormality is detected, it can be switched back to the TCP transmission mode, which can not only take advantage of the efficient transmission speed of RDMA, but also avoid the problem of RDMA unilateral operation when dealing with network abnormality scenarios.

[0086] It can be seen that the data reading and writing method in the storage system provided by the embodiment of the present application, in a business scenario in which data reading and writing in the storage system is implemented based on a client, a master node and multiple storage nodes, the master node can determine the target storage node based on the client's read and write control information, and forward the generated data read and write request to each target storage node, so that the target storage node can directly interact with the client to implement data reading and writing operations according to the data read and write request. In other words, the control information about the read and write business between the client and the storage node is forwarded only through the master node, and the transmission of actual business data is realized through direct data interaction between the client and the storage node, which effectively avoids the secondary forwarding of actual business data through the master node. Therefore, the technical solution can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by network card forwarding, and further improve the data reading and writing efficiency in the storage system.

[0087] Based on the above embodiments:

[0088] In one embodiment of the present application, when the data read / write request is a data read request, the data read / write method in the storage system may further include:

[0089] When abnormal device information is detected, obtain the data version log of each storage node;

[0090] Determine the authoritative version log based on each data version log;

[0091] Generate a data rollback request based on the data version information in the authoritative version log;

[0092] Send a data rollback request to each storage node so that each storage node can roll back the data according to the data version information.

[0093] The data reading and writing method provided in the embodiment of the present application can further realize the data rollback function in the device abnormality scenario. It is understandable that when a device abnormality occurs in the storage system (such as a storage node hard disk failure, server downtime, etc.), it is likely to cause data reading and writing failure. Based on this, in order to effectively ensure the data consistency between the storage nodes in the storage system, all storage nodes can be controlled to perform data rollback operations to ensure that all storage nodes are rolled back to the same data version.

[0094] During the implementation process, the storage system can be monitored for real-time device anomalies during data reading and writing. Once device anomaly information is monitored, the data version log on each storage node is obtained. The data version log records the data version information on the corresponding storage node. Therefore, the authoritative version log can be determined by referring to the data version information, and the data version information in the authoritative version log can be used to control all storage nodes to perform data rollback, that is, roll back to the data version recorded in the authoritative version log.

[0095] Wherein, determining the authoritative version log according to each data version log may include:

[0096] Determine the data version on the corresponding storage node based on each data version log;

[0097] Determine the latest data version based on each data version;

[0098] The data version log to which the latest data version belongs is used as the authoritative version log.

[0099] The embodiment of the present application provides a method for selecting and determining an authoritative version log from among numerous data version logs, which can be implemented by referring to the latest version idol principle. In other words, the data version log with the latest data version can be selected from all data version logs as the authoritative version log, ensuring data consistency while ensuring that the stored data is the latest data.

[0100] Of course, the above-mentioned method of screening the authoritative version log is only one implementation method provided in the embodiment of the present application, and is not the only one. Other methods may also be used to implement it, such as the minimum coverage repair principle, the conflict log discarding principle, etc.

[0101] It can be seen that in the embodiment of the present application, real-time device abnormality monitoring is performed in the data reading scenario. Once device abnormality information is monitored, the authoritative version log is immediately filtered out from the data version logs of all storage nodes, and the data version information in the authoritative version log is used to control each storage node to perform data rollback, so as to effectively ensure that all storage nodes in the storage system are rolled back to the same data version, effectively ensuring the data consistency between the storage nodes in the storage system, and thus ensuring the normal and stability of the storage system.

[0102] Based on the above embodiments, an embodiment of the present application provides another method for reading and writing data in a storage system.

[0103] First, please refer to Figure 3 , Figure 3A flow chart of a data write business scenario in a storage system provided by the present application, and its implementation process is as follows: First, the erasure code calculation that should be performed on the master node is pulled up to the client, that is, the client sends the read-write control information (such as the name of the target data object, data size, etc.) to the master node, and the master node calculates the target storage node where the corresponding data should be saved through the consistent hashing algorithm and sends a data write request to it. Secondly, after receiving the data write request sent by the master node, each target storage node directly pulls the first target data from the client through the RDMA protocol, and submits the transaction in the order of the request sent by the master node, and persists the first target data and the data version log in the hard disk device. It can be understood that since the order in which each target storage node receives the data write request is the same, the order in which it falls to the disk and the transaction is submitted is also consistent, which ensures the consistency of the data write timing on the storage node. Finally, when the target storage node completes the transaction, it sends a successful completion message to the master node, so that the master node replies to the client that the write is completed after receiving all completion messages. Among them, each storage node may have write failure problems caused by hard disk failure or service downtime. Therefore, when dealing with write failure scenarios, you can also select the authoritative version log by comparing the data version logs of each storage node, and then roll back the storage data on each storage node to the same version to maintain data consistency.

[0104] For further information, please refer to Figure 4 , Figure 4 A flow chart of a data reading business scenario in a storage system provided by the present application, the implementation process is as follows: First, after the client sends a data read request to the master node, it applies for a memory of the corresponding length locally to save the data (second target data) to be read from each storage node, and maintains the request information and other metadata corresponding to this memory data, and the master node sends the corresponding data read request to each target storage node respectively. Secondly, after each target storage node reads the corresponding data in the hard disk device, it directly pushes it to the client through the RDMA protocol, and no longer forwards it to the client after merging and decoding in the master node, and sends and summarizes the completed message to the master node. Finally, after the master node receives the completion message fed back by all target storage nodes, it considers that the request has been completed and replies to the client, and the client decodes and verifies the received data, and feeds back the final result to the upper-level module after the verification passes. Among them, if an exception occurs during the client reading data, for example, the request has not been returned for a long time or a read error is returned, the client will clean up the corresponding memory space and metadata applied for, and resend the read request.

[0105] Finally, since the implementation of data read and write services has high requirements for the service processing order of storage nodes, the TCP protocol can be used in message interaction to ensure order and reliability. Specifically, the overall read and write service interaction can use the TCP+RDMA unilateral operation mixed processing method, that is, the TCP protocol is used to transmit the relevant control information of the data read and write operation, and the RDMA protocol is used to transmit the actual read and write data of the data read and write operation. In addition, during the data reading and writing process, the network status of the storage system can also be monitored in real time. Once a network abnormality is detected, the RDMA protocol is switched back to the TCP protocol.

[0106] In addition, the internal queue can be used to avoid the disorder problem caused by RDMA unilateral operation. Specifically, since the read and write data are transmitted separately from the control message, and the RDMA unilateral operation may be slower than the TCP control message transmission or a certain transmission failure, when a certain write business is executed, the data part is not ready yet. If the current operation is simply discarded or waiting for the client to resend, it will cause problems such as disordered business. Therefore, all businesses can be saved in the internal queue, and the callback completed by the RDMA unilateral operation triggers the dequeue of this business. Therefore, a single RMDA unilateral operation will not affect the execution order of this business and other businesses.

[0107] It can be seen that the data reading and writing method in the storage system provided by the embodiment of the present application, in a business scenario in which data reading and writing in the storage system is implemented based on a client, a master node and multiple storage nodes, the master node can determine the target storage node based on the client's read and write control information, and forward the generated data read and write request to each target storage node, so that the target storage node can directly interact with the client to implement data reading and writing operations according to the data read and write request. In other words, the control information about the read and write business between the client and the storage node is forwarded only through the master node, and the transmission of actual business data is realized through direct data interaction between the client and the storage node, which effectively avoids the secondary forwarding of actual business data through the master node. Therefore, the technical solution can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by network card forwarding, and further improve the data reading and writing efficiency in the storage system.

[0108] An embodiment of the present application provides a data reading and writing device in a storage system.

[0109] Please refer to Figure 5 , Figure 5 This is a structural schematic diagram of a data reading and writing device in a storage system provided by the present application. The storage system includes a client, a master node and multiple storage nodes. The client and each storage node are connected to the master node. The data reading and writing device in the storage system is applied to the master node and may include:

[0110] Determination module 1, used to receive read-write control information sent by the client, determine the target storage node according to the read-write control information, and generate data read-write requests corresponding to each target storage node;

[0111] Reading module 2, used for sending each data read request to a corresponding target storage node when the data read / write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0112] A writing module 3, used for sending each data writing request to a corresponding target storage node when the data read / write request is a data writing request, so that the target storage node obtains and stores the second target data corresponding to the data writing request from the client;

[0113] The feedback module 4 is used to receive the read / write completion message fed back by each target storage node, and feed back the read / write completion message to the client to complete the data read / write operation.

[0114] It can be seen that the data reading and writing device in the storage system provided by the embodiment of the present application, in a business scenario in which data reading and writing in the storage system is implemented based on a client, a master node and multiple storage nodes, the master node can determine the target storage node according to the client's read and write control information, and forward the generated data read and write request to each target storage node, so that the target storage node can directly interact with the client to implement data reading and writing operations according to the data read and write request. In other words, the control information about the read and write business between the client and the storage node is forwarded only through the master node, and the transmission of actual business data is realized through direct data interaction between the client and the storage node, which effectively avoids the secondary forwarding of actual business data through the master node. Therefore, the technical solution can effectively solve the network card bottleneck problem in the data reading and writing scenario, reduce the delay consumption caused by network card forwarding, and further improve the data reading and writing efficiency in the storage system.

[0115] In one embodiment of the present application, the data reading and writing device in the storage system may also include an abnormal rollback module, which is used to obtain the data version log of each storage node when the data reading and writing request is a data read request and when device abnormal information is monitored; determine the authoritative version log according to each data version log; generate a data rollback request according to the data version information in the authoritative version log; and send the data rollback request to each storage node so that each storage node rolls back the data according to the data version information.

[0116] In one embodiment of the present application, the above-mentioned abnormal rollback module can be specifically used to determine the data version on the corresponding storage node according to each data version log; determine the latest data version according to each data version; and use the data version log to which the latest data version belongs as the authoritative version log.

[0117] In one embodiment of the present application, the above-mentioned reading module 2 can be specifically used to send each data read request to the corresponding target storage node when the data read and write request is a data read request, so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

[0118] In one embodiment of the present application, the reading module 2 may be specifically used to send each data read request to the corresponding target storage node through the transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client through the remote direct data access protocol; the writing module 3 may be specifically used to send each data write request to the corresponding target storage node through the transmission control protocol, so that the target storage node obtains and saves the second target data corresponding to the data write request from the client through the remote direct data access protocol;

[0119] Accordingly, the determination module 1 may be specifically used to receive read / write control information sent by the client via the transmission control protocol; the feedback module 4 may be specifically used to feedback a read / write completion message to the client via the transmission control protocol.

[0120] In one embodiment of the present application, the data reading and writing device in the storage system may further include an abnormal switching module, which is used to switch the remote direct data access protocol to the transmission control protocol when network abnormal information is monitored.

[0121] In one embodiment of the present application, the above-mentioned determination module 1 can be specifically used to determine the target read-write object and the metadata information of the target read-write object based on the read-write control information; use a preset distribution algorithm to calculate each metadata information, determine each target storage node, and generate a data read-write request corresponding to each target storage node.

[0122] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0123] An embodiment of the present application provides an electronic device.

[0124] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:

[0125] A memory 11, used for storing computer programs;

[0126] The processor 10 can implement the steps of any of the above-mentioned methods for reading and writing data in the storage system when executing a computer program.

[0127] like Figure 6 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0128] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0129] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of a method for reading and writing data in a storage system.

[0130] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0131] Receive the read-write control information sent by the client, determine the target storage node according to the read-write control information, and generate data read-write requests corresponding to each target storage node;

[0132] When the data read / write request is a data read request, each data read request is sent to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client;

[0133] When the data read / write request is a data write request, each data write request is sent to a corresponding target storage node, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client;

[0134] Receive the read and write completion message fed back by each target storage node, and feed back the read and write completion message to the client to complete the data read and write operation.

[0135] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0136] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0137] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0138] Of course, it should be noted that Figure 6 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0139] An embodiment of the present application provides a computer-readable storage medium.

[0140] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of the data reading and writing method in any of the above-mentioned storage systems can be implemented.

[0141] Among them, the computer-readable storage medium can be any available medium that can be stored in a computer or a data storage device such as a server or data center that includes one or more available media integrated. For example, it can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD) or a semiconductor medium (such as a solid-state hard disk) and other media that can store computer program codes.

[0142] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0143] An embodiment of the present application provides a computer program product.

[0144] The computer program product provided in the embodiments of the present application includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of any of the above-mentioned methods for reading and writing data in a storage system can be implemented.

[0145] Specifically, in the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of them can be implemented in the form of a computer program product.

[0146] Among them, the computer program product may include one or more computer programs / instructions, and when the computer program / instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application may be generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0147] For an introduction to the computer program product provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0148] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0149] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0151] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A method for reading and writing data in a storage system, characterized in that: The storage system includes a client, a master node and a plurality of storage nodes, the client and each of the storage nodes are connected to the master node, and the method is applied to the master node, including: Receive the read / write control information sent by the client, determine the target storage node according to the read / write control information, and generate a data read / write request corresponding to each target storage node; When the data read / write request is a data read request, each of the data read requests is sent to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client; When the data read / write request is a data write request, each of the data write requests is sent to a corresponding target storage node, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client; Receive the read / write completion message fed back by each of the target storage nodes, and feed back the read / write completion message to the client to complete the data read / write operation.

2. The data reading and writing method in the storage system according to claim 1, characterized in that: When the data read and write request is the data read request, the method further includes: When device abnormal information is detected, the data version log of each storage node is obtained; Determine the authoritative version log according to each of the data version logs; Generate a data rollback request according to the data version information in the authoritative version log; The data rollback request is sent to each of the storage nodes, so that each of the storage nodes performs data rollback according to the data version information.

3. The data reading and writing method in the storage system according to claim 2, characterized in that: Determining the authoritative version log according to each of the data version logs includes: Determine the data version on the corresponding storage node according to each of the data version logs; Determine the latest data version according to each of the data versions; The data version log to which the latest data version belongs is used as the authoritative version log.

4. The data reading and writing method in the storage system according to claim 1, characterized in that: When the data read / write request is a data read request, sending each of the data read requests to a corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, including: When the data read and write request is the data read request, each of the data read requests is sent to the corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client, so that the client can verify all the first target data.

5. The data reading and writing method in the storage system according to claim 1, characterized in that: Sending each of the data read requests to a corresponding target storage node so that the target storage node sends the first target data corresponding to the data read request to the client, comprising: Sending each of the data read requests to a corresponding target storage node through a transmission control protocol, so that the target storage node sends the first target data corresponding to the data read request to the client through a remote direct data access protocol; Sending each of the data write requests to a corresponding target storage node so that the target storage node obtains and saves the second target data corresponding to the data write request from the client, including: Sending each of the data write requests to a corresponding target storage node through the transmission control protocol, so that the target storage node obtains and stores the second target data corresponding to the data write request from the client through the remote direct data access protocol; Correspondingly, receiving the read-write control information sent by the client includes: receiving the read-write control information sent by the client through the transmission control protocol; Feeding back the read / write completion message to the client includes: feeding back the read / write completion message to the client via the transmission control protocol.

6. The data reading and writing method in the storage system according to claim 5, characterized in that: Also includes: When network abnormal information is monitored, the remote direct data access protocol is switched to the transmission control protocol.

7. The data reading and writing method in a storage system according to claim 1, characterized in that: Determining the target storage node according to the read / write control information, and generating a data read / write request corresponding to each of the target storage nodes, including: Determine a target read / write object and metadata information of the target read / write object according to the read / write control information; The metadata information is calculated using a preset distribution algorithm to determine the target storage nodes, and generate data read and write requests corresponding to the target storage nodes.

8. A data reading and writing device in a storage system, characterized in that: The storage system includes a client, a master node and a plurality of storage nodes, the client and each of the storage nodes are connected to the master node, and the device is applied to the master node, including: A determination module, configured to receive the read-write control information sent by the client, determine the target storage node according to the read-write control information, and generate a data read-write request corresponding to each of the target storage nodes; a reading module, configured to, when the data read / write request is a data read request, send each of the data read requests to a corresponding target storage node, so that the target storage node sends the first target data corresponding to the data read request to the client; A writing module, used for sending each data writing request to a corresponding target storage node when the data read / write request is a data writing request, so that the target storage node obtains and saves the second target data corresponding to the data writing request from the client; A feedback module is used to receive the read and write completion message fed back by each of the target storage nodes, and feed back the read and write completion message to the client to complete the data read and write operation.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for reading and writing data in a storage system as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for reading and writing data in a storage system according to any one of claims 1 to 7 are implemented.

Images