Data storage method and device, NVM subsystem and host

Abstract

The data storage method comprises the steps that an NVM subsystem receives a request message sent by a host, the request message is used for requesting the NVM subsystem to receive service data sent by the host through a first port through a second port, and the request message comprises communication parameters of the first port; and when the communication parameter of the first port is inconsistent with the communication parameter of the downlink equipment port, the NVM subsystem sends a first response message to the host, the downlink equipment port comprises a second port, and the first response message indicates that the NVM subsystem refuses to receive the service data through the second port. According to the method, when the host communicates with the NVM subsystem through the NVMe over Fabr ic, service instability caused by inconsistency of communication parameters of the host and communication parameters of the NVM subsystem is prevented.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a data storage method, device, NVM subsystem and host. Background Art

[0002] Non-volatile memory express (NVMe) is an interface bus specification for communication between a host and a non-volatile memory (NVM) subsystem. NVMe is a high-speed specification that enables the host to access the NVM subsystem with high performance. Among them, NVMe (NVMe over Fabric) based on a switching network enables the host to remotely access the NVM subsystem with high performance.

[0003] The communication parameters of the host and the NVM subsystem may be inconsistent due to configuration operation errors, configuration file errors, insufficient configuration personnel experience, etc. When the host and the NVM subsystem transmit service data through NVMe over Fabric, if the communication parameters of the host and the NVM subsystem are inconsistent, the network performance will be degraded, which will lead to instability of the host service. Summary of the invention

[0004] Provided are a data storage method, device, NVM subsystem and host, which can prevent business instability caused by inconsistency between communication parameters of the host and communication parameters of the NVM subsystem.

[0005] In a first aspect, a data storage method is provided, which is applied to a non-volatile memory NVM subsystem that communicates with a host through NVMe over Fabric, the host having a first port, and the NVM subsystem having a second port; the method comprises: the NVM subsystem receives a request message sent by the host, the request message is used to request the NVM subsystem to receive business data sent by the host through the first port through the second port, wherein the request message includes communication parameters of the first port; when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the NVM subsystem sends a first response message to the host, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive business data through the second port.

[0006] NVMe over Fabric enables the host to remotely access the NVM subsystem with high performance. Among them, high-performance remote access depends on the consistency of communication parameters between the host and the NVM subsystem. In other words, if the communication parameters of the host are consistent with the communication parameters of the NVM subsystem, NVMe over Fabric enables the host to remotely access the NVM subsystem with high performance. If the communication parameters of the host and the communication parameters of the NVM subsystem are inconsistent, the network performance of the host accessing the NVM subsystem through NVMe over Fabric will be reduced, resulting in unstable host business. Configuration errors (configuration operation errors, configuration file errors, insufficient configuration personnel experience, etc.), as well as configuration changes of the host or NVM subsystem during operation, may cause the communication parameters of the host and the communication parameters of the NVM subsystem to be inconsistent.

[0007] In the related art, when the host needs to store business data to the NVM subsystem, it does not consider whether the communication parameters of the host and the communication parameters of the NVM subsystem are consistent. In other words, in the related art, when the host and the NVM subsystem start the business data transmission based on NVMe over Fabric, they do not perceive whether the communication parameters of the host and the NVM subsystem are consistent, or ignore the inconsistency between the host and the NVM subsystem. Therefore, in the related art, regardless of whether the communication parameters of the host and the communication parameters of the NVM subsystem are consistent, the host sends business data to the NVM subsystem through NVMe over Fabric, which is prone to business instability caused by the inconsistency between the communication parameters of the host and the communication parameters of the NVM subsystem.

[0008] In the data storage method provided in the embodiment of the present application, when the host starts to run a service that needs to store business data to the NVM subsystem, the host sends a request message to the NVM subsystem. The request message includes the communication parameters of the first port used by the host to send business data, and the request message indicates the downstream device port through which the host sends business data to the NVM subsystem, for example, the request message indicates the second port used by the NVM subsystem to receive business data sent by the host. In this way, when the NVM subsystem receives the request message, it can obtain the communication parameters of the first port based on the request message, and determine whether the communication parameters of the first port are consistent with the communication parameters of the downstream device port. For example, the communication parameters of the second port are obtained based on the request message, and it is determined whether the communication parameters of the first port are consistent with the communication parameters of the second port.

[0009] When the NVM subsystem confirms that the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the NVM subsystem can send a rejection message (i.e., a first response message) to the host to notify the host that the NVM subsystem refuses to receive service data through the second port. In this way, when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the host no longer sends service data to the NVM subsystem through the first port using NVMe over Fabric, thereby preventing service instability caused by the inconsistency between the communication parameters of the host and the communication parameters of the NVM subsystem.

[0010] Furthermore, through the rejection message, the host can perceive that the communication parameters of the sending port used for the host to send business data are inconsistent with the communication parameters of the downstream device port, thereby triggering the host to take measures or other communication methods for ensuring the consistency of communication parameters, so as to ensure the stability of the business while the business is carried out. For example, the rejection message can trigger the host to provide prompt information, which prompts the user to reconfigure the communication parameters of the first port and / or the downstream device port so that the communication parameters of the first port are consistent with the communication parameters of the downstream device port. For another example, the rejection message can trigger the host to try to send business data through a port other than the first port, or request the NVM subsystem to receive business data through a port other than the second port. For another example, the rejection message can trigger the host to adopt a communication method other than NVMe over Fabric that is not affected by inconsistent communication parameters to send business data to the NVM subsystem. And so on.

[0011] In short, through the data storage method provided in the embodiment of the present application, when the host and the NVM subsystem use NVMe over Fabric to communicate with the NVM subsystem, the host can sense whether the stability of the business can be guaranteed by transmitting business data between the host and the NVM subsystem through the first port and the second port. This can avoid the situation where the stability of the business cannot be guaranteed by transmitting business data between the host and the NVM subsystem through the first port and the second port, and still transmit business data through the first port and the second port, thereby preventing business instability.

[0012] In a possible implementation, the method further includes: when the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem sends a second response message to the host, and the second response message indicates that the NVM subsystem agrees to receive the service data through the second port.

[0013] In this implementation, when the NVM subsystem confirms that the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem can send an agreement message (i.e., a second response message) to the host to notify the host that the NVM subsystem agrees to receive business data through the second port. In this way, the host can use NVMe over Fabric to send business data to the NVM subsystem through the first port, and the NVM subsystem receives and stores the business data through the second port. The communication parameters of the first port and the communication parameters of the second port are consistent, so that the performance of NVMe over Fabric can be fully utilized, thereby ensuring that business data can be transmitted at high speed and stably, and then the business can be carried out stably.

[0014] In one possible implementation, the request message includes an update indication, and the method further includes: when the communication parameters of the first port and the communication parameters of the second port are inconsistent, the NVM subsystem responds to the update indication and updates the communication parameters of the second port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0015] In this implementation, an update indication may be configured in the request message, and the update indication is used to instruct the NVM subsystem to update the communication parameters of the receiving port when the sending port of the business data is inconsistent with the receiving port of the business data. Thus, when the NVM subsystem confirms that the first port (i.e., the sending port of the business data) and the second port (i.e., the receiving port of the business data) are inconsistent, it may respond to the update indication and actively update the communication parameters of the second port, so that the communication parameters of the first port and the communication parameters of the second port are consistent, thereby enabling the host and the NVM subsystem to transmit business data through the first port and the second port, and ensuring the stability of the business.

[0016] In a possible implementation, an intermediate node is included between the host and the NVM subsystem, and the downstream device port further includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0017] In this implementation, when the data sent from the first port to the second port needs to be forwarded by an intermediate node, the NVM subsystem also determines whether the communication parameters used for the first port and the communication parameters of the third port for forwarding data by the intermediate node are consistent. If they are inconsistent, the NVM subsystem refuses to receive the service data sent by the first port through the second port. In this way, the service instability caused by the inconsistency between the communication parameters of the port for sending service data and the communication parameters of the port for forwarding service data can be prevented.

[0018] In one possible implementation, the method also includes: when the request message includes a mark, the NVN subsystem confirms that the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port; wherein the mark is added to the request message by the intermediate node when the communication parameters of the first port are inconsistent with the communication parameters of the third port.

[0019] In this implementation, the intermediate node can obtain the communication parameters of the first port and the communication parameters of the third port from the request message, and determine whether the communication parameters of the first port and the communication parameters of the third port are consistent. When it is confirmed that the communication parameters of the first port and the communication parameters of the third port are inconsistent, the intermediate node adds a mark to the request message, so that the NVM subsystem can determine whether the communication parameters of the first port and the communication parameters of the third port are consistent by determining whether the request message it receives includes the mark.

[0020] In a possible implementation, NVMe over Fabric is NVMe over RoCE, and the request message is an RDMA_IP_CM message, wherein the communication parameters of the first port are recorded in a reserved byte in the RDMA_IP_CM message.

[0021] In this implementation, the method can be applied to NVMe over RoCE to ensure the service stability when the host communicates with the NVM subsystem through NVMe over RoCE. Specifically, NVMe over RoCE uses Ethernet to carry the NVMe protocol, and the communication parameters may include the communication parameters used in the lossless Ethernet construction technology, such as PFC priority. The communication parameters of the first port are consistent with the communication parameters of the downstream device port, which can ensure the construction of a lossless Ethernet. The communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, which makes it difficult to build a lossless Ethernet, making the network performance of NVMe over RoCE fail to meet expectations, resulting in business instability. In this implementation, when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the NVM subsystem refuses to receive the business data sent by the host through the first port, thereby preventing business instability.

[0022] In addition, in this implementation, by extending the reserved bytes in the RDMA_IP_CM message, a request message for the communication parameters of the sending port (such as the first port) carrying the business data is obtained, thereby achieving the sending of the communication parameters of the sending port to the NVM subsystem without designing a new message.

[0023] In a possible implementation manner, the communication parameter includes at least one of a priority based on priority flow control (PFC) and a maximum transmission unit (MTU).

[0024] Among them, the priority of PFC represents the virtual channel used for data transmission. When congestion occurs, data transmission on the corresponding virtual channel can be suspended according to the priority of PFC. The communication parameters of the first port and the communication parameters of the downstream device port are consistent, including that the PFC priority of the first port is the same as the PFC priority of the downstream device port. In this way, when congestion occurs, the first port and the downstream device port can perform flow control on the same virtual channel to ensure network performance.

[0025] MTU refers to the size of the maximum data frame. The communication parameters of the first port are consistent with the communication parameters of the downstream device port, including that the MTU of the first port is less than or equal to the MTU of the downstream device port. In this way, the cache space of the data frame of the downstream device port can cache the data frame sent by the first port, ensuring that the downstream device can normally receive service data, thereby ensuring network performance.

[0026] In a second aspect, a data storage method is provided, which is applied to a host that communicates with an NVM subsystem through NVMe over Fabric, the host having a first port, and the NVM subsystem having a second port; the method comprising: the host sending a request message to the NVM subsystem, the request message being used to request the NVM subsystem to receive, through the second port, business data sent by the host through the first port, wherein the request message comprises communication parameters of the first port; the host receiving a first response message sent by the NVM subsystem, the first response message indicating that the NVM subsystem refuses to receive business data through the second port; wherein the first response message is sent by the NVM subsystem when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, and the downstream device port comprises the second port.

[0027] In a possible implementation, the method further includes: the host receives a second response message sent by the NVM subsystem, the second response message indicating that the NVM subsystem agrees to receive business data through the second port; wherein the second response message is sent by the NVM subsystem when the communication parameters of the first port are consistent with the communication parameters of the downstream device port.

[0028] In one possible implementation, the request message includes an update indication; wherein, when the communication parameters of the first port and the communication parameters of the second port are inconsistent, the NVM subsystem is used to respond to the update indication and update the communication parameters of the second port based on the communication parameters of the first port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0029] In a possible implementation, an intermediate node is included between the host and the NVM subsystem, and the downstream device port includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0030] In a possible implementation, NVMe over Fabric is NVMe over RoCE, and the request message is an RDMA_IP_CM message, wherein the communication parameters of the first port are recorded in a reserved byte in the RDMA_IP_CM message.

[0031] In a possible implementation manner, the communication parameter includes at least one of a priority based on priority flow control (PFC) and a maximum transmission unit (MTU).

[0032] In a third aspect, a data storage device is provided, which is configured in a non-volatile memory NVM subsystem that communicates with a host through NVMe over Fabric, the host has a first port, and the NVM subsystem has a second port; the device includes: a receiving unit, used to receive a request message sent by the host, the request message is used to request the NVM subsystem to receive business data sent by the host through the first port through the second port, wherein the request message includes communication parameters of the first port; a sending unit, used to send a first response message to the host when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive business data through the second port.

[0033] In a possible implementation, the sending unit is further used to: when the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem sends a second response message to the host, and the second response message indicates that the NVM subsystem agrees to receive service data through the second port.

[0034] In one possible implementation, the request message includes an update indication, and the device also includes: an update unit, which is used to respond to the update indication and update the communication parameters of the second port when the communication parameters of the first port are inconsistent with the communication parameters of the second port, so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0035] In a possible implementation, an intermediate node is included between the host and the NVM subsystem, and the downstream device port further includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0036] In one possible implementation, the device also includes: a confirmation unit, used to confirm that the communication parameters of the first port and the communication parameters of the downstream device port are inconsistent when the request message includes a mark; wherein the mark is added to the request message by the intermediate node when the communication parameters of the first port and the communication parameters of the third port are inconsistent.

[0037] In a fourth aspect, a data storage device is provided, which is configured on a host that communicates with an NVM subsystem through NVMe over Fabric, the host having a first port, and the NVM subsystem having a second port; the device includes: a sending unit, used to send a request message to the NVM subsystem, the request message is used to request the NVM subsystem to receive business data sent by the host through the first port through the second port, wherein the request message includes communication parameters of the first port; a receiving unit, used to receive a first response message sent by the NVM subsystem, the first response message indicating that the NVM subsystem refuses to receive business data through the second port; wherein the first response message is sent by the NVM subsystem when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, and the downstream device port includes the second port.

[0038] In one possible implementation, the receiving unit is also used to: receive a second response message sent by the NVM subsystem, the second response message indicating that the NVM subsystem agrees to receive business data through the second port; wherein the second response message is sent by the NVM subsystem when the communication parameters of the first port are consistent with the communication parameters of the downstream device port.

[0039] In one possible implementation, the request message includes an update indication; wherein, when the communication parameters of the first port and the communication parameters of the second port are inconsistent, the NVM subsystem is used to respond to the update indication and update the communication parameters of the second port based on the communication parameters of the first port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0040] In a possible implementation, an intermediate node is included between the host and the NVM subsystem, and the downstream device port includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0041] In a fifth aspect, a NVM subsystem is provided, comprising: a second port; a memory for storing an executable program; and a controller for executing the method provided in the first aspect by running the executable program.

[0042] In a sixth aspect, a host is provided, comprising: a first port; a memory for storing an executable program; and a processor for executing the method provided in the second aspect by running the executable program.

[0043] In a seventh aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method provided in the first aspect.

[0044] In an eighth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method provided in the second aspect.

[0045] In a ninth aspect, a computer program product comprising instructions is provided, which, when executed by a computing device, causes the computing device to execute the method provided in the first aspect.

[0046] In a tenth aspect, a computer program product comprising instructions is provided, which, when executed by a computing device, causes the computing device to execute the method provided in the second aspect.

[0047] In an eleventh aspect, a storage system is provided, comprising a host and an NVM subsystem communicating with the host through NVMe over Fabric; the host having a first port and the NVM subsystem having a second port; wherein the host is configured to send a request message to the NVM subsystem, the request message being configured to request the NVM subsystem to receive, through the second port, service data sent by the host through the first port, wherein the request message includes communication parameters of the first port; the NVM subsystem is configured to send a first response message to the host when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the downstream device port including the second port, the first response message indicating that the NVM subsystem refuses to receive service data through the second port.

[0048] The beneficial effects of the second to eleventh aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the structure of a storage system provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of a data storage method provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of a message format provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of a message format provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of a message format provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of a data storage device provided in an embodiment of the present application;

[0055] Figure 7A schematic diagram of the structure of a data storage device provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of an NVM subsystem provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of a host provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The scheme provided by the embodiment of the present application will be described below in conjunction with the accompanying drawings. In the embodiment of the present application, "plurality" refers to two or more than two. "First", "second", etc. are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.

[0059] To facilitate understanding of the solutions provided by the embodiments of the present application, some technical terms that may be involved in the embodiments of the present application are first introduced before the solutions provided by the embodiments of the present application are introduced.

[0060] NVMe over Fabric: A technology that uses a switching network (fabric) to carry the NVMe protocol, which enables the host to remotely access the NVM subsystem with high performance (e.g., high throughput, low latency, etc.). The switching network can be Infiniband (IB), Ethernet (Ethernet) or Fibre-Channel (FC).

[0061] Internet wide-area RDMA protocol (iWARP): An Ethernet network used by NVMe over Fabric technology to carry the NVMe protocol.

[0062] RDMA over converged ethernet (RoCE) is another Ethernet used by NVMe over Fabric technology to carry the NVMe protocol.

[0063] NVMe over RoCE: NVMe over Fabric technology that uses RoCE to carry the NVMe protocol.

[0064] NVM subsystem: A system that provides data storage services for a host, including one or more controllers, one or more storage media, and one or more ports. The storage medium can be packaged as one or more storage chips. The storage chip can be a flash chip (flash) or a dynamic random access memory (DRAM), etc. The port is used to receive data, and the controller can store the data received by the port in the storage medium.

[0065] Since NVMe enables the host to remotely access the NVM subsystem with high performance, more and more hosts communicate with the NVM subsystem through NVMeover Fabric to store business data. The communication performance of NVMe over Fabric depends on the consistency of communication parameters at both ends of the communication. The performance of NVMe over Fabric can only be brought into play when the communication parameters at both ends of the communication are consistent. If the communication parameters at both ends of the communication are inconsistent, it may lead to a decrease in network performance, such as increased packet loss rate and increased latency. In particular, NVMe over RoCE uses Ethernet to carry the NVMe protocol. Ethernet is "lossy". When the Ethernet network is congested, the transmitted data packets will be discarded, resulting in a serious decrease in network performance. For this reason, the industry has proposed technologies such as priority-based flow control (PFC) and explicit congestion notification (ECN) to build lossless Ethernet. However, technologies such as PFC to build lossless Ethernet rely on the communication parameters at both ends of the Ethernet. When the communication parameters at both ends are consistent, the expected effect can be achieved.

[0066] Due to incorrect configuration operations of communication parameters, errors in configuration files, and insufficient experience or capabilities of configuration personnel, there is a mismatch between the communication parameters of the host and the communication parameters of the NVM subsystem. As a result, when the host communicates with the NVM subsystem through NVMeover Fabric, network performance degrades, leading to unstable business.

[0067] An embodiment of the present application provides a data storage method. When the host needs to store business data in the NVM subsystem, the host can send a request message to the NVM subsystem, and the request message includes the communication parameters of the sending port for the host to send business data, and the request message is used to request the NVM subsystem to receive the business data sent by the host through the sending port through the receiving port. When the NVM subsystem receives the request message, it can determine whether the communication parameters of the receiving port are consistent with the communication parameters of the sending port. If they are inconsistent, the NVM subsystem refuses to receive the business data sent by the host through the sending port through the receiving port. In this way, it can be prevented that when the communication parameters of the port for sending business data and the communication parameters of the port for receiving business data are inconsistent, the host still sends business data to the NVM subsystem through NVMeover Fabric, thereby avoiding business instability.

[0068] Next, the data storage method provided in the embodiment of the present application is described.

[0069] Figure 1 A storage system 100 that can implement the data storage method is shown. Figure 1 As shown, the storage system 100 includes a host 110 and an NVM subsystem 120. The storage system 100 is a storage system based on the NVMe over Fabric architecture. The host 110 may be called an NVMe host, supports the NVMe protocol, and can communicate with the NVM subsystem 120 via NVMe over Fabric.

[0070] The host 110 can run a business and store data generated by the business in the NVM subsystem 120. The data generated by the business can be referred to as business data. The host 110 can be any device, equipment, platform or cluster with data processing and communication functions, such as a server, a virtual machine (VM), a container, etc.

[0071] like Figure 1As shown, the host 110 may include a processor 111 and at least one port, such as port A1. The processor 111 may control port A1 to send data to the NVM subsystem 120 in accordance with the relevant protocols or specifications of NVMe over Fabric. Exemplarily, the processor 111 may control port A1 by executing an NVMe-driven program, so that port A1 sends data to the NVM subsystem 120 in accordance with the relevant protocols or specifications of NVMe over Fabric. The data sent may be business data or signaling data. Signaling data refers to data used for communication negotiation or management between the host and the NVM subsystem, such as request messages, response messages, etc.

[0072] The NVM subsystem 120 can provide data storage services for the host 110. Figure 1 As shown, the NVM subsystem 120 may include: at least one controller such as a controller 121, at least one storage medium such as a storage medium 122, and at least one port such as a port A2. In some embodiments, the NVM subsystem 120 may adopt a distributed architecture, wherein the controller 121 and the storage medium 122 may be located locally at the port A2 or remotely at the port A2. In some embodiments, the NVM subsystem may be an independent NVMe storage device or a storage array composed of multiple NVMe storage devices. When the NVM subsystem is a storage array composed of multiple storage devices, the storage array may adopt a distributed architecture, that is, some storage devices are located locally at the port A2, and some storage devices are located remotely at the port A2. In some embodiments, the NVMe storage device may be an NVMe solid state disk (SSD).

[0073] Port A2 can receive data according to the relevant protocol or specification of NVMe over Fabric, and send the received data to the controller 121. If the data is business data, the controller 121 can store the business data in the storage medium 122. If the data is signaling data, the controller 121 can parse the signaling data and perform corresponding processing in response to the signaling data.

[0074] In some embodiments, the storage system 110 further includes an intermediate node 130 disposed between the host 110 and the NVM subsystem 120. Specifically, the intermediate node 130 is located on a network path between the host 110 and the NVM subsystem 120, and the intermediate node 130 is used to forward data sent from the host 110 to the NVM subsystem 120, and forward data sent from the NVM subsystem 120 to the host 110. More specifically, as Figure 1As shown, the intermediate node 130 includes a port A3 and a processor 131. Port A3 can receive data sent by port A1 and send the data to port A2 under the control of the processor 131 in accordance with the relevant protocol or specification of NVMe over Fabric. Exemplarily, the intermediate node 130 can be a switch. The intermediate node 130 can be a physical switch or a virtual switch.

[0075] In some embodiments, the above-mentioned port (e.g., port A1, port A2, or port A3) may be a network card, such as a remote direct data access network interface card (RDMA network interface card, RNIC). In some embodiments, the above-mentioned port may be a logical port in the network card. In some embodiments, the above-mentioned port is a transport layer port, that is, the above-mentioned port performs data transmission according to the transport layer protocol.

[0076] In some embodiments, the above-mentioned processor (for example, processor 111, processor 131) can be any one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a field programmable gate array (FPGA), etc.

[0077] In addition, in the scenario where the host 110 sends business data to the NVM subsystem 120, the NVM subsystem 120 belongs to a downstream device. If the storage system 100 also includes an intermediate node 130, the intermediate node 130 also belongs to a downstream device. Therefore, in the following description, when the NVM subsystem 120 and the intermediate node 130 are not distinguished, they can be referred to as downstream devices. That is to say, in the embodiment of the present application, the downstream device refers to the NVM subsystem 120, and may also refer to the intermediate node 130, or may refer to the NVM subsystem 120 and the intermediate node 130 at the same time. Correspondingly, when no special distinction is made between port A2 and port A3, they can be referred to as downstream device ports.

[0078] The above example introduces the storage system 100 provided in the embodiment of the present application. Next, in conjunction with the storage system 100, the process of the data storage method provided in the embodiment of the present application is introduced.

[0079] The host 110 may select a port for sending business data and a port for receiving business data in step 201. The port for sending business data is a port of the host 110, also called a source port, specifically a port for the host 110 to send business data. The port for receiving business data is a port of the NVM subsystem 120, also called a destination port, specifically a port for the NVM subsystem 120 to receive business data sent by the host 110. Generally speaking, when the host 110 runs a business, it is necessary to store business data in the NVM subsystem 120. Therefore, when the host 110 starts a business, step 201 and the subsequent steps to be described below may be executed. The host 110 may run one or more businesses, and each time the host 110 starts a business, step 201 and the subsequent steps to be described below may be executed.

[0080] like Figure 2 As shown, it can be set that in step 201, the host 110 selects port A1 as the port for sending business data. When port A1 is the only port of the host 110, port A1 is selected as the port for sending business data by default. When the host 110 has multiple ports, port A1 can be randomly selected from the multiple ports by the host 110, or can be selected from the multiple ports by the host 110 based on a preset rule (for example, giving priority to ports with low usage or ports with large bandwidth, etc.).

[0081] When the host 110 selects port A1 as the sending port for business data, the host 110 can obtain the communication parameters of port A1. The communication parameters of the port can also be called network configuration parameters, which are parameters used to regulate or control the behavior of the port sending or receiving data. For example, the communication parameter can be the priority of PFC. Among them, the priority of PFC represents the virtual channel used for data transmission. When congestion occurs, the data transmission on the corresponding virtual channel can be suspended according to the priority of PFC. For another example, the communication parameter can be the maximum transmission unit (MTU). MTU refers to the size of the maximum data frame. Through MTU, the size of the data frame sent or received by the port can be regulated. The communication parameters of the port can also be other parameters, which are not listed here one by one.

[0082] Continue reading Figure 2 , it can be set that in step 201, the host 110 selects port A2 as the receiving port for service data. The host 110 can select a receiving port for service data from the ports of the NVM subsystem 120, and request the NVM subsystem 120 to receive the service data sent by the host 110 through the receiving port selected by the host 110.

[0083] In some embodiments, as described above, the storage system 100 is an NVMe over Fabric architecture, and the NVMe over Fabric architecture has a routing server. The routing server saves the routing information of the NVM subsystem. The routing information of the NVM subsystem includes at least one port identifier (port identifier, port ID) of the NVM subsystem. Among them, the at least one port identifier corresponds one-to-one to at least one port of the NVM subsystem. In step 201, the host 110 can obtain the routing information of the NVM subsystem 120 from the routing server, and obtain at least one port identifier of the NVM subsystem 1210 from the routing information of the NVM subsystem 120. Then, the host 110 can select the port identifier of port A2 from the at least one port identifier to select port A2 as the receiving port for business data.

[0084] Continue reading Figure 2 When selecting the sending port of the business data and the receiving port of the business data, the host 110 may send a request message C1 to the NVM subsystem 120 in step 202. The request message C1 is used to request the NVM subsystem 120 to receive the business data sent by the host 111 through port A1 through port A2. The request message C1 may also be called a connection request, which is used to request the NVM subsystem 120 to agree to establish a link between port A1 and port A2, and the link is used to transmit business data between port A1 and port A2.

[0085] The request message C1 includes the port identifier of port A2. That is, the host 110 may add the port identifier of the receiving port selected in step 201 to the request message C1, so that the NVM subsystem 120 recognizes the receiving port as port A2 when receiving the request message C1.

[0086] The request message C1 includes the communication parameters of port A1. That is, the host 110 can add the communication parameters of port A1 acquired in step 201 to the request message C1, so that when the NVM subsystem 120 receives the request message C1, it can determine whether the communication parameters of port A1 are consistent with the communication parameters of port A2.

[0087] In some embodiments, the request message C1 also includes an update indication. The update indication is also called a consistency configuration indication, which is used to instruct the downstream device to update the communication parameters of the downstream device port based on the communication parameters of port A1 when the communication parameters of the downstream device port are inconsistent with the communication parameters of port A1. The communication parameters of the downstream device port after the update are consistent with the communication parameters of port A1.

[0088] In some embodiments, the request message C1 further includes a check indication, which is also called a consistency check indication, and is used to instruct the downstream device to detect whether the communication parameters of the downstream device port are consistent with the communication parameters of the port A1.

[0089] In some embodiments, the NVMe over Fabric used for communication between the host 110 and the NVM subsystem 120 is NVMe over RoCE. The relevant protocols or specifications of NVMe over RoCE define the RDMA internet protocol connection management (RDMA internet protocol connection management, RDMA_IP_CM) service and the RDMA_IP_CM message. In this embodiment, the reserved byte in the RDMA_IP_CM message can be extended to obtain the request message C1. In other words, the request message C1 belongs to the RDMA_IP_CM message, wherein the reserved byte of the RDMA_IP_CM message carries the communication parameters of port A1, that is, the communication parameters of port A1 are recorded in the reserved bytes of the RDMA_IP_CM message. Exemplarily, the RDMA_IP_CM message includes a connection management request (connection management request, CM REQ) message, and the request message C1 can be obtained by extending the reserved bytes of the connection management request message. The specific description is as follows.

[0090] The relevant protocols or specifications of NVMe over RoCE include: NVMe RDMA transport layer specification (NVM-Express-RDMA-transport specification) and Infiniband protocol. Among them, the NVMe RDMA transport layer specification specifies that the RDMA transport layer connection management service used to establish the link between the NVMe host and the NVMe subsystem is the RDMA_IP_CM service. The Infiniband protocol specifically defines the RDMA_IP_CM service and the RDMA_IP_CM message (such as the connection management request message).

[0091] in, Figure 3 The message format of the connection management request message is shown in FIG. Figure 3 As shown in FIG. 1 , the connection management request message has a total of 92 bytes. The InfiniBand protocol defines the first 36 bytes of the 92 bytes. The details are as follows.

[0092] In the following description, the counting starts from 0, that is, the 0th one represents the beginning or the front one.

[0093] The 0th to 7th bits and the 8th to 7th bits of the 0th to 3rd bytes of the 92 bytes record the source port. The 16th to 23rd bits of the 0th to 3rd bytes record the IP version (IPversion) and the reserved field (reserved, Res). The 16th to 23rd bits of the 0th to 3rd bytes record the maximum version (maj version, MajV) and the minimum version (min version, MinV).

[0094] The 4th to 7th bytes, the 8th to 11th bytes, the 12th to 15th bytes, and the 16th to 19th bytes of the 92 bytes are all used to record the source IP address. The source IP address can occupy up to 128 bits. The 16th to 19th bytes are used to record the data on the 0th to 31st bits of the 128 bits, the 12th to 15th bytes are used to record the data on the 32nd to 63rd bits of the 128 bits, the 8th to 11th bytes are used to record the data on the 64th to 95th bits of the 128 bits, and the 4th to 7th bytes are used to record the data on the 96th to 127th bits of the 128 bits.

[0095] The 20th to 23rd bytes, the 24th to 27th bytes, the 28th to 31st bytes, and the 32nd to 35th bytes of the 92 bytes are all used to record the destination IP address. The destination IP address can occupy up to 128 bits. The 32nd to 35th bytes are used to record the data on the 0th to 31st bits of the 128 bits, the 28th to 31st bytes are used to record the data on the 32nd to 63rd bits of the 128 bits, the 24th to 27th bytes are used to record the data on the 64th to 95th bits of the 128 bits, and the 20th to 23rd bytes are used to record the data on the 96th to 127th bits of the 128 bits.

[0096] The remaining 56 bytes of the 92 bytes, except for the 36 bytes mentioned above, can be used to record application layer private data. The application layer can also be called a consumer, and accordingly, the application layer private data can also be called consumer private data.

[0097] In addition, the NVMe RDMA transport layer specification defines the first 10 bytes of the 56 bytes, and the remaining 46 bytes are reserved bytes for the connection management request message. Figure 4 The format of the first 42 bytes of the 56 bytes is shown. Figure 4 As shown, the 0th to 1st bytes of the 42 bytes record the format of the RDMA private data, the 2nd to 3rd bytes record the queue ID (queue ID, QID), the 4th to 5th bytes record the size of the RDMA queue pair (queue pair, QP) host receive queue (receive queue, RQ), the 6th to 7th bytes record the size of the RDMA queue pair (queuepair, QP) host send queue (send queue, SQ), the 6th to 7th bytes record the controller ID (controller ID) of the solid-state drive, and the 10th to 31st bytes are reserved bytes for the connection management request message.

[0098] In step 202, Figure 4 One or more bytes from the 10th to 31st bytes (ie, reserved bytes of the connection management request message) are extended to carry the communication parameters of port A1 using the one or more bytes, ie, to record the communication parameters of port A1 using the one or more bytes.

[0099] In one example, Figure 4 The 10th byte of the 42 bytes shown can be used to record an update indication and / or a verification indication, that is, the 10th byte is used to record an update indication and / or a verification indication. Figure 5 , the value (value) recorded in the 10th byte represents an update indication or a check indication. Among them, the value recorded in the 10th byte is 0, indicating that the host 110 instructs the downstream device not to consider whether the communication parameters of port A1 are consistent with the communication parameters of the downstream device port. That is, the 0 recorded in the 10th byte means that neither consistency check nor consistency configuration is performed. The value recorded in the 10th byte is 1, indicating that the host 110 instructs the downstream device to detect whether the communication parameters of port A1 are consistent with the communication parameters of the downstream device port. That is, the 1 recorded in the 10th byte represents a consistency check. The value recorded in the 10th byte is 2, indicating that the host 110 instructs the downstream device to update the communication parameters of the downstream device port when the communication parameters of port A1 are inconsistent with the communication parameters of the downstream device port, so that the updated communication parameters of the downstream device port are consistent with the communication parameters of port A1. That is, the 2 recorded in the 10th byte represents an update indication.

[0100] In one example, Figure 4The 11th byte of the 42 bytes shown is used to record the network type of the network between the host 110 and the NVM subsystem 120. Among them, when the network between the host 110 and the NVM subsystem 120 is an NVMe over Fabric network, the update indication and / or the detection indication in the request message C1 is valid. That is to say, when the network between the host 110 and the NVM subsystem 120 is an NVMe over Fabric network, the downstream device responds to the update indication and / or the detection indication in the request message C1 and performs an update operation and / or a detection operation. When the network between the host 110 and the NVM subsystem 120 is not an NVMe over Fabric network, the update indication and / or the detection indication in the request message C1 is invalid. That is to say, when the network between the host 110 and the NVM subsystem 120 is not an NVMe over Fabric network, the downstream device does not respond to the update indication and / or the detection indication in the request message C1. In one example, when the network between the host 110 and the NVM subsystem 120 is not an NVMe over Fabric network, the NVM subsystem 120 may directly reply with a reject message. The reject message will be described in detail below and will not be described here. In one example, the value recorded in the 11th byte represents the network type of the network between the host 110 and the NVM subsystem 120. The value recorded in the 11th byte is 1, indicating that the network type of the network between the host 110 and the NVM subsystem 120 is an NVMe over Fabric network. The value recorded in the 11th byte is not 1, indicating that the network type of the network between the host 110 and the NVM subsystem 120 is not an NVMe over Fabric network.

[0101] In one example, Figure 4 The 12th byte of the 42 bytes shown is used for the intermediate node 130 to add a flag in the request message C1. When the communication parameters of port A1 and port A3 are inconsistent, the intermediate node 130 records the flag in the 12th byte to add the flag to the request message C1. The flag is also called a switch path verification flag, which is used to indicate that the communication parameters of port A1 and port A3 are inconsistent. In one example, the value recorded in the 12th byte is 0, which indicates that the intermediate node 130 did not add the flag to the request message C1. If the value recorded in the 12th byte is not 0, it indicates that the intermediate node 130 added the flag to the request message C1. Among them, there can be one or more communication parameters of the port, and the value recorded in the 12th byte represents the number of inconsistent communication parameters.

[0102] In one example, the request message C1 may be set to include n communication parameters of the port A1, where n is an integer greater than or equal to 1. Figure 4 The 12+1 to 12+n bytes of the 42 bytes shown record the n communication parameters. Figure 5 As shown, one byte records one communication parameter, for example, the 13th byte records communication parameter 1, ..., and the 12+nth byte records communication parameter n.

[0103] The above example introduces the request message C1. Next, the transmission process of the request message C1 is introduced.

[0104] Continue reading Figure 2 In step 202 , the host 110 may send a request message C1 to the NVM subsystem 120 .

[0105] In some embodiments, the host 110 may send a request message C1 to the NVM subsystem 120 through the physical layer. That is, in this embodiment, the request message C1 does not need to be encapsulated according to the upper layer protocol, and the request message C1 is directly sent to the NVM subsystem 120 through the physical layer. The upper layer protocol here refers to the upper layer protocol of the physical layer, such as the application layer, the transport layer, the network layer, and the data link layer. In some embodiments, the host 110 may send the request message C1 to the NVM subsystem 120 through the port A1.

[0106] The host 110 may add the IP address of the NVM subsystem 120 (e.g., the IP address of the port A2) to the request message C1, and send the request message C1 to send the request message C1 to the NVM subsystem 120. In some embodiments, as described above, the request message C1 may be a connection management request message, and the IP address of the NVM subsystem may be used as the destination address, and recorded in one or more bytes from the 20th to the 35th bytes of the request message C1, thereby adding the IP address of the NVM subsystem to the request message C1.

[0107] In some embodiments, the storage system 100 further includes an intermediate node 130. Figure 2 As shown, step 202 includes: step 2021, the host 110 sends the request message C1 to the intermediate node 130. Exemplarily, the intermediate node 130 can receive the request message C1 sent by the host 110 through port A3.

[0108] In the first example of this embodiment, when receiving the request message C1, the intermediate node 130 may directly send the request message C1 to the NVM subsystem 120. That is, in this example, the intermediate node 130 is only used to forward the request message C1.

[0109] In the second example of this embodiment, Figure 2 As shown, step 202 also includes step 2022, in which the intermediate node 130 determines whether the communication parameters of port A1 are consistent with the communication parameters of port A3. Specifically, when receiving the request message C1, the intermediate node 130 can parse the request message C1 and obtain the communication parameters of port A1 in the request message C1. Then, the intermediate node 130 determines whether the communication parameters of port A1 are consistent with the communication parameters of port A3. In one example, the communication parameter can be a PFC priority, and the consistency of the communication parameters of port A1 and port A3 means that the PFC priority of port A1 is the same as the PFC priority of port A3. In another example, the communication parameter can be an MTU, and the consistency of the communication parameters of port A1 and port A3 means that the MTU of port A3 is greater than or equal to the MTU of port A1. And so on.

[0110] When the request message C1 includes a verification indication, the intermediate node 130 executes the solution of the second example; otherwise, the intermediate node 130 executes the solution of the first example.

[0111] In the first possible implementation of the second example, if the judgment result of step 2022 is no, that is, the communication parameters of port A1 and port A3 are inconsistent, the intermediate node 130 executes step 2023 to add a mark in the request message C1. Figure 4 and 5 In the illustrated embodiment, the intermediate node 130 may be Figure 4 The tag is recorded in the 12th byte of the 42 bytes shown to add the tag to the request message C1. When the tag is added to the request message C1, the intermediate node 130 can send the request message C1 added with the tag (i.e., the request message C1 with the tag added) to the NVM subsystem 120 through step 2023. That is, if the communication parameters of port A1 and the communication parameters of port A3 are inconsistent, the request message C1 sent by the intermediate node 130 to the NVM subsystem 120 includes the tag. The tag is used to indicate that the communication parameters of port A1 and the communication parameters of port A3 are inconsistent.

[0112] If the result of the determination in step 202 is yes, that is, the communication parameters of port A1 and port A3 are consistent, the intermediate node 130 may directly execute step 2024 to send a request message C1 to the NVM subsystem 120. In other words, if the communication parameters of port A1 and port A3 are consistent, the request message C1 sent by the intermediate node 130 to the NVM subsystem 120 does not include the above-mentioned mark.

[0113] In the second possible implementation of the second example, if the judgment result of step 2022 is no, and the request message C1 includes an update indication, the intermediate node 130 can respond to the update indication and update the communication parameters of port A3 based on the communication parameters of port A1, so that the communication parameters of port A3 are consistent with the communication parameters of port A1. For example, if the PFC priority of port A3 is different from the PFC priority of port A1, the updated PFC priority of port A3 is the same as the PFC priority of port A1. For another example, if the MTU of port A3 is smaller than the MTU of port A1, the updated MTU of port A3 is greater than or equal to the MTU of port A1.

[0114] After completing the above update, the intermediate node 130 may send a request message C1 to the NVM subsystem 120. The request message C1 here does not include the above tag.

[0115] NVM subsystem 120 may receive the request message C1. The request message C1 may be sent directly from host 110 to NVM subsystem 120, or may be forwarded by intermediate node 130. In some embodiments, NVM subsystem 120 may receive the request message C1 through port A2.

[0116] Continue reading Figure 2 When the NVM subsystem 120 receives the request message C1, it may execute step 203 to determine whether the communication parameters of the port A1 and the communication parameters of the port A2 are consistent.

[0117] As described above, the request message C1 includes a port identifier. The NVM subsystem 120 obtains the port identifier from the request message C1. When the port identifier is identified as the port identifier of port A2, it can be identified that the request message C1 is used to request the NVM subsystem 120 to receive service data sent by the host 110 through port A1 through port A2.

[0118] The NVM subsystem 120 may obtain the communication parameters of port A2, and obtain the communication parameters of port A1 from the request message C1. Then, the NVM subsystem 120 may execute step 203 to determine whether the communication parameters of port A1 are consistent with the communication parameters of port A2. In one example, the communication parameter may be a PFC priority, and the consistency of the communication parameters of port A1 and port A2 means that the PFC priority of port A1 is the same as the PFC priority of port A2. In another example, the communication parameter may be an MTU, and the consistency of the communication parameters of port A1 and port A2 means that the MTU of port A2 is greater than or equal to the MTU of port A1. And so on.

[0119] In some embodiments, Figure 2 As shown, if the judgment result of step 203 is yes, that is, the communication parameters of port A1 and port A2 are consistent, the NVM subsystem 120 can execute step 204a to send a response message B1 to the host 110. The response message B1 can also be called an agree message, which is used to indicate that the NVM subsystem 120 agrees to receive the service data sent by the host 110 through port A1 through port A2.

[0120] In some embodiments, the request message C1 includes an update indication. If the judgment result of step 203 is no, the NVM subsystem 120 can respond to the update indication and update the communication parameters of port A2 based on the communication parameters of port A1, so that the communication parameters of port A2 are consistent with the communication parameters of port A1. For example, the PFC priority of port A2 is different from the PFC priority of port A1, then the updated PFC priority of port A2 is the same as the PFC priority of port A1. For another example, the MTU of port A2 is smaller than the MTU of port A1, then the updated MTU of port A2 is greater than or equal to the MTU of port A1. After completing the update of the communication parameters of port A2, the NVM subsystem 120 can execute step 204a to send a response message B1 to the host 110.

[0121] When the host 110 receives the response message B1, the host 110 may send the business data through the port A1. The business data is transmitted through the NVMe over Fabric between the host 110 and the NVM subsystem 120 and reaches the NVM subsystem 120. The NVM subsystem 120 receives the business data through the port A2. Among them, the controller 121 of the NVM subsystem 120 may store the business data in the storage medium 122, thereby completing the storage of the business data.

[0122] In some embodiments, if the result of the determination in step 203 is no, that is, the communication parameters of port A1 and port A2 are inconsistent, the NVM subsystem 120 may execute step 204b to send a response message B2 to the host 110. The response message B2 may also be referred to as a rejection message or a connection rejection message, which indicates that the NVM subsystem 120 refuses to receive service data sent by the host 110 through port A1 through port A2, or indicates that the NVM subsystem 120 refuses to establish a link between port A1 and port A2.

[0123] In some embodiments, the NVM subsystem 120 may further detect whether the request message C1 includes a tag. If the request message C1 includes a tag, the NVM subsystem 120 executes step 204b to send a response message B2 to the host 110 regardless of whether the communication parameters of the port A1 and the communication parameters of the port A2 are consistent.

[0124] In some embodiments, when receiving the response message B2, the host 110 may provide prompt information, such as displaying prompt information or voice broadcasting prompt information. The prompt information is used to prompt the user to modify the communication parameters of port A1 and / or port A2 so that the communication parameters of port A1 and port A2 are consistent.

[0125] In an example of this embodiment, the user may trigger the host 110 to send a request message C1 to the NVM subsystem 120 again, so as to request the NVM subsystem 120 to receive the service data sent by the host 110 through the port A1 through the port A2 again.

[0126] In another example of this embodiment, the host 110 may start a timer when receiving the response message B2. The duration of the timer may be preset, such as 10 seconds, 60 seconds, etc. When the timer times out, the host 110 may send a request message C1 to the NVM subsystem 120 again to request the NVM subsystem 120 to receive the service data sent by the host 110 through the port A1 through the port A2 again.

[0127] In some embodiments, the host 110 also includes a port A4 (not shown). Port A4 can be implemented with reference to the introduction of port A1 above, which will not be repeated here. When the host 110 receives the response message B2, it can send a request message C2 to the NVM subsystem 120. The request message C2 is used to request the NVM subsystem 120 to receive the business data sent by the host 110 through port A4 through port A2. Among them, the request message C2 includes the communication parameters of port A4. When the NVM subsystem 120 receives the request message C2, it can determine whether the communication parameters of port A4 are consistent with the communication parameters of port A2. If they are consistent, the NVM subsystem 120 sends a response message B1 to the host 110. If they are inconsistent, the NVM subsystem 120 sends a response message B2 to the host 110. For details, please refer to the above description of the response message B2. Figure 2 The introduction and implementation of the illustrated embodiment will not be repeated here.

[0128] In some embodiments, the NVM subsystem 120 also includes port A5 (not shown). Port A5 can be implemented with reference to the introduction of port A2 above, which will not be repeated here. When the host 110 receives the response message B2, it can send a request message C3 to the NVM subsystem 120. The request message C3 is used to request the NVM subsystem 120 to receive the business data sent by the host 110 through port A1 through port A5. Among them, the request message C3 includes the communication parameters of port A1. When the NVM subsystem 120 receives the request message C3, it can determine whether the communication parameters of port A5 are consistent with the communication parameters of port A1. If they are consistent, the NVM subsystem 120 sends a response message B1 to the host 110. If they are inconsistent, the NVM subsystem 120 sends a response message B2 to the host 110. For details, please refer to the above description of the response message B2. Figure 2 The introduction and implementation of the illustrated embodiment will not be repeated here.

[0129] In summary, when the host and the NVM subsystem use NVMe over Fabric to communicate with the NVM subsystem, when it is necessary to store business data in the NVM subsystem, the NVM subsystem can first determine whether the communication parameters of the sending port used by the host to send business data and the communication parameters of the receiving port used by the NVM subsystem to receive business data are consistent. If they are inconsistent, the NVM subsystem can refuse to receive business data through the receiving port, thereby preventing business instability caused by the inconsistency between the communication parameters of the sending port and the communication parameters of the receiving port. In other words, through this method, when the host and the NVM subsystem use NVMe over Fabric to communicate with the NVM subsystem, the host can perceive whether the stability of the business can be guaranteed by transmitting business data between the host and the NVM subsystem through the first port and the second port. This avoids the situation where the stability of the business cannot be guaranteed by transmitting business data between the host and the NVM subsystem through the first port and the second port, and still transmits business data through the first port and the second port, thereby preventing business instability.

[0130] The embodiment of the present application provides a data storage device 600, which is configured in an NVM subsystem (e.g., NVM subsystem 120). The NVM subsystem communicates with a host (e.g., host 110) via NVMe over Fabric. The host has a first port (e.g., port A1), and the NVM subsystem has a second port (e.g., port A2). Figure 6 As shown, the device 600 includes:

[0131] A receiving unit 610 is configured to receive a request message sent by the host, wherein the request message is used to request the NVM subsystem to receive service data sent by the host through the first port through the second port, wherein the request message includes communication parameters of the first port;

[0132] The sending unit 620 is used to send a first response message to the host when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive the service data through the second port.

[0133] In some embodiments, the sending unit 620 is further used for: when the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem sends a second response message to the host, and the second response message indicates that the NVM subsystem agrees to receive the service data through the second port.

[0134] In an example of this embodiment, the request message includes an update indication, and the device 600 also includes: an update unit 630, which is used to respond to the update indication and update the communication parameters of the second port when the communication parameters of the first port and the communication parameters of the second port are inconsistent, so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0135] In some embodiments, an intermediate node is included between the host and the NVM subsystem, and the downstream device port further includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0136] In an example of this embodiment, the device 600 also includes: a confirmation unit 640, which is used to confirm that the communication parameters of the first port and the communication parameters of the downstream device port are inconsistent when the request message includes a mark; wherein the mark is added to the request message by the intermediate node when the communication parameters of the first port and the communication parameters of the third port are inconsistent.

[0137] The functions of the functional units of the device 600 may be implemented with reference to the above description of the NVM subsystem 120 and will not be described in detail here.

[0138] The present application embodiment provides a data storage device 700, which is configured on a host, such as the host 110. The host communicates with an NVM subsystem (such as the NVM subsystem 120) via NVMe over Fabric. The host has a first port, and the NVM subsystem has a second port. Figure 7 As shown, the device 700 includes:

[0139] A sending unit 710 is used to send a request message to the NVM subsystem, wherein the request message is used to request the NVM subsystem to receive, through the second port, the service data sent by the host through the first port, wherein the request message includes a communication parameter of the first port;

[0140] The receiving unit 720 is used to receive a first response message sent by the NVM subsystem, wherein the first response message indicates that the NVM subsystem refuses to receive the business data through the second port; wherein the first response message is sent by the NVM subsystem when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, and the downstream device port includes the second port.

[0141] In some embodiments, the receiving unit 720 is also used to: receive a second response message sent by the NVM subsystem, the second response message indicating that the NVM subsystem agrees to receive the business data through the second port; wherein the second response message is sent by the NVM subsystem when the communication parameters of the first port are consistent with the communication parameters of the downstream device port.

[0142] In an example of this embodiment, the request message includes an update indication; wherein, when the communication parameters of the first port and the communication parameters of the second port are inconsistent, the NVM subsystem is used to respond to the update indication, and based on the communication parameters of the first port, update the communication parameters of the second port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

[0143] In some embodiments, an intermediate node is included between the host and the NVM subsystem, and the downstream device port includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

[0144] The present application embodiment provides a NVM subsystem 800. Figure 8 As shown, the NVM subsystem 800 includes: a port 810, a memory 820, and a controller 830. The memory 820 is used to store executable programs; the controller 830 is used to execute the operations performed by the NVM subsystem 120 described above by running the executable programs.

[0145] The present application embodiment provides a host 900. Fig. 9 As shown, the host 900 includes: a port 910, a memory 920 and a processor 930. The memory 920 is used to store executable programs; the processor 930 is used to execute the operations performed by the host 110 as described above by running the executable programs.

[0146] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device is caused to perform the operations performed by the NVM subsystem 120 described above.

[0147] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device is caused to perform the operations performed by the host 110 described above.

[0148] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to perform the operations performed by the NVM subsystem 120 described above.

[0149] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium. The computer-readable storage medium includes instructions that instruct the computing device to perform the operations performed by the host 110 described above.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data storage method, characterized in that: A non-volatile memory NVM subsystem is applied to communicate with a host through NVMe over Fabric, the host has a first port, and the NVM subsystem has a second port; the method includes: The NVM subsystem receives a request message sent by the host, wherein the request message is used to request the NVM subsystem to receive service data sent by the host through the first port through the second port, wherein the request message includes communication parameters of the first port; When the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the NVM subsystem sends a first response message to the host, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive the service data through the second port.

2. The method according to claim 1, characterized in that The method further includes: when the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem sends a second response message to the host, and the second response message indicates that the NVM subsystem agrees to receive the service data through the second port.

3. The method according to claim 2, characterized in that The request message includes an update indication, and the method further includes: When the communication parameters of the first port are inconsistent with the communication parameters of the second port, the NVM subsystem updates the communication parameters of the second port in response to the update indication so that the communication parameters of the second port are consistent with the communication parameters of the first port.

4. The method according to any one of claims 1 to 3, characterized in that An intermediate node is included between the host and the NVM subsystem, and the downstream device port further includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

5. The method according to claim 4, characterized in that The method further includes: when the request message includes a tag, the NVN subsystem confirming that the communication parameters of the first port and the communication parameters of the downstream device port are inconsistent; The mark is added by the intermediate node to the request message when the communication parameters of the first port and the communication parameters of the third port are inconsistent.

6. The method according to any one of claims 1 to 5, characterized in that The NVMe over Fabric is NVMe over RoCE, and the request message is an RDMA_IP_CM message, wherein the communication parameters of the first port are recorded in reserved bytes in the RDMA_IP_CM message.

7. The method according to any one of claims 1 to 6, characterized in that The communication parameters include at least one of a priority based on priority flow control PFC and a maximum transmission unit MTU.

8. A data storage method, characterized in that: The method is applied to a host communicating with an NVM subsystem via NVMe over Fabric, wherein the host has a first port and the NVM subsystem has a second port; the method comprises: The host sends a request message to the NVM subsystem, wherein the request message is used to request the NVM subsystem to receive, through the second port, service data sent by the host through the first port, wherein the request message includes communication parameters of the first port; The host receives a first response message sent by the NVM subsystem, where the first response message indicates that the NVM subsystem refuses to receive the service data through the second port; The first response message is sent by the NVM subsystem when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, and the downstream device port includes the second port.

9. The method according to claim 8, characterized in that The method further includes: the host receiving a second response message sent by the NVM subsystem, the second response message indicating that the NVM subsystem agrees to receive the service data through the second port; The second response message is sent by the NVM subsystem when the communication parameters of the first port are consistent with the communication parameters of the downstream device port.

10. The method according to claim 9, characterized in that The request message includes an update indication; wherein, When the communication parameters of the first port are inconsistent with the communication parameters of the second port, the NVM subsystem is used to respond to the update indication and update the communication parameters of the second port based on the communication parameters of the first port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

11. The method according to any one of claims 8 to 10, characterized in that: An intermediate node is included between the host and the NVM subsystem, and the downstream device port includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

12. The method according to any one of claims 8 to 11, characterized in that The NVMe over Fabric is NVMe over RoCE, and the request message is an RDMA_IP_CM message, wherein the communication parameters of the first port are recorded in reserved bytes in the RDMA_IP_CM message.

13. The method according to any one of claims 8 to 12, characterized in that: The communication parameters include at least one of a priority based on priority flow control PFC and a maximum transmission unit MTU.

14. A data storage device, characterized in that: A non-volatile memory NVM subsystem configured to communicate with a host via NVMe over Fabric, the host having a first port, and the NVM subsystem having a second port; the device comprising: a receiving unit, configured to receive a request message sent by the host, wherein the request message is used to request the NVM subsystem to receive, through the second port, service data sent by the host through the first port, wherein the request message includes communication parameters of the first port; A sending unit is used to send a first response message to the host when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive the service data through the second port.

15. The device according to claim 14, characterized in that The sending unit is further used for: when the communication parameters of the first port are consistent with the communication parameters of the downstream device port, the NVM subsystem sends a second response message to the host, and the second response message indicates that the NVM subsystem agrees to receive the service data through the second port.

16. The device according to claim 15, characterized in that The request message includes an update indication, and the apparatus further includes: An updating unit is used for updating the communication parameters of the second port in response to the update indication when the communication parameters of the first port are inconsistent with the communication parameters of the second port, so that the communication parameters of the second port are consistent with the communication parameters of the first port.

17. The device according to any one of claims 14 to 16, characterized in that An intermediate node is included between the host and the NVM subsystem, and the downstream device port further includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

18. The device according to claim 17, characterized in that The device also includes: a confirmation unit, configured to confirm that the communication parameters of the first port and the communication parameters of the downstream device port are inconsistent when the request message includes a tag; The mark is added by the intermediate node to the request message when the communication parameters of the first port and the communication parameters of the third port are inconsistent.

19. A data storage device, characterized in that: A device configured for a host communicating with an NVM subsystem via NVMe over Fabric, wherein the host has a first port and the NVM subsystem has a second port; the device comprises: a sending unit, configured to send a request message to the NVM subsystem, wherein the request message is used to request the NVM subsystem to receive, through the second port, the service data sent by the host through the first port, wherein the request message includes a communication parameter of the first port; a receiving unit, configured to receive a first response message sent by the NVM subsystem, wherein the first response message indicates that the NVM subsystem refuses to receive the service data through the second port; The first response message is sent by the NVM subsystem when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, and the downstream device port includes the second port.

20. The device according to claim 19, characterized in that The receiving unit is further used to: receive a second response message sent by the NVM subsystem, where the second response message indicates that the NVM subsystem agrees to receive the service data through the second port; The second response message is sent by the NVM subsystem when the communication parameters of the first port are consistent with the communication parameters of the downstream device port.

21. The device according to claim 20, characterized in that The request message includes an update indication; wherein, When the communication parameters of the first port are inconsistent with the communication parameters of the second port, the NVM subsystem is used to respond to the update indication and update the communication parameters of the second port based on the communication parameters of the first port so that the communication parameters of the second port are consistent with the communication parameters of the first port.

22. The device according to any one of claims 19 to 21, characterized in that An intermediate node is included between the host and the NVM subsystem, and the downstream device port includes a third port, and the third port is used by the intermediate node to forward data sent from the first port to the second port.

23. A NVM subsystem, characterized in that: include: Second port; A memory for storing executable programs; A controller, configured to execute the method according to any one of claims 1 to 7 by running the executable program.

24. A host, characterized in that: include: First port; A memory for storing executable programs; A processor, configured to execute the method according to any one of claims 8 to 13 by running the executable program.

25. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 13.

26. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 13.

27. A storage system, characterized in that: The invention comprises a host and an NVM subsystem communicating with the host via NVMe over Fabric; the host has a first port, and the NVM subsystem has a second port; wherein, The host is used to send a request message to the NVM subsystem, wherein the request message is used to request the NVM subsystem to receive, through the second port, service data sent by the host through the first port, wherein the request message includes communication parameters of the first port; The NVM subsystem is used to send a first response message to the host when the communication parameters of the first port are inconsistent with the communication parameters of the downstream device port, the downstream device port includes the second port, and the first response message indicates that the NVM subsystem refuses to receive the service data through the second port.

Images