Data storage method, data storage device and data storage system

Abstract

The present application relates to a data storage method, a data storage device and a data storage system, wherein a controller obtains a target command from an external host through a communication bus, caches the parsed target command in a command queue cache, and sends a notification of processing a new command to each target hard disk through the communication bus; each target hard disk reads and processes the parsed target command from the command queue cache of the controller through the communication bus, and after the processing is completed, writes the first command completion to the completion queue cache of the controller through the communication bus; the controller monitors the write operation of the completion queue cache through a built-in monitoring circuit, processes the completion of the first command, and pushes the completion of the second command to the external host through the communication bus after the processing is completed; if the target command is an I / O command, each target hard disk directly interacts with the external host through the communication bus for user data. Therefore, the response delay of the independent redundant disk array to the command of the external host can be shortened.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a data storage method, a data storage device and a data storage system. Background Art

[0002] Redundant Arrays of Independent Disks (RAID) is a hard disk group composed of multiple independent hard disks. It improves access performance through concurrent hard disk operations and improves data security through data redundancy. Traditional RAID controllers are connected in series between the host and the hard disk through more than two ports, one of which is connected to the host as a terminal node, and the remaining ports are connected to the hard disk as root users, and the control logic of RAID is implemented between the two ports.

[0003] In a traditional serial RAID controller, since all user commands and data need to pass through the RAID controller, the command response delay is increased, and the bandwidth of the RAID controller and host interface is limited, which affects the access performance of the RAID. Summary of the invention

[0004] Based on this, it is necessary to provide a data storage method, a data storage device and a data storage system to address the above technical problems, and connect each hard disk and its controller in an independent redundant disk array to the communication bus of an external host in a parallel manner, thereby breaking through the performance limitation of the interface bandwidth of the RAID controller on the independent redundant disk array, shortening the response delay of the independent redundant disk array to the command of the external host, and improving the data access efficiency of the independent redundant disk array. The RAID controller is connected to the external host communication bus only through one bus interface, which is more conducive to the RAID controller to achieve the goal of reducing power consumption and optimizing costs.

[0005] The present application provides a data storage device, which includes an independent redundant disk array and a controller of the independent redundant disk array. Each hard disk in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices, and a monitoring circuit is built in the controller; wherein the controller obtains a target command from the external host through the communication bus, parses the target command, caches the parsed target command into a command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk through the communication bus; after each target hard disk receives the notification of processing a new command, it reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and writes the first command completion to the completion queue cache of the controller through the communication bus after the processing is completed; the controller monitors the completion queue cache of the controller through the built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command completion is processed, and after the processing is completed, the second command completion is pushed to the external host through the communication bus; wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host for user data through the communication bus.

[0006] In one of the embodiments, if the target command is a management command, each target hard disk responds to the parsed management command; if each target hard disk needs to return management data to the external host, the returned first management data is written into the designated cache area of ​​the controller through the communication bus; after each target hard disk responds to the parsed management command and completes the operation, the first management command is completed and written into the completion queue cache of the controller through the communication bus.

[0007] In one of the embodiments, the completion queue cache includes a management completion queue cache; after the response operation of the parsed management command is completed, each target hard disk completes writing the first management command to the management completion queue cache of the controller through the communication bus; when the controller monitors the write operation of the management completion queue cache through the built-in monitoring circuit, it identifies the hard disk identifier of the corresponding target hard disk and determines that if the management command needs to return management data to the external host, the first management data returned by the corresponding target hard disk is read and processed from the designated cache area, and the second management data is written to the cache designated by the external host through the communication bus; the controller processes the completion of the written first management command, and after the processing is completed, the second management command is completed and written to the external host through the communication bus, and an interrupt signal is sent to the external host.

[0008] In one of the embodiments, if the target command is a write command, each target hard disk responds to the parsed write command, reads user data from the memory of the external host or reads verification data from the memory of the controller through the communication bus, and writes it to the target hard disk; after the target hard disk completes the data writing operation, it completes writing the first write command to the completion queue cache of the controller through the communication bus.

[0009] In one of the embodiments, the completion queue cache includes an I / O completion queue cache; each target hard disk completes writing a first write command to the I / O completion queue cache of the controller via a communication bus; when the controller detects the write operation of the I / O completion queue cache via a built-in monitoring circuit, it obtains the hard disk identification, completion queue sequence number and related information of the first write command completion of the corresponding target hard disk, processes the first write command completion, writes the second write command completion to the external host via the communication bus after the processing is completed, and sends an interrupt signal to the external host.

[0010] In one of the embodiments, if the target command is a read command, each target hard disk responds to the parsed read command, obtains user data and writes the user data to the memory of the external host through the communication bus; after the target hard disk completes the user data write operation, it completes writing the first read command to the completion queue cache of the controller through the communication bus.

[0011] In one of the embodiments, the completion queue cache includes an I / O completion queue cache; each target hard disk completes writing the first read command to the I / O completion queue cache of the controller via a communication bus; when the controller detects the write operation of the I / O completion queue cache via a built-in monitoring circuit, it obtains the hard disk identification, completion queue sequence number and related information of the first read command completion of the corresponding target hard disk, processes the first read command completion, writes the second read command completion to the external host via the communication bus after the processing is completed, and sends an interrupt signal to the external host.

[0012] In one embodiment, each hard disk and controller adopts a program-controlled input / output transmission mode or a direct memory operation transmission mode and is connected to an external host through a communication bus; each hard disk and controller adopts a point-to-point transmission mode and is connected through a communication bus.

[0013] The present application also provides a data storage system, comprising any of the data storage devices described above, and also comprising a host; each hard disk and controller of the independent redundant disk array in the data storage device are connected to the host via a communication bus as endpoint devices.

[0014] The present application also provides a data storage method, which is applied to a data storage device, wherein the data storage device includes an independent redundant disk array and a controller of the independent redundant disk array, wherein each hard disk in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices, and a monitoring circuit is built in the controller; the data storage method includes: the controller obtains a target command from the external host through the communication bus, parses the target command, caches the parsed target command into a command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk through the communication bus; after each target hard disk receives the notification of processing a new command, it reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and writes the first command completion to the completion queue cache of the controller through the communication bus after the processing is completed; the controller monitors the completion queue cache of the controller through the built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command completion is processed, and after the processing is completed, the second command completion is pushed to the external host through the communication bus; wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host for user data through the communication bus.

[0015] In the above-mentioned data storage method, data storage device and data storage system, each hard disk and controller in the independent redundant disk array are connected to the external host through a communication bus as endpoint devices, and a monitoring circuit is built in the controller; the controller obtains the target command from the external host through the communication bus, parses the target command, caches the parsed target command into the command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk through the communication bus; after each target hard disk receives the notification of processing a new command, it reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and after the processing is completed, writes the first command to the completion queue cache of the controller through the communication bus; the controller monitors the completion queue cache of the controller through the built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command written is completed and processed, and after the processing is completed, the second command is completed and pushed to the external host through the communication bus; wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host for user data through the communication bus.

[0016] Therefore, each hard disk and its controller in the independent redundant disk array are connected to the communication bus of the external host in parallel. The command initiated by the external host passes through the RAID controller, which is processed by the RAID controller and then forwarded to each hard disk. All user data of each hard disk in response to the command can directly interact with the external host through the communication bus. The transmission channels of the external host's command and the user data in response to the command are separated. The user data transmitted between the external host and the independent redundant disk array no longer passes through the RAID controller, thereby breaking through the performance limitation of the interface bandwidth of the RAID controller on the independent redundant disk array, shortening the response delay of the independent redundant disk array to the command of the external host, and improving the data access efficiency of the independent redundant disk array. The RAID controller is connected to the external host communication bus only through one bus interface, which is more conducive to the RAID controller to achieve the goal of reducing power consumption and optimizing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A structural block diagram of a data storage device of an independent redundant disk array in one embodiment;

[0018] Figure 2 A schematic diagram of data transmission in which a host writes user data to each SSD in one embodiment;

[0019] Figure 3 A schematic diagram of data transmission in which a RAID controller acquires user data from a host and an SSD to generate verification data in one embodiment;

[0020] Figure 4 A schematic diagram of data transmission in which a RAID controller obtains remaining data from an SSD for data recovery in one embodiment;

[0021] Figure 5 A schematic diagram of data transmission in which a RAID controller stores restored user data to a newly added SSD of a RAID in one embodiment;

[0022] Figure 6 A schematic diagram of data transmission in which a host reads user data from each SSD in an embodiment;

[0023] Figure 7 A schematic diagram of transmission of I / O control commands of a host, a RAID controller, and each SSD in one embodiment;

[0024] Figure 8 The figure is a flow chart of a data storage method in one embodiment. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] The present application provides a data storage device. In one embodiment, Figure 1 As shown, a data storage device includes an independent redundant disk array (RAID) and a controller of the independent redundant disk array (RAID controller). Each solid state drive in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices. Figure 1 As shown in the figure, each solid-state disk (SSD) of the independent redundant disk array includes , . Each solid-state drive and RAID controller are connected to the external host in parallel as endpoint devices. Among them, the solid-state drive can communicate using the NVMe (Non Volatile Memory Host Controller Interface Specification) protocol, and the external host is an NVMe host. The communication bus is the PCIe (peripheral component interconnect express, a high-speed serial computer expansion bus standard) bus. Among them, the controller of the independent redundant disk array has a built-in monitoring circuit for monitoring the read and write operations of each solid-state drive on the submission queue SQ and completion queue CQ in the controller, and generating relevant internal interrupt signals.

[0027] Specifically, each solid-state drive in the above-mentioned independent redundant disk array is connected in parallel with the controller of the independent redundant disk array to the communication bus of the external host. The controller obtains the target command from the external host through the communication bus, parses the target command, caches the parsed target command into the command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target solid-state drive through the communication bus; after receiving the notification of processing a new command, each target solid-state drive reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and after the processing is completed, writes the first command to the completion queue cache of the controller through the communication bus; the controller monitors the completion queue cache of the controller through the built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command written is processed, and after the processing is completed, the second command is pushed to the external host through the communication bus; wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host for user data through the communication bus.

[0028] Specifically, after each solid state drive in the independent redundant disk array and the controller of the independent redundant disk array are connected in parallel to the communication bus of the external host, a system initialization process needs to be performed. For example, the system initialization process includes the following operations:

[0029] (1) The external host performs PCIe initialization on the RAID controller and each SSD, such as identifying the device type, manufacturer, capability structure and other information, and assigning BDF number, BAR address, etc.

[0030] (2) The external host sends the SSD initialization information, such as the BAR address, to the RAID controller.

[0031] (3) The RAID controller initializes NVMe for each SSD.

[0032] (4) The external host initializes NVMe on the RAID controller.

[0033] (5) The external host performs IO operations on the RAID, and the RAID controller processes and responds to the host's IO operations.

[0034] Specifically, according to the NVMe protocol specification, after the SSD executes the command and writes the command completion entry to the completion queue CQ, it needs to send an interrupt signal to the external host to notify the external host to perform subsequent processing. In the architecture of the parallel RAID controller of the present application, the RAID controller appears as an external host to the SSD, but because the RAID controller is a PCIe endpoint device in the system, it is unable to receive and process the interrupt signal of the SSD, so it is necessary to internally monitor the operation of the SSD on the completion queue CQ in the RAID controller and generate an interrupt signal inside the RAID controller. Therefore, a monitoring circuit is designed in the RAID controller, namely the SQCQ_Monitor module, which is used to monitor the read and write operations of the SSD on the submission queue SQ and the completion queue CQ, and generate relevant internal interrupt signals to notify the internal software and hardware of the controller to perform relevant processing.

[0035] In this embodiment, the RAID controller exists in the system as a terminal node, but realizes the dual functions of NVMe host and controller. For the system host, the RAID controller behaves as an NVMe controller, realizing the function of virtual NVMe SSD; for the physical NVMe SSD, the RAID controller behaves as an NVMe host, realizing the management and read-write control of the physical NVMe SSD through the PCIe peer to peer transmission mode. Specifically, the RAID controller is responsible for implementing the control logic of RAID. The RAID controller is responsible for obtaining the host's read and write commands from the external host, converting and splitting the host's read and write commands into read and write commands for multiple physical SSDs, and distributing the commands to each physical disk, thereby utilizing the concurrent operations of multiple physical SSDs to improve access performance.

[0036] In one embodiment, each solid-state drive and controller adopts a programmable input / output transmission mode or a direct memory operation transmission mode and is connected to an external host through a communication bus; each solid-state drive and controller adopts a point-to-point transmission mode and is connected through a communication bus.

[0037] Specifically, Figure 1 As shown, the external host communicates with the RAID controller and each SSD through a communication bus, and can use the PCIe PIO (Programming Input / Output Model, a data exchange mode in which the CPU executes I / O port instructions to read and write data) / DMA (Programmed IO / Direct Memory Access, direct memory operation) transmission mode for normal communication, and the RAID controller can also communicate with each SSD through the PCIe peer to peer transmission mode.

[0038] In one embodiment, if the target command is a management command, each target solid-state hard disk responds to the parsed management command; if each target hard disk needs to return management data to the external host, the returned first management data is written to the designated cache area of ​​the controller through the communication bus; after each target hard disk responds to the parsed management command and completes the operation, the first management command is completed and written to the completion queue cache of the controller through the communication bus. In one example, the completion queue cache includes a management completion queue cache; after the response operation of the parsed management command is completed, each target hard disk completes the writing of the first management command to the management completion queue cache of the controller through the communication bus; when the controller detects the write operation of the management completion queue cache through the built-in monitoring circuit, it identifies the hard disk identification of the corresponding target hard disk and determines that if the management command needs to return management data to the external host, the first management data returned by the corresponding target hard disk is read and processed from the designated cache area, and the second management data is written to the cache designated by the external host through the communication bus; the controller processes the completion of the written first management command, and after the processing is completed, the second management command is completed and written to the external host through the communication bus, and an interrupt signal is sent to the external host.

[0039] Specifically, according to the NVMe protocol specification, the interactive commands between the external host and the SSD are roughly divided into management commands and I / O commands. The processing flow of management commands is as follows:

[0040] (1) The external host writes the management command to the local command submission queue cache and sends a doorbell to notify the RAID controller that there is a new command to be processed.

[0041] (2) The RAID controller obtains new management commands from the command submission queue cache of the external host.

[0042] (3) After the RAID controller parses and converts the management command, it writes it into the command submission queue cache of the local corresponding physical SSD of the RAID controller, and sends a doorbell to notify the corresponding physical SSD that there is a new command to be processed.

[0043] (4) The physical SSD obtains new management commands from the command submission queue cache of the RAID controller.

[0044] (5) The physical SSD parses and processes the management command. If there is management data to be returned, the management data is written to the RAID controller cache specified in the management command.

[0045] (6) The physical SSD responds to the management command, and after completing the management operation, writes the management command completion to the management command completion queue cache of the RAID controller.

[0046] (7) The RAID controller detects the writing operation of the management command completion queue through the internal monitoring circuit and makes a judgment. If there is management data to be returned, the management data returned by the physical SSD will be processed and written to the cache address specified in the command of the external host.

[0047] (8) After the RAID controller parses and processes the command completion of the physical SSD, it generates a command completion to be returned to the external host, writes it to the completion queue cache of the external host, and sends an interrupt to notify the external host that the command has been processed.

[0048] In one embodiment, if the target command is a write command, each target hard disk responds to the parsed write command, reads user data from the memory of the external host or reads verification data from the memory of the controller through the communication bus, and writes it to the target hard disk; after the target hard disk completes the data write operation, the first write command is completed and written to the completion queue cache of the controller through the communication bus. In one example, the completion queue cache includes an I / O completion queue cache; each target hard disk completes writing the first write command to the I / O completion queue cache of the controller through the communication bus; when the controller detects the write operation of the I / O completion queue cache through the built-in monitoring circuit, it obtains the hard disk identification of the corresponding target hard disk, the completion queue sequence number, and the relevant information of the completion of the first write command, processes the completion of the first write command, and after the processing is completed, writes the second write command to the external host through the communication bus, and sends an interrupt signal to the external host.

[0049] Specifically, when user data needs to be written to the SSD, the external host pushes a write command to the controller. The controller parses and / or splits the write command from the external host and pushes it to the corresponding SSD. The corresponding SSD directly reads the user data from the external host memory or reads the verification data from the controller memory through the communication bus and writes it to the SSD. After completion, the SSD writes the write command to the I / O completion queue cache of the controller. In addition, if the controller detects the write operation of the I / O completion queue cache through the built-in monitoring circuit, it extracts the SSD's hard disk ID, completion queue sequence number ID, and related information of the write command completion and pushes it to the hardware inside the controller for corresponding processing.

[0050] Specifically, when the external host writes user data, the SSD directly reads the user data from the memory of the external host to the SSD through a communication bus, such as a PCIe Bus. Figure 2As shown in the figure. Since user data does not pass through the RAID controller, the response delay of I / O control commands can be greatly reduced, improving the access performance of RAID. More importantly, the user data interaction between the host and SSD is no longer limited by the interface bandwidth of the RAID controller itself, so the performance of the SSD can be fully released.

[0051] In one embodiment, when the target command is a write command, the controller of the independent redundant disk array obtains the user data to be written from the memory of the external host, and determines the data storage block of the target sort number of the first target solid-state hard disk, reads the written user data from the data storage block of the same target sort number in the second target solid-state hard disk, generates verification data based on the user data to be written and the user data already written and based on the set operation rules, and stores the verification data in the data storage block of the same target sort number of the third target solid-state hard disk. Any first target solid-state hard disk, second target solid-state hard disk and third target solid-state hard disk are different solid-state hard disks and may be a single or multiple solid-state hard disks. The first target solid-state hard disk pulls the user data to be written from the memory of the external host through the communication bus and writes it into the data storage block of the corresponding target sort number.

[0052] Among them, after the first target solid-state hard disk writes the user data to be written into the data storage block of the target sorting number, when the controller of the independent redundant disk array receives a read command transmitted by the external host communication bus to read the user data of the data storage block of the target sorting number of the first target solid-state hard disk, if the user data of the data storage block of the target sorting number of the first target solid-state hard disk is lost, the user data in the data storage block of the target sorting number of the second target solid-state hard disk and the verification data in the data storage block of the target sorting number of the third target solid-state hard disk are read, and the user data in the data storage block of the target sorting number of the second target solid-state hard disk and the verification data in the data storage block of the target sorting number of the third target solid-state hard disk are calculated based on the inverse operation of the set operation rule to restore the user data of the data storage block of the target sorting number of any first target solid-state hard disk, and the restored user data is transmitted to the host through the communication bus.

[0053] In this embodiment, when the external host sends a write command to the RAID controller, the RAID controller needs to generate verification data for the user data to be written, so that when a storage failure occurs in the subsequent SSD and the user data is lost, data recovery can be performed through the corresponding verification data. The first target solid-state hard disk reads the user data to be written from the memory of the external host through the communication bus and writes it into the data storage block of the target sequence number.

[0054] After the user data is written to the data storage block of the target sequence number of the first target solid-state hard disk, if the RAID controller receives a read command for the user data of the data storage block of the target sequence number of the first target solid-state hard disk, it recognizes that the data storage block of the target sequence number of the first target solid-state hard disk is lost due to a failure, then the complete user data is restored through the user data of the data storage block of the target sequence number of the second target solid-state hard disk and the verification data of the data storage block of the target sequence number of the third target solid-state hard disk and based on the inverse operation of the set operation rules, and the restored user data is transmitted to an external host, thereby ensuring the integrity of the user data and avoiding the loss of user data due to SSD failure.

[0055] In one embodiment, when the controller of an independent redundant disk array recognizes that any first target solid-state drive to which user data and / or verification data has been written has lost data due to a failure and determines to add a new solid-state drive in the independent redundant disk array, the controller recovers the user data and / or verification data of the failed first target solid-state drive by performing an inverse operation based on a set operation rule using the remaining user data and / or verification data in solid-state drives other than the first target solid-state drive, and stores the recovered user data and / or verification data in the newly added solid-state drive.

[0056] In this embodiment, the RAID controller can use the method of the above embodiment to recover the user data and / or verification data of any solid-state hard drive, and when confirming the addition of a new solid-state hard drive to the RAID, the recovered user data and / or verification data will be stored in the new solid-state hard drive, thereby avoiding the loss of user data.

[0057] Specifically, the RAID controller is responsible for implementing the operations and inverse operations related to the RAID check data, and using data redundancy to achieve security protection for user data. Figure 3 As shown in the figure, the RAID controller obtains user data from the external host and SSD, calculates and generates verification data according to the corresponding rules, and writes it to the corresponding SSD to provide redundant data protection. Figure 4 The RAID controller obtains the remaining data and / or verification data from the SSD, performs inverse operations according to corresponding rules to recover the lost user data, and returns the recovered user data to the external host. Figure 5 The recovered user data and / or verification data may also be stored in a newly added SSD of the RAID to rebuild the RAID.

[0058] Specifically, the data processing flow of the write command is:

[0059] (1) The external host writes the write command to the local command submission queue cache and sends a doorbell to notify the RAID controller that there is a new command to be processed.

[0060] (2) The RAID controller obtains new commands from the command submission queue of the external host.

[0061] (3) After the RAID controller parses, converts, and splits the command, it writes it into the command submission queue cache of the corresponding physical SSD in the RAID controller, and sends a doorbell to notify the corresponding physical SSD that there is a new command to be processed.

[0062] (4) If the configured RAID level requires calculation of verification data, the RAID controller simultaneously obtains relevant user data from the external host and the physical SSD, calculates and generates verification data and writes it to the local cache, writes the command to write the verification data to the command submission queue cache of the local corresponding physical SSD, and sends a doorbell to notify the corresponding physical SSD that there is a new command to be processed.

[0063] (5) Each physical SSD reads new commands from the command submission queue cache of the RAID controller.

[0064] (6) Each physical SSD parses and processes the command, and reads the user data to be written from the cache address of the specified external host to the local physical SSD for storage, or writes the user data stored in the SSD to the specified RAID controller cache for calculating the verification data, or reads the verification data to be written from the specified RAID controller cache to the local physical SSD for storage.

[0065] (7) Each physical SSD writes the write command completion to the completion queue cache of the RAID controller.

[0066] (8) After the RAID controller detects the write operation of the I / O completion queue through the internal monitoring circuit, it parses, merges, and processes the write command completion of each physical SSD, writes it to the completion queue cache of the external host, and sends an interrupt to notify the external host that the command has been processed.

[0067] In one embodiment, if the target command is a read command, each target hard disk responds to the parsed read command, obtains user data and writes the user data into the memory of the external host through the communication bus; after the target hard disk completes the user data write operation, the first read command is completed and written into the completion queue cache of the controller through the communication bus. In one example, the completion queue cache includes an I / O completion queue cache; each target hard disk completes writing the first read command into the I / O completion queue cache of the controller through the communication bus; when the controller detects the write operation of the I / O completion queue cache through the built-in monitoring circuit, it obtains the hard disk identification of the corresponding target hard disk, the completion queue sequence number and the related information of the completion of the first read command, processes the completion of the first read command, and after the processing is completed, writes the second read command into the external host through the communication bus, and sends an interrupt signal to the external host.

[0068] When the external host reads user data, the SSD directly writes the user data from the SSD to the memory of the external host through a communication bus, such as PCIe Bus. Figure 6 As shown in the figure. Since user data does not pass through the RAID controller, the response delay of I / O control commands can be greatly reduced, improving the access performance of RAID. More importantly, the user data interaction between the external host and the SSD is no longer limited by the PCIe interface bandwidth of the RAID controller itself, so the performance of the SSD can be fully released.

[0069] Specifically, the read command processing flow:

[0070] (1) The external host writes the read command to the local command submission queue cache and sends a doorbell to notify the RAID controller that there is a new command to be processed.

[0071] (2) The RAID controller obtains new commands from the command submission queue of the external host.

[0072] (3) After the RAID controller parses, converts, and splits the command, it writes it into the command submission queue cache of the corresponding physical SSD in the RAID controller, and sends a doorbell to notify the corresponding physical SSD that there is a new command to be processed.

[0073] (4) Each physical SSD obtains new commands from the command submission queue cache of the RAID controller.

[0074] (5) Each physical SSD parses and processes the command, and writes the user data read by the external host to the cache address specified by the external host.

[0075] (6) Each physical SSD writes the read command completion to the completion queue cache of the RAID controller.

[0076] (7) When the RAID controller detects the write operation of the I / O completion queue cache through the built-in monitoring circuit, it parses, merges and processes the command completion of each physical SSD, writes it to the completion queue cache of the external host, and sends an interrupt to notify the external host that the command has been processed.

[0077] In the above embodiment, the channel for the external host to access the I / O control command of each solid-state hard disk in the RAID is separated from the channel for transmitting user data in response to the I / O control command. Figure 7 As shown, after the I / O control command of the external host accessing the RAID is converted by the RAID controller, the RAID controller and the SSD interact through a point-to-point communication method.

[0078] The above-mentioned data storage device connects each solid-state hard disk and its controller in the independent redundant disk array to the communication bus of the external host in parallel. The command initiated by the external host passes through the RAID controller, and is processed by the RAID controller and then forwarded to each solid-state hard disk. All user data of each solid-state hard disk in response to the command can be directly transmitted through the communication bus. The transmission channels of the external host's command and the user data of the response command are separated, and the user data transmitted between the external host and the independent redundant disk array no longer passes through the RAID controller, thereby breaking through the performance limitation of the interface bandwidth of the RAID controller on the independent redundant disk array, shortening the response delay of the independent redundant disk array to the command of the external host, and improving the data access efficiency of the independent redundant disk array. The RAID controller is connected to the external host communication bus only through a bus interface, which is more conducive to the RAID controller to achieve the goal of reducing power consumption and optimizing costs.

[0079] In summary, the above data storage device:

[0080] (1) Both the RAID controller and each SSD are PCIe terminal devices, and they are connected to the PCIe bus of the external host in parallel. The external host communicates with the RAID controller and each SSD via the PCIe bus in PIO / DMA transmission mode, while the RAID controller communicates with each SSD via the PCIe bus in PCIe point-to-point transmission mode.

[0081] (2) The read / write command path for the external host to access each SSD in the RAID is separated from the user data path for responding to the read / write command. The read / write command for the external host to access each SSD in the RAID is converted and forwarded by the RAID controller, and the RAID controller and each SSD interact through the PCIe point-to-point communication method; the read / write user data between the external host and each SSD is directly interacted through the PCIe bus without passing through the RAID controller.

[0082] (3) When the external host writes user data to each SSD in the RAID, the RAID controller reads the relevant user data from the external host and the corresponding SSD, calculates and generates verification data according to the corresponding rules, and controls the writing of the verification data to the corresponding SSD.

[0083] (4) When the RAID group loses part of its data and the external host needs to read the lost user data from the RAID, the RAID controller reads the remaining data and / or verification data from the SSD, performs inverse operations according to the corresponding rules to restore the lost user data, and sends the restored user data to the external host.

[0084] (5) When a RAID group loses part of its data and a new solid-state drive is added to the RAID group for RAID reconstruction, the RAID controller reads the remaining data and / or verification data from the solid-state drive, performs inverse operations according to corresponding rules to recover the lost user data and / or verification data, and writes the recovered user data and / or verification data to the new solid-state drive.

[0085] The present application also provides a data storage system. A data storage system includes the data storage device of any of the above embodiments, and also includes a host. Each solid state hard disk and controller of the independent redundant disk array in the data storage device are connected to the host through a communication bus as endpoint devices.

[0086] In the above data storage system, the RAID controller separates the user data path for accessing the RAID from the RAID controller, reduces the response delay of the command, breaks through the limitation of the RAID controller interface bandwidth on the overall performance of the RAID, and fully releases the performance of the NVMe SSD in the RAID group to achieve the ultimate RAID read and write performance. In addition, the traditional RAID controller requires more than two PCIe ports to be connected to the host and SSD respectively. In this application, the RAID controller is connected to the PCIe Bus through only one PCIe port, which reduces the number of PCIe ports compared to the traditional RAID controller, and is therefore more conducive to the RAID controller to achieve the goal of reducing power consumption and optimizing costs.

[0087] The present application also provides a data storage method, which is applied to a data storage device. The data storage device includes an independent redundant disk array and a controller of the independent redundant disk array. Each solid-state hard disk in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices, and a monitoring circuit is built into the controller. In one embodiment, Figure 8 As shown, a data storage method includes the following steps:

[0088] S802, the controller obtains a target command from an external host via a communication bus, parses the target command, caches the parsed target command into a command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk via the communication bus.

[0089] S804, after receiving the notification of processing the new command, each target hard disk reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and after the processing is completed, writes the first command to the completion queue cache of the controller through the communication bus.

[0090] S806, the controller monitors the completion queue cache of the controller through the built-in monitoring circuit. If a write operation of the completion queue cache is detected, the first command written is processed. After the processing is completed, the second command is pushed to the external host through the communication bus.

[0091] If the target command is an I / O command, each target hard disk directly exchanges user data with the external host through the communication bus.

[0092] In this embodiment, the execution operations of the controller of the independent redundant disk array and its solid state hard disks refer to the operations mentioned in the various embodiments of the above-mentioned data storage device, which will not be described in detail here.

[0093] It should be understood that, although the steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0094] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data storage device, characterized in that: The data storage device includes an independent redundant disk array and a controller of the independent redundant disk array. Each hard disk in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices, and a monitoring circuit is built in the controller; wherein, The controller obtains a target command from the external host through the communication bus, parses the target command, caches the parsed target command in a command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk through the communication bus; After receiving the notification of processing the new command, each target hard disk reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and after the processing is completed, writes the first command to the completion queue cache of the controller through the communication bus; The controller monitors the completion queue cache of the controller through a built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command completion of the write is processed, and after the processing is completed, the second command completion is pushed to the external host through the communication bus; Wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host via a communication bus to exchange user data; Wherein, if the target command is a management command, each target hard disk responds to the parsed management command; If each target hard disk needs to return management data to the external host, the returned first management data is written into the designated buffer area of ​​the controller through the communication bus; After each target hard disk responds to the parsed management command and completes the operation, the first management command is completed and written into the completion queue cache of the controller through the communication bus; Wherein, the completion queue cache includes managing the completion queue cache; After completing the response operation of the parsed management command, each target hard disk writes the first management command into the management completion queue cache of the controller through the communication bus; When the controller detects the write operation of the management completion queue buffer through the built-in monitoring circuit, it identifies the hard disk identifier of the corresponding target hard disk and determines that if the management command needs to return management data to the external host, it reads and processes the first management data returned by the corresponding target hard disk from the designated buffer area, and writes the second management data into the buffer designated by the external host through the communication bus; The controller processes the first management command after it is written, writes the second management command to the external host through the communication bus after the processing is completed, and sends an interrupt signal to the external host.

2. The data storage device according to claim 1, characterized in that If the target command is a write command, each target hard disk responds to the parsed write command, reads user data from the memory of the external host or reads verification data from the memory of the controller through the communication bus, and writes the data into the target hard disk; After the target hard disk completes the data writing operation, the first write command is written into the completion queue buffer of the controller through the communication bus.

3. The data storage device according to claim 2, characterized in that: The completion queue cache includes an I / O completion queue cache; Each target hard disk completes writing the first write command into the I / O completion queue cache of the controller through the communication bus; When the controller monitors the write operation of the I / O completion queue cache through the built-in monitoring circuit, it obtains the hard disk identification of the corresponding target hard disk, the completion queue number and the relevant information of the completion of the first write command, processes the completion of the first write command, and after the processing is completed, writes the second write command completion to the external host through the communication bus, and sends an interrupt signal to the external host.

4. The data storage device according to claim 1, characterized in that: If the target command is a read command, each target hard disk responds to the parsed read command, obtains the user data, and writes the user data into the memory of the external host through the communication bus; After completing the user data writing operation, the target hard disk completes writing the first read command into the completion queue buffer of the controller through the communication bus.

5. The data storage device according to claim 4, characterized in that: The completion queue cache includes an I / O completion queue cache; Each target hard disk completes writing the first read command into the I / O completion queue cache of the controller via the communication bus; When the controller monitors the write operation of the I / O completion queue cache through the built-in monitoring circuit, it obtains the hard disk identification of the corresponding target hard disk, the completion queue number and the relevant information of the completion of the first read command, processes the completion of the first read command, and after the processing is completed, writes the second read command completion to the external host through the communication bus, and sends an interrupt signal to the external host.

6. The data storage device according to claim 1, characterized in that: Each hard disk and the controller adopt a program-controlled input / output transmission mode or a direct memory operation transmission mode and are connected to an external host through a communication bus; Each hard disk and the controller adopt a point-to-point transmission mode and are connected through a communication bus.

7. A data storage system, characterized in that: The data storage device comprising any one of claims 1 to 6, further comprising a host; Each hard disk and controller of the independent redundant disk array in the data storage device are connected to the host through a communication bus as an endpoint device.

8. A data storage method, characterized in that: Applicable to a data storage device, the data storage device includes an independent redundant disk array and a controller of the independent redundant disk array, each hard disk in the independent redundant disk array and the controller are connected to an external host through a communication bus as endpoint devices, and a monitoring circuit is built in the controller; The data storage method comprises: The controller obtains a target command from the external host through the communication bus, parses the target command, caches the parsed target command in a command queue cache of one or more target hard disks in the controller, and sends a notification of processing a new command to each target hard disk through the communication bus; After receiving the notification of processing the new command, each target hard disk reads the parsed target command from the command queue cache of the controller through the communication bus for processing, and after the processing is completed, writes the first command to the completion queue cache of the controller through the communication bus; The controller monitors the completion queue cache of the controller through a built-in monitoring circuit, and if a write operation of the completion queue cache is detected, the first command written is processed, and after the processing is completed, the second command is pushed to the external host through a communication bus; Wherein, if the target command is an I / O command, each target hard disk directly interacts with the external host via a communication bus to exchange user data; Wherein, if the target command is a management command, each target hard disk responds to the parsed management command; If each target hard disk needs to return management data to the external host, the returned first management data is written into the designated buffer area of ​​the controller through the communication bus; After each target hard disk responds to the parsed management command and completes the operation, the first management command is completed and written into the completion queue cache of the controller through the communication bus; Wherein, the completion queue cache includes managing the completion queue cache; After completing the response operation of the parsed management command, each target hard disk writes the first management command into the management completion queue cache of the controller through the communication bus; When the controller detects the write operation of the management completion queue buffer through the built-in monitoring circuit, it identifies the hard disk identifier of the corresponding target hard disk and determines that if the management command needs to return management data to the external host, it reads and processes the first management data returned by the corresponding target hard disk from the designated buffer area, and writes the second management data into the buffer designated by the external host through the communication bus; The controller processes the first management command after it is written, writes the second management command to the external host through the communication bus after the processing is completed, and sends an interrupt signal to the external host.

Images