Storage device read-write method and device, storage medium, and program product

Abstract

This disclosure relates to the field of computer technology and discloses a method, apparatus, device, storage medium, and program product for reading and writing storage devices. The method includes: responding to a processing instruction for a target submission queue at the front end of the storage device, mapping the target submission queue to a first submission queue and a second submission queue at the back end of the storage device, wherein the target submission queue includes at least one of read commands and write commands; reading read commands from the first submission queue using a first data transmission channel and writing the read read commands to a read buffer; reading write commands from the second submission queue using a second data transmission channel and writing the read write commands to a write buffer; processing the read commands in the read buffer to perform a read operation; and processing the write commands in the write buffer to perform a write operation. This disclosure can solve the problem of head-of-queue blocking caused by reading and writing rate limiting of storage devices.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to methods, apparatus, devices, storage media, and program products for reading and writing storage devices. Background Technology

[0002] In related technologies, read and write speed limits are often applied to storage devices to improve data security, optimize system performance, facilitate storage resource management, and control storage costs. However, during the process of limiting read and write speeds of storage devices, the backend of the storage device has difficulty knowing whether the next operation to be processed is a read or a write operation. Therefore, if a write operation is limited, the queued read operations will be blocked, and vice versa, resulting in congestion at the head of the queue. Summary of the Invention

[0003] In view of this, the present disclosure provides a method, apparatus, device, storage medium, and program product for reading and writing storage devices to solve the problem of head-of-line congestion caused by limiting the read and write speed of storage devices.

[0004] In a first aspect, this disclosure provides a method for reading and writing to a storage device, the method comprising:

[0005] In response to a processing instruction for a target commit queue at the front end of the storage device, the target commit queue is mapped to a first commit queue and a second commit queue at the back end of the storage device, wherein the target commit queue includes at least one of a read command and a write command;

[0006] The first data transmission channel is used to read the read command from the first submission queue, and the read read command is written to the read buffer;

[0007] The write command in the second submission queue is read using the second data transmission channel, and the read write command is written to the write buffer;

[0008] The read commands in the read buffer are processed to perform a read operation;

[0009] The write commands in the write buffer are processed to perform the write operation.

[0010] Secondly, this disclosure provides a storage device read / write apparatus, the apparatus comprising:

[0011] A queue mapping module is configured to, in response to a processing instruction for a target submission queue at the front end of the storage device, map the target submission queue to a first submission queue and a second submission queue at the back end of the storage device, wherein the target submission queue includes at least one of a read command and a write command;

[0012] The read command writing module is used to read the read commands in the first submission queue using the first data transmission channel and write the read read commands to the read buffer;

[0013] The write command writing module is used to read the write commands in the second submission queue using the second data transmission channel and write the read write commands to the write buffer;

[0014] The read command processing module is used to process the read commands in the read buffer to perform read operations;

[0015] The write command processing module is used to process the write commands in the write buffer to perform write operations.

[0016] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the memory device read / write method described in the first aspect.

[0017] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions that cause a computer to perform the storage device read / write method described in the first aspect.

[0018] Fifthly, this disclosure provides a computer program product, including computer instructions for causing a computer to execute the storage device read / write method described in the first aspect.

[0019] The storage device read / write method provided in this disclosure utilizes a first data transmission channel and a second data transmission channel to map the target submission queue at the front end of the storage device into a first submission queue and a second submission queue at the back end, so as to process read commands and write commands respectively. Therefore, read rate-limiting scenarios and write rate-limiting scenarios do not affect each other, and can effectively solve the problem of head-of-queue blocking in storage device rate-limiting scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of a first storage device read / write method according to an embodiment of the present disclosure;

[0022] Figure 2This is a schematic diagram of a read / write rate limiting method according to an embodiment of the present disclosure;

[0023] Figure 3 This is a schematic diagram of a buffer according to an embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of another related technology read / write rate limiting method according to an embodiment of this disclosure;

[0025] Figure 5 This is a schematic diagram of a buffer according to another related technology in accordance with an embodiment of this disclosure;

[0026] Figure 6 This is a schematic diagram of a read / write rate limiting method according to an embodiment of the present disclosure;

[0027] Figure 7 This is a schematic flowchart of a second storage device read / write method according to an embodiment of the present disclosure;

[0028] Figure 8 This is a schematic diagram of a target submission queue according to an embodiment of the present disclosure;

[0029] Figure 9 This is a schematic diagram illustrating command writing and processing in a read buffer and a write buffer according to an embodiment of the present disclosure;

[0030] Figure 10 This is a schematic diagram of a third storage device read / write method according to an embodiment of the present disclosure;

[0031] Figure 11 This is a flowchart illustrating a fourth method for reading and writing a storage device according to an embodiment of the present disclosure;

[0032] Figure 12 This is a flowchart illustrating a fifth method for reading and writing a storage device according to an embodiment of the present disclosure;

[0033] Figure 13 This is a structural block diagram of a storage device read / write apparatus according to an embodiment of the present disclosure;

[0034] Figure 14 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0036] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0037] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0038] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0039] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0040] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0041] In storage scenarios, limiting the read and write speeds of storage devices offers the following advantages: First, it improves data security. Limiting the write speed of storage devices prevents data from being written too quickly, thus reducing the risk of data loss or corruption. It also prevents malware or hacker attacks from rapidly reading large amounts of data to obtain sensitive information. Second, it optimizes system performance. Limiting the write speed of storage devices avoids performance degradation caused by excessively fast data writing during high-concurrency writes. Limiting the read speed ensures system stability and responsiveness during heavy concurrent reads. Third, it facilitates storage resource management. Limiting the read and write speeds of storage devices helps manage storage resources, preventing excessive use of storage resources by a particular application or user, which could impact the performance of other applications or users. Fourth, it controls storage costs. Limiting the read and write speeds of storage devices helps control storage costs by reducing data transfer volume and the load on storage devices, thereby lowering storage costs.

[0042] Therefore, limiting the read and write speeds of storage devices can improve data security, optimize system performance, facilitate the management of storage resources, and control storage costs.

[0043] In related technologies, there are two main read / write rate limiting schemes. The first scheme places all input / output (I / O) commands received from the front end into a shared buffer. Since write and read commands share the same buffer, if a write command is currently waiting at the queue exit, and the write operation is restricted, the queue will be in a waiting state. Read commands in the queue will be intercepted, causing head-of-line blocking and resulting in significant fluctuations in read rate limiting. The second scheme divides the command buffer into two parts, with read and write commands each occupying a separate storage space. Assume the storage space corresponding to a write command is full, while the storage space corresponding to a read command is empty. However, since it's unknown whether the next operation type on the host side will be a read or write command, backpressure blocking will occur. Only after a write operation processes a portion of the data to release the buffer can new commands be read, and only then can read operations be processed. Therefore, the second read / write rate limiting scheme still suffers from head-of-line blocking.

[0044] Therefore, it is evident that the methods of limiting the read and write speeds of storage devices in related technologies are prone to head-of-line congestion.

[0045] In view of the above, according to the embodiments of this disclosure, an embodiment of a storage device read / write method is provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] This embodiment provides a first method for reading and writing storage devices, which can be used in the backend of the aforementioned storage device. Figure 1 This is a flowchart illustrating a first storage device read / write method according to an embodiment of the present disclosure, as shown below. Figure 1 As shown, the process includes the following steps:

[0047] Step S101, in response to a processing instruction for a target commit queue at the front end of the storage device, the target commit queue is mapped to a first commit queue and a second commit queue at the back end of the storage device, wherein the target commit queue includes at least one of a read command and a write command.

[0048] Specifically, the input / output (IO) commands of the Non-Volatile Memory Host Controller Interface Specification (NVME) protocol are stored in the target submission queue (SQ) at the front end of the storage device. The target submission queue is a ring based on head and tail pointers, and both read and write commands are stored in the target submission queue.

[0049] It should be noted that the first commit queue, the second commit queue, and the target commit queue are the same.

[0050] Step S102: Use the first data transmission channel to read the read command in the first submission queue and write the read read command to the read buffer.

[0051] Specifically, the first data transmission channel is a Direct Memory Access (DMA) channel.

[0052] Specifically, the first data transmission channel reads commands from the first submission queue and determines the type of the command being read. If the command being read is a read command, the read command is written to the read buffer.

[0053] Step S103: Use the second data transmission channel to read the write command in the second submission queue and write the read write command to the write buffer.

[0054] Specifically, the second data transmission channel is a Direct Memory Access (DMA) channel.

[0055] Specifically, the second data transmission channel reads commands from the second submission queue and determines the type of the command read. If the command read is a write command, the read write command is written to the write buffer.

[0056] Understandably, if the first data transmission channel and the second data transmission channel can be used to read the first submission queue and the second submission queue respectively, then the commands pointed to by the current head pointer of the submission queue corresponding to each data transmission channel will not be blocked due to read / write rate limits or insufficient storage space in the buffer.

[0057] Step S104: Process the read commands in the read buffer to perform a read operation.

[0058] Specifically, the first data transmission channel is used to read the read command from the read buffer to perform the read operation.

[0059] Step S105: Process the write commands in the write buffer to perform the write operation.

[0060] Specifically, the write command is read from the write buffer using the second data transmission channel to perform the write operation.

[0061] The storage device read / write method provided in this embodiment utilizes a first data transmission channel and a second data transmission channel to map the target submission queue at the front end of the storage device into a first submission queue and a second submission queue at the back end, so as to process read commands and write commands respectively. Therefore, read rate-limiting scenarios and write rate-limiting scenarios do not affect each other, and can effectively solve the problem of head-of-queue blocking in storage device rate-limiting scenarios.

[0062] For ease of understanding, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the read / write rate limiting method in the first related technology. In this first related technology, all input and output data obtained by the backend from the frontend are placed in a shared buffer. Quality of Service (QoS) reads commands from the shared buffer and determines the type of command to execute the corresponding operation. For example, a read command reads data from a cloud disk or local disk, while a write command reads data from a storage device and writes it to the cloud disk or local disk. After the command is processed, completion information is written to the frontend's completion queue (CQ) so that the frontend releases the storage units of completed commands in the submission queue. Since write and read commands share a buffer, if a write command is waiting at the exit of the shared buffer's current queue, and the write operation is restricted, this queue will be in a waiting state. Read commands in the queue will be intercepted, resulting in head-of-queue blocking and causing significant jitter in the read operation rate limiting. For example... Figure 3As shown, assume the shared buffer queues read command 0, read command 1, write command 0, read command 2, write command 1, etc. If the commands preceding write command 0 are completed, write command 0 will queue at the exit. At this time, read command 2 must wait for write command 0 to complete before it can be processed. If write operations are restricted at this point, it will cause head-of-line blocking.

[0063] As for the second type of read / write speed limiting method, such as Figure 4 As shown, the second related technique divides the command buffer into a read buffer and a write buffer, with read and write commands written to their respective buffers. Assume the write buffer is full and the read buffer is empty. However, since the next operation type on the front end is unknown (whether it's a read or write command), backpressure blocking occurs. Only after a write operation processes a portion of the data to release the buffer can a new command be read, and only then can read operations be processed. For example... Figure 5 As shown, the read buffer stores read commands 0 to n and has free units. At this time, the write buffer is full. Since the backend does not know whether the next command retrieved from the frontend is a read command or a write command, back pressure blocking will occur. It is necessary to wait for the write buffer to release some space before it can continue to read new commands and then it may be time to process read operations.

[0064] As the above analysis shows, for the first related technology, the backend of the storage device does not know the type of command currently pointed to by the head pointer before retrieving the command from the frontend's commit queue. If it is just processing a regular read or write command, it will inevitably encounter the head-of-queue blocking problem. For the second related technology, if conventional single-channel processing is used, the entire frontend commit queue descriptor needs to be moved to the backend storage to completely solve the head-of-queue blocking problem, which is obviously impractical.

[0065] Compared to the two related technologies, such as Figure 6 As shown, the storage device read / write method disclosed herein maps the target commit queue of the front end of the storage device into a first commit queue and a second commit queue to process read and write commands respectively. This avoids the limitation imposed by one commit queue corresponding to one data transmission channel. Furthermore, since both data transmission channels can read the target commit queue simultaneously, the head pointers of the commit queues corresponding to each data transmission channel will not experience head-of-queue blocking due to read / write rate limits or insufficient storage space in the read / write buffer.

[0066] This embodiment provides a second method for reading and writing storage devices, which can be used in the backend of the aforementioned storage device. Figure 7 This is a flowchart illustrating a second storage device read / write method according to an embodiment of this disclosure, as shown below. Figure 7 As shown, the process includes the following steps:

[0067] Step S701: In response to a processing instruction for a target commit queue at the front end of the storage device, the target commit queue is mapped to a first commit queue and a second commit queue at the back end of the storage device. The target commit queue includes at least one of a read command and a write command. See step S101 above; further details will not be provided here.

[0068] Step S702: Use the first data transmission channel to read the read command in the first submission queue and write the read command to the read buffer.

[0069] Specifically, step S702 includes:

[0070] Step S7021: Obtain the first head pointer and the first tail pointer of the first submission queue.

[0071] Specifically, the first head pointer and the first tail pointer of the first submission queue are obtained using the first data transmission channel.

[0072] Step S7022: If the first head pointer is different from the first tail pointer, then the first data transmission channel is used to read the read command in the first submission queue and the read read command is written to the read buffer.

[0073] Understandably, if the first head pointer and the first tail pointer are different, it indicates that there are unread commands in the first commit queue. Therefore, when the first head pointer and the first tail pointer are different, the first data transmission channel is used to continue reading read commands from the first commit queue, and the read read commands are written to the read buffer. If the first head pointer and the first tail pointer are the same, it indicates that all commands in the first commit queue have been read, and reading read commands from the first commit queue is stopped at this time.

[0074] Step S703: Use the second data transmission channel to read the write command in the second submission queue and write the read write command to the write buffer.

[0075] Specifically, step S703 includes:

[0076] Step S7031: Obtain the second head pointer and the second tail pointer of the second submission queue.

[0077] Specifically, the second head pointer and the second tail pointer of the second submission queue are obtained using the second data transmission channel.

[0078] In step S7032, if the second head pointer is different from the second tail pointer, the write command in the second submission queue is read using the second data transmission channel, and the read write command is written to the write buffer.

[0079] Understandably, if the second head pointer and the second tail pointer are different, it indicates that there are unread commands in the second commit queue. Therefore, when the second head pointer and the second tail pointer are different, the second data transmission channel is used to continue reading write commands from the second commit queue, and the read write commands are written to the write buffer. If the second head pointer and the second tail pointer are the same, it indicates that all commands in the second commit queue have been read, and reading write commands from the second commit queue stops at this point.

[0080] Step S704 involves processing the read commands in the read buffer to perform a read operation. See step S104 above for further details.

[0081] Step S705 involves processing the write commands in the write buffer to perform the write operation. See step S105 above for further details.

[0082] The storage device read / write method provided in this embodiment reads a read command from the first commit queue only if the first head pointer and the first tail pointer of the first commit queue are different. Therefore, the state of the first commit queue can be accurately determined using the first head pointer and the first tail pointer. Similarly, a write command from the second commit queue is read using the second data transmission channel only if the second head pointer and the second tail pointer are different. Therefore, the state of the second commit queue can be accurately determined using the second head pointer and the second tail pointer, thus avoiding data reading errors or program logic errors between the first and second commit queues.

[0083] In some optional implementations, step S7022 above, which involves reading read commands from the first submission queue using the first data transmission channel and writing the read commands to the read buffer, includes:

[0084] Step a1: Use the first data transmission channel to read the first target command in the first submission queue and update the first head pointer.

[0085] It should be noted that the queue follows the "first-in, first-out" (FIFO) data structure. In actual operation, the first data transmission channel starts reading the first target command from the position pointed to by the first head pointer in the first submission queue. Each time a first target command is read, the first head pointer is updated to point to the position of the next first target command.

[0086] Step a2: If the first target command is a read command, then write the first target command to the read buffer.

[0087] Specifically, the first target command is determined using the first data transmission channel. If the first target command is a read command, it is written to the read buffer. If the first target command is a write command, it is discarded.

[0088] The storage device read / write method provided in this embodiment uses a first data transmission channel to read a first target command from a first submission queue. Only if the first target command is a read command is it written to a read buffer. Therefore, commands written to the read buffer can be filtered, avoiding head-of-queue blocking when processing commands in the read buffer. Simultaneously, it saves storage resources.

[0089] In some optional implementations, step S7032 above, which involves reading the write command from the second submission queue using the second data transmission channel and writing the read write command to the write buffer, includes:

[0090] Step b1: Use the second data transmission channel to read the second target command in the second submission queue and update the second head pointer.

[0091] Specifically, the second data transmission channel reads the second target command from the second submission queue starting from the position pointed to by the second head pointer. Each time a second target command is read, the second head pointer is updated to point to the position of the next second target command.

[0092] Step b2: If the second target command is a write command, then write the second target command to the write buffer.

[0093] Specifically, the second data transmission channel is used to determine the type of the second target command. If the second target command is a write command, it is written to the write buffer. If the second target command is a read command, it is discarded.

[0094] The storage device read / write method provided in this embodiment uses a second data transmission channel to read a second target command from a second submission queue. Only if the second target command is a write command is it written to the write buffer. Therefore, commands written to the write buffer can be filtered, avoiding head-of-queue blocking when processing commands in the write buffer. Simultaneously, it saves storage resources.

[0095] For example, such as Figure 8As shown, assume the target submission queue contains 13 commands: read commands 0-9, write command 0, write command 1, and read command 10. The unit following read command 10 is an idle unit. The head pointer of the target submission queue points to read command 0, and the tail pointer points to the idle unit following read command 10. The first and second submission queues, mapped from the target submission queue, are identical to the target submission queue. Assuming there are a few unexpected events in the processing of read commands, the processing of subsequent write commands will be blocked according to relevant technologies. Assuming the first data transmission channel reads 8 commands initially, all 8 commands are read commands, and these read commands are cached in their corresponding read buffers for scheduling. The first head pointer of the first data transmission channel is updated to point to read command 8. Assuming the front end does not issue any new commands at this time, the first tail pointer remains unchanged. If the front end issues new commands, the first tail pointer is updated to point to the unit following the last command in the first submission queue. Since the head pointer and tail pointer are different at this point, indicating that there are still commands to be processed, a second command read is initiated from the first submission queue, retrieving the remaining 5 commands. Of these, 3 are read commands and 2 are write commands. The 3 read commands are written to the read buffer, and the 2 write commands are discarded. Simultaneously, the head pointer is updated to the unit following the last command, meaning the head pointer and tail pointer are the same. This indicates that there are no commands to be processed in the first submission queue, and it awaits new command notifications.

[0096] Similarly, when the second data transmission channel reads 8 commands for the first time, all 8 are read commands. Since the second data transmission channel handles write commands, all 8 read commands are discarded. The second head pointer of the second data transmission channel is updated to point to read command 8. Assuming the front end does not issue any new commands at this time, the second tail pointer remains unchanged. If the front end issues a new command, the second tail pointer is updated to point to the unit after the last command in the second submission queue. Since the second head pointer and the second tail pointer are different at this time, it indicates that there are still commands to be processed, so a second command reading of the second submission queue is initiated, reading the remaining 5 commands. Among them, 3 read commands and 2 write commands are read. The 3 read commands are discarded, and the 2 write commands are written to the write buffer. At the same time, the second head pointer is updated to the unit after the last command, that is, the second head pointer and the second tail pointer are the same. At this time, it means that there are no commands to be processed in the second submission queue, and it is waiting for new command notification.

[0097] Understandably, in actual operation, the first and second data transmission channels are subject to back pressure from their respective buffers and the consumption of rate-limiting tokens. The commands for reading the submission queues from the first and second data transmission channels are not synchronized. Therefore, it is necessary to wait for the head and tail pointers of both the first and second data transmission channels to be aligned (i.e., the first head pointer is the same as the first tail pointer, and the second head pointer is the same as the second tail pointer) before determining that both backend submission queues are empty. When the submission queue's lifecycle ends, the emptying check is performed based on the fact that both backend submission queues are empty.

[0098] It should be noted that the first and second data transmission channels execute in parallel. After processing the corresponding commands, both channels send command completion information back to the front end and write it to the front end's completion queue. The release of the front end's target submission queue depends on the completion status of the completion queue. Therefore, the updating and release of the front end's target submission queue is consistent for both the first and second data transmission channels.

[0099] It is worth noting that in the storage device read / write method disclosed herein, since dual-channel direct memory access interfaces (i.e., the first data transmission channel and the second data transmission channel) are opened, read commands and write commands each occupy one direct memory access interface, and both the first and second data transmission channels normally handle commands. Because the first and second data transmission channels do not know the type of the command to be read before reading the corresponding commit queue, they can only filter out unnecessary commands based on the attributes of the current data transmission channel (also known as DMA CHN attributes) after reading the command, and store the remaining commands in the corresponding buffers. Therefore, when token bucket rate limiting is applied to the exits of the read and write buffers, head-of-queue blocking will not occur at the exits of the read and write buffers, allowing for precise independent rate limiting of reads and writes.

[0100] For example, such as Figure 9 As shown, the first data transmission channel, after reading a command, performs a type determination on the command. If it is a read command, it is stored in the read buffer; for example, read commands 0 to n are stored in the read buffer. If it is a write command, the command is discarded. Then, the read commands in the read buffer are processed. Similarly, the second data transmission channel, after reading a command, performs a type determination on the command. If it is a write command, it is stored in the write buffer; for example, write commands 0 to m are stored in the write buffer. If it is a read command, the command is discarded. Then, the write commands in the write buffer are processed.

[0101] This embodiment provides a third method for reading and writing storage devices, which can be used in the backend of the aforementioned storage devices. Figure 10This is a flowchart illustrating a second storage device read / write method according to an embodiment of this disclosure, as shown below. Figure 10 As shown, the process includes the following steps:

[0102] Step S1001: In response to a processing instruction for a target commit queue at the front end of the storage device, the target commit queue is mapped to a first commit queue and a second commit queue at the back end of the storage device. The target commit queue includes at least one of a read command and a write command. See step S101 above; further details will not be provided here.

[0103] Step S1002: Read the read command from the first submission queue using the first data transmission channel, and write the read read command to the read buffer. See step S702 above; further details will not be provided here.

[0104] Step S1003: Use the second data transmission channel to read the write commands from the second submission queue and write the read write commands to the write buffer. See step S703 above, and will not be elaborated further here.

[0105] Step S1004 involves processing the read commands in the read buffer to perform a read operation. See step S104 above for further details.

[0106] Step S1005 involves processing the write commands in the write buffer to perform the write operation. See step S105 above for further details.

[0107] Step S1006: If the read buffer reaches the first preset storage threshold, then backpressure is applied to the read commands in the first submission queue.

[0108] Optionally, the first preset storage threshold is the number of storage units in the read buffer, or the maximum storage capacity of the read buffer, and the first preset storage threshold can be adjusted according to the actual situation.

[0109] Step S1007: If the write buffer reaches the second preset storage threshold, then backpressure is applied to the read of write commands in the second submission queue.

[0110] Optionally, the second preset storage threshold is the number of storage units in the write buffer, or the maximum storage capacity of the write buffer, and the second preset storage threshold can be adjusted according to the actual situation.

[0111] The storage device read / write method provided in this embodiment applies back pressure to read commands in the first commit queue if the read buffer reaches a first preset storage threshold. This allows for pausing or slowing down the reading of read commands in the first commit queue. Similarly, applying back pressure to write commands in the second commit queue if the write buffer reaches a second preset storage threshold allows for pausing or slowing down the reading of write commands in the second commit queue to prevent data loss or system crashes.

[0112] This embodiment provides a fourth method for reading and writing storage devices, which can be used in the backend of the aforementioned storage devices. Figure 11 This is a flowchart illustrating a third storage device read / write method according to an embodiment of this disclosure, as shown below. Figure 11 As shown, the process includes the following steps:

[0113] Step S1101, in response to a processing instruction for a target commit queue at the front end of the storage device, maps the target commit queue to a first commit queue and a second commit queue at the back end of the storage device, wherein the target commit queue includes at least one of a read command and a write command. See step S101 above; further details will not be provided here.

[0114] Step S1102: Read the read command from the first submission queue using the first data transmission channel, and write the read read command to the read buffer. See step S702 above; further details will not be provided here.

[0115] Step S1103: Read the write commands from the second submission queue using the second data transmission channel, and write the read write commands to the write buffer. See step S703 above; further details will not be provided here.

[0116] Step S1104 involves processing the read commands in the read buffer to perform a read operation. See step S104 above for further details.

[0117] Step S1105 involves processing the write commands in the write buffer to perform the write operation. See step S105 above for further details.

[0118] Step S1106: In response to the rate limiting instruction for the read buffer, the processing of read commands in the read buffer is rate-limited.

[0119] Specifically, the first data transmission channel responds to a rate-limiting instruction for the read buffer by limiting the processing rate of read commands in the read buffer.

[0120] Step S1107: In response to the rate limiting instruction for the write buffer, the processing of write commands in the write buffer is rate-limited.

[0121] Specifically, the second data transmission channel responds to a rate-limiting instruction for the write buffer by limiting the processing rate of write commands in the write buffer.

[0122] The storage device read / write method provided in this embodiment limits the processing rate of read commands in the read buffer in response to a rate-limiting instruction for the read buffer. Therefore, it can limit the data transfer rate at the exit of the read buffer, avoiding excessive consumption of system resources due to excessive data transfer, and also preventing system crashes or instability caused by processing a large number of read commands, thus improving the stability and reliability of read command processing. Similarly, it limits the processing rate of write commands in the write buffer in response to a rate-limiting instruction for the write buffer. Therefore, it can limit the data transfer rate at the exit of the write buffer, avoiding excessive consumption of system resources due to excessive data transfer, and also preventing system crashes or instability caused by processing a large number of write commands, thus improving the stability and reliability of write command processing.

[0123] For example, such as Figure 12 The computer bus interface standard (Peripheral Component Interconnect Express, PCIe) opens two sets of direct memory access channels: a first data transmission channel for handling read commands and a second data transmission channel for handling write commands. The front-end driver sends a notification via a message passing mechanism, simultaneously updating the tail pointer states of both the first and second data transmission channels to obtain the first head and tail pointers of the first submission queue. Then, the first and second data transmission channels each maintain their corresponding head and tail pointer states and initiate read requests to the target submission queue from the front-end. Specifically, the first data transmission channel reads the first target command from the first submission queue mapped from the target submission queue, and the second data transmission channel reads the second target command from the second submission queue mapped from the target submission queue.

[0124] Next, the first data transmission channel determines whether the first head pointer and the first tail pointer are different. If they are the same, it continues to determine whether the first head pointer and the first tail pointer are different. If they are different, it determines whether the read buffer is not full. If it is full, it pushes back the read request of the first data transmission channel. If it is not full, the first data transmission channel reads the first target command of the first submission queue obtained by mapping the target submission queue. It determines whether the first target command is a read command. If it is not a read command, it discards the first target command and updates the first head pointer, i.e., the first head pointer jumps and increments by 1. If it is a read command, it writes the first target command into the read buffer and updates the first head pointer, i.e., the first head pointer jumps and increments by 1. It determines whether to rate-limit the processing of read commands in the read buffer. If rate-limiting is required, it does so. If not, it re-determines whether to rate-limit the processing of read commands in the read buffer, i.e., input / output / byte (also known as IO / Byte) rate limiting.

[0125] Similarly, the second data transmission channel checks if the second head pointer and the second tail pointer are different. If they are the same, it returns to check if they are different. If they are different, it checks if the write buffer is not full. If it is full, it pushes back the read request to the second data transmission channel. If it is not full, the second data transmission channel reads the second target command from the second submission queue mapped from the target submission queue. It checks if the second target command is a write command. If it is not a write command, it discards the second target command. It updates the second head pointer, i.e., increments the second head pointer by 1. If it is a write command, it writes the second target command to the write buffer and updates the second head pointer, i.e., increments the second head pointer by 1. It checks whether to rate-limit the processing of write commands in the write buffer. If rate-limiting is required, it is applied. If not, it re-checks whether to rate-limit the processing of write commands in the write buffer, i.e., input / output / byte rate limiting.

[0126] Understandably, the first and second data transmission channels apply backpressure to read the submission queue commands based on the empty / full status of their respective buffers. At this time, read commands and write commands are completely isolated, and there will be no head-of-queue blocking problem.

[0127] As a specific application example, a target program is installed on the backend of the storage device for reading and writing to the storage device. Within this target program, the storage device read / write method of this disclosure can be used to virtualize the frontend target submission queue into a first submission queue and a second submission queue on the backend, respectively for processing read and write commands. Simultaneously, command type filtering is performed on the data transmission channels corresponding to the read and write commands. Therefore, the head-of-line blocking problem at two key points—the front-end / back-end interaction part of the storage device and the internal storage buffer—is effectively solved, achieving read / write decoupling and saving storage resources. Furthermore, while solving the head-of-line blocking problem, it also ensures that the bits per second (bps) and the number of input / output operations per second (IOPS) rate limiting are both stable without jitter. Compared to read / write rate limiting schemes in related technologies, the number of cached read and write commands is consistent, without adding additional storage resources.

[0128] This embodiment also provides a storage device read / write apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0129] This embodiment provides a storage device read / write device, such as... Figure 13 As shown, it includes:

[0130] The queue mapping module 1301 is used to map the target submission queue to a first submission queue and a second submission queue in the back end of the storage device in response to a processing instruction for a target submission queue at the front end of the storage device, wherein the target submission queue includes at least one of a read command and a write command;

[0131] The read command writing module 1302 is used to read the read commands in the first submission queue using the first data transmission channel and write the read read commands to the read buffer;

[0132] The write command writing module 1303 is used to read the write commands in the second submission queue using the second data transmission channel and write the read write commands to the write buffer;

[0133] The read command processing module 1304 is used to process read commands in the read buffer to perform read operations;

[0134] The write command processing module 1305 is used to process the write commands in the write buffer to perform write operations.

[0135] In some optional implementations, the read command writing module 1302 includes:

[0136] The first pointer acquisition unit is used to acquire the first head pointer and the first tail pointer of the first submission queue;

[0137] The read command processing unit is used to read the read command in the first submission queue using the first data transmission channel if the first head pointer and the first tail pointer are different, and to write the read read command to the read buffer.

[0138] In some optional implementations, the first pointer acquisition unit includes:

[0139] The first command reading subunit is used to read the first target command in the first submission queue using the first data transmission channel and update the first head pointer;

[0140] The read command processing subunit is used to write the first target command to the read buffer if the first target command is a read command.

[0141] In some alternative implementations, the write command writing module 1303 includes:

[0142] The second pointer acquisition unit is used to acquire the second head pointer and the second tail pointer of the second submission queue;

[0143] The write command processing unit is used to read the write command in the second submission queue using the second data transmission channel if the second head pointer is different from the second tail pointer, and write the read write command to the write buffer.

[0144] In some optional implementations, the second pointer acquisition unit includes:

[0145] The second command reading subunit is used to read the second target command in the second submission queue using the second data transmission channel and update the second head pointer;

[0146] The write command processing subunit is used to write the second target command to the write buffer if the second target command is a write command.

[0147] In some optional embodiments, the storage device read / write apparatus of this disclosure further includes:

[0148] The first backpressure module is used to apply backpressure to the reading of read commands in the first submission queue if the read buffer reaches the first preset storage threshold.

[0149] The second backpressure module is used to apply backpressure to the reading of write commands in the second submission queue if the write buffer reaches the second preset storage threshold.

[0150] In some optional embodiments, the storage device read / write apparatus of this disclosure further includes:

[0151] The first rate limiting module is used to limit the processing rate of read commands in the read buffer in response to rate limiting instructions for the read buffer.

[0152] The second rate limiting module is used to limit the processing rate of write commands in the write buffer in response to rate limiting instructions for the write buffer.

[0153] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0154] In this embodiment, the storage device read / write device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0155] This disclosure also provides an electronic device having the above-described features. Figure 13 The storage device read / write device shown.

[0156] Please see Figure 14 , Figure 14 This is a structural block diagram of an electronic device provided in an optional embodiment of this disclosure, such as... Figure 14 As shown, the electronic device includes one or more processors 1401, a memory 1402, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 14 Take a processor 1401 as an example.

[0157] Processor 1401 may be a central processing unit, a network processor, or a combination thereof. Processor 1401 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0158] The memory 1402 stores instructions executable by at least one processor 1401 to cause at least one processor 1401 to perform the method shown in the above embodiments.

[0159] Memory 1402 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, memory 1402 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 1402 may optionally include memory remotely located relative to processor 1401, and this remote memory may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0160] The memory 1402 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 1402 may also include a combination of the above types of memory.

[0161] The electronic device also includes an input device 1403 and an output device 1404. The processor 1401, memory 1402, input device 1403, and output device 1404 can be connected via a bus or other means. Figure 14 Taking the example of a connection between China and Israel via a bus.

[0162] Input device 1403 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1404 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.

[0163] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0164] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0165] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for reading and writing a storage device, characterized in that, The method includes: In response to a processing instruction for a target commit queue at the front end of the storage device, the target commit queue is mapped to a first commit queue and a second commit queue at the back end of the storage device, wherein the target commit queue includes at least one of a read command and a write command; the first commit queue and the second commit queue are the same as the target commit queue; The first data transmission channel is used to read the read command from the first submission queue and write the read command to the read buffer; wherein, the first data transmission channel is configured to read the command from the first submission queue and determine the type of the read command. If the read command is a read command, the read command is written to the read buffer. The second data transmission channel is used to read the write command in the second submission queue and write the read write command to the write buffer; wherein, the second data transmission channel is configured to read the command in the second submission queue and determine the type of the read command. If the read command is a write command, the read write command is written to the write buffer. The read commands in the read buffer are processed to perform a read operation; The write commands in the write buffer are processed to perform the write operation.

2. The method according to claim 1, characterized in that, The step of reading read commands from the first submission queue using the first data transmission channel and writing the read read commands to the read buffer includes: Obtain the first head pointer and the first tail pointer of the first submission queue; If the first head pointer is different from the first tail pointer, then the first data transmission channel is used to read the read command in the first submission queue, and the read read command is written to the read buffer.

3. The method according to claim 2, characterized in that, The step of reading read commands from the first submission queue using the first data transmission channel and writing the read read commands to the read buffer includes: The first target command in the first submission queue is read using the first data transmission channel, and the first head pointer is updated. If the first target command is a read command, then the first target command is written to the read buffer.

4. The method according to claim 1, characterized in that, The step of reading the write command from the second submission queue using the second data transmission channel and writing the read write command to the write buffer includes: Obtain the second head pointer and the second tail pointer of the second submission queue; If the second head pointer is different from the second tail pointer, then the write command in the second submission queue is read using the second data transmission channel, and the read write command is written to the write buffer.

5. The method according to claim 4, characterized in that, The step of reading the write command from the second submission queue using the second data transmission channel and writing the read write command to the write buffer includes: The second target command in the second submission queue is read using the second data transmission channel, and the second head pointer is updated. If the second target command is a write command, then the second target command is written to the write buffer.

6. The method according to claim 1, characterized in that, The method further includes: If the read buffer reaches the first preset storage threshold, then the reading of read commands in the first submission queue will be subject to back pressure. If the write buffer reaches the second preset storage threshold, then backpressure is applied to the reading of write commands in the second submission queue.

7. The method according to claim 1, characterized in that, The method further includes: In response to a rate-limiting instruction for the read buffer, the processing rate of read commands in the read buffer is limited; In response to a rate-limiting instruction for the write buffer, the processing of write commands in the write buffer is rate-limited.

8. A storage device read / write apparatus, characterized in that, The device includes: A queue mapping module is configured to, in response to a processing instruction for a target commit queue at the front end of the storage device, map the target commit queue to a first commit queue and a second commit queue at the back end of the storage device, wherein the target commit queue includes at least one of a read command and a write command; the first commit queue and the second commit queue are identical to the target commit queue. The read command writing module is used to read read commands from the first submission queue using the first data transmission channel and write the read read commands to the read buffer; wherein, the first data transmission channel is configured to read commands from the first submission queue and determine the type of the read command; if the read command is a read command, then the read read command is written to the read buffer. The write command writing module is used to read write commands from the second submission queue using the second data transmission channel and write the read write commands to the write buffer; wherein, the second data transmission channel is configured to read commands from the second submission queue and determine the type of the read command; if the read command is a write command, then the read write command is written to the write buffer. The read command processing module is used to process the read commands in the read buffer to perform read operations; The write command processing module is used to process the write commands in the write buffer to perform write operations.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the storage device read / write method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a computer to perform the storage device read / write method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions for causing a computer to perform the storage device read / write method according to any one of claims 1 to 7.

Images