Method for scheduling input / output requests to main memory and storage device
By measuring and grouping the delay of memory devices, setting the target processing size and integral value, and scheduling input/output requests, the QoS inconsistency and resource scarcity caused by heterogeneous memory devices is solved, and stable input/output performance and service quality are achieved. .
Patent Information
- Application Number
- CN202410421955.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-04-09
- Publication Date
- 2024-11-08
AI Technical Summary
In electronic systems, different delays of heterogeneous memory devices lead to inconsistent input/output request processing times, difficult to ensure a unified quality of service (QoS), and may lead to resource scarcity and operation timeouts.
By obtaining the information of each memory device in the memory pool, measuring its delay, and grouping it, setting the target processing size and integral value according to the delay of each memory group, and using integral value to schedule input/output requests to ensure efficient resources Allocate and schedule QoS.
It realizes unified performance scheduling of heterogeneous memory devices, prevents resource scarcity, ensures that input/output requests are processed within a predetermined time, and maintains stable service quality.
Smart Images

Figure CN118915959A_ABST
Abstract
Description
Technical Field
[0001] The inventive concept relates to a storage device and a method for scheduling input / output requests for access to a memory pool. Background Art
[0002] In response to the trend of increasing the capacity of the main memory required by the electronic system, the main memory may include a memory pool, and the memory pool includes a plurality of memory devices. Each of the plurality of memory devices may have a different delay required for access. For example, when the plurality of memory devices include heterogeneous memory devices, the delays between the heterogeneous memory devices may be different.
[0003] The electronic system may include an input / output device, such as a storage device. The input / output device may access a memory pool to perform an operation. Different delays in each area of the memory pool may affect the time period required for the input / output device to perform an operation. For example, when the input / output device does not complete an operation within the required time period, a timeout may occur, and the input / output device may have difficulty in ensuring a uniform quality of service (QoS). Summary of the invention
[0004] An aspect of the inventive concept is to provide a method and a memory capable of scheduling an input / output request to a main memory based on a delay of each region of the main memory.
[0005] According to one aspect of the present disclosure, a method for scheduling input / output requests to a main memory includes: obtaining information of multiple memory devices included in the main memory; measuring multiple delays of the multiple memory devices respectively according to the unit data size of the type of each memory device of the multiple memory devices; grouping the multiple memory devices into multiple memory groups; determining the size of a time window to be repeated periodically; setting a target processing size of each of the multiple memory groups based on the size of the time window and the delay of each of the multiple memory groups; allocating an integral value of each memory group for each time window based on the target processing size of each of the multiple memory groups; and using the allocated integral of each of the multiple memory groups to process multiple input / output requests queued in an input / output queue associated with a corresponding memory group of the multiple memory groups.
[0006] According to one aspect of the present disclosure, a method for scheduling input / output requests to a main memory includes: obtaining information of multiple memory devices included in the main memory; measuring multiple delays of the multiple memory devices respectively according to the unit data size of the type of each memory device of the multiple memory devices; grouping the multiple memory devices into multiple memory groups according to the memory type and delay of each memory device in the multiple memory devices, wherein each memory type includes at least two or more memory groups; determining the size of a time window to be repeated periodically; setting a target processing size of each memory type of the multiple memory devices based on the size of the time window and the delay of each memory type; allocating an integral value of each memory type based on the target processing size of each memory type; distributing the integral value of each memory type to the at least two or more memory groups of each memory type based on the delay of each memory group in the at least two or more memory groups of each memory type; and using the allocated integral of each memory group to process multiple input / output requests queued in an input / output queue associated with a corresponding memory group in the multiple memory groups.
[0007] According to one aspect of the present disclosure, a storage device includes: a non-volatile memory device; and a storage controller, the storage controller including a buffer memory and configured to: virtualize resources of the non-volatile memory device to provide multiple physical functions to a host; and based on target quality of service (QoS) of the multiple physical functions and a latency of a main memory area of a main memory associated with the host, assign each of the multiple physical functions to a corresponding area of the buffer memory, the main memory area being accessed by the multiple physical functions. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The above and other aspects, features and advantages of the present invention will be more clearly understood through the following detailed description in conjunction with the accompanying drawings, in which:
[0009] Figure 1 is a diagram illustrating an electronic system according to an embodiment.
[0010] Figure 2 is a diagram illustrating transactions of an electronic system according to an embodiment.
[0011] Figure 3 is a flow chart illustrating a method for scheduling input / output requests according to an embodiment.
[0012] Figure 4 is a diagram showing a delay table according to an embodiment.
[0013] Figure 5is a flow chart illustrating in detail a method for scheduling input / output requests according to an embodiment.
[0014] Figures 6 to 8 is a diagram illustrating a method for scheduling input / output requests according to an embodiment.
[0015] Fig. 9 and Fig.10 is a diagram illustrating an electronic system according to an embodiment.
[0016] Fig.11 is a diagram showing a delay table according to an embodiment.
[0017] Fig.12 is a flow chart illustrating in detail a method for scheduling input / output requests according to an embodiment.
[0018] Fig.13 is a diagram for explaining a method for scheduling input / output requests according to an embodiment.
[0019] Fig.14 is a diagram showing a storage device according to an embodiment. DETAILED DESCRIPTION
[0020] Hereinafter, preferred embodiments will be described with reference to the accompanying drawings.
[0021] Figure 1 is a diagram illustrating an electronic system according to an embodiment. Figure 2 is a diagram illustrating transactions of an electronic system according to an embodiment.
[0022] The electronic system 100 may include electronic devices such as a desktop computer, a laptop computer, a network server, and a mobile device. The electronic system 100 may run at least one operating system. The operating system may control the functions and operations of the electronic system 100.
[0023] Reference Figure 1 , the electronic system 100 may include a host processor 110, a main memory 120, an input / output device 130, and a bus 101. The bus 101 may support communication between the host processor 110, the main memory 120, and the input / output device 130.
[0024] The host processor 110 may control the overall operation of the electronic system 100 by executing an operating system. The host processor 110 may include a central processing unit (CPU). The CPU may include a processor, a memory, and an input / output interface, and may communicate with other components of the electronic system 100 via the bus 101. According to an embodiment, the host processor 110 may be implemented as a system on chip (SoC).
[0025] The main memory 120 may store data required for the operation of the electronic system 100. For example, an operating system may be loaded into the main memory 120. The host processor 110 or the input / output device 130 may access the main memory 120 through the bus 101.
[0026] According to an embodiment, the main memory 120 may be implemented as a volatile memory such as a dynamic random access memory (DRAM) and a static random access memory (SRAM). The inventive concept is not limited thereto, and the main memory 120 may include a nonvolatile memory.
[0027] The input / output device 130 may expand the function of the electronic system 100. For example, the input / output device 130 may include at least one of devices such as a storage device, a graphic card, and a network card, which may be installed in a hardware slot such as a peripheral component interconnect express (PCIe) slot.
[0028] The input / output device 130 can access the main memory 120 to perform operations assigned thereto. Figure 2 , a case where the input / output device 130 is, for example, a storage device and the input / output device 130 accesses the main memory 120 will be described in detail.
[0029] Figure 2 A transaction between a host and a storage device is shown.
[0030] The host can include references such as Figure 1 The host processor 110 and the main memory 120 described above, and the storage device may correspond to the ... Figure 1 An input / output device 130 is described.
[0031] In operation S11, the storage device may obtain a command from the host. For example, a command to be provided by the host to the storage device may be generated by the host processor 110, and the command may be queued in the main memory 120. In order to obtain the command queued in the main memory 120, the storage device may generate a read request to the main memory 120, and may process the read request.
[0032] In operation S12 , the storage device may execute the command acquired from the host.
[0033] For example, the storage device may interpret whether the command is a read command, a write command, or other commands by decoding the command. The storage device may perform the operation indicated by the command based on the command decoding result.
[0034] When the command is a read command in which the host requests data stored in the storage device, the storage device may buffer the data requested from the nonvolatile memory device therein in a buffer memory therein.
[0035] In operation S13, the storage device may output the requested data to the host. The operation of outputting data by the storage device may include an operation of the storage device writing the data buffered therein to a predetermined area of the main memory 120. For example, in order to output data, the storage device may generate a write request to the main memory 120 and may process the write request.
[0036] When the command is a write command, in a manner different from operation S13, the storage device may obtain data from a predetermined area of the main memory 120 and may write the data to the nonvolatile memory device therein. In this case, in order to obtain the data, the storage device may generate a read request to the main memory 120 and may process the read request.
[0037] In operation S14, the storage device may provide a response to the command operation from the host to the host. For example, the response may indicate completion, failure, etc. of the command operation. The storage device may write data to a predetermined area, such as a completion queue of the main memory 120, to provide a response. The storage device may generate a write request to the main memory 120, and may process the write request to provide a response.
[0038] In short, in order to process a command from the host, the input / output device 130 such as a storage device may generate an input / output request to the main memory 120 and may access the main memory 120 .
[0039] In response to the demand for the electronic system 100 with a high-capacity main memory, the main memory 120 may include a memory pool including a plurality of storage devices. For example, when the electronic system 100 runs a plurality of virtual machines, the main memory 120 may require a large amount of memory space allocated to each of the plurality of virtual machines. In order to ensure a high-capacity memory space, the main memory 120 may include a plurality of memory devices.
[0040] According to an embodiment, the main memory 120 may include heterogeneous memory devices. Figure 1 , the main memory 120 may include a first type memory device 121 and a second type memory device 122 .
[0041] For example, the first type memory device 121 may include a dual in-line memory module (DIMM), and the second type memory device 122 may include a compute express link (CXL) memory module.
[0042] The DIMM can be inserted into a memory slot of the electronic system 100 and can communicate with the host processor 110 through a memory controller. The DIMM can have various types, such as double data rate (DDR), DDR2, DDR3, and DDR4, and can provide various speeds depending on the type. The CXL memory module can communicate with the host processor 110 through a PCIe-based CXL interface and can provide high-bandwidth memory access. In some embodiments, the memory controller can be integrated into the host processor 110.
[0043] Regions of the main memory 120 may have different delays. For example, heterogeneous memory devices such as the first type memory device 121 and the second type memory device 122 may have different delays. Even the same type of memory device may have different delays depending on the physical location where the memory device is installed, etc. Even in one memory device, the delay may vary for each region, or may vary depending on the pattern of accessing the region.
[0044] Different delays for each area of main memory 120 may make it difficult to guarantee the performance of input / output device 130. Since input / output device 130 needs to process an input / output request to main memory 120 in order to process a command from the host, the processing time of the command may vary depending on the area of main memory 120 accessed.
[0045] In addition, when the input / output device 130 performs an input / output request to a relatively high-delay area of the main memory 120, resource scarcity therein may occur. Specifically, the input / output device 130 may allocate resources such as a buffer memory therein to perform input / output operations with the main memory 120. When the input / output device 130 performs an input / output request to a relatively high-delay area in the main memory 120, the resources therein may be occupied for a long time to perform the input / output request, and resource scarcity may occur. For example, because resources such as a buffer memory are allocated to an input / output request for a long time, the buffer memory cannot be allocated to another input / output request as needed. The efficiency of using resources will be reduced.
[0046] As a result, the different delays in each area of main memory 120 may make it difficult to uniformly maintain the quality of service (QoS) provided by input / output device 130 to the host, and this difficulty may cause a particular input / output request to not be processed within a specified time (i.e., may cause a particular input / output request to time out and not be completed).
[0047] According to an embodiment, the input / output device 130 may schedule input / output requests based on the latency of each region of the main memory 120 to prevent resource starvation therein and provide a predetermined QoS.
[0048] Figure 3 is a flow chart illustrating a method for scheduling input / output requests according to an embodiment.
[0049] In operation S21, the input / output device may identify a memory device included in a main memory.
[0050] For example, the memory devices included in the main memory may be identified and controlled by a memory controller, a CXL interface, etc. of the electronic system according to the type. The memory controller may be integrated into the Figure 1 The host processor 110 is described. The present disclosure is not limited thereto. For example, the memory controller may be implemented separately from the host processor 110 and may communicate with the main memory 120 via the bus 101.
[0051] When the input / output device is initialized, the input / output device may obtain information about the memory device from the host processor 110, or may identify the memory device by directly sending and receiving packets with the memory device. For example, the input / output device may obtain information such as the type of the memory device and assign an identifier to the memory device. For example, the input / output device may receive a packet including the type of the memory device and assign an identifier to the memory device so that the memory devices providing the packet can be distinguished from each other. The type of the memory device may be one of various memory types such as a DIMM and a CXL memory module.
[0052] In operation S22 , the input / output device may measure a delay of each memory device included in the main memory, and may generate a delay table.
[0053] For example, the input / output device may provide an input / output request to each memory device of the main memory, and a turnaround time may be measured as a delay until the input / output request is completed. In an embodiment, in order to measure the turnaround time, the input / output request may be performed in units of a data size of the type of the memory device that performs the input / output request. In an embodiment, the turnaround time may be measured during the booting process of the electronic system 100.
[0054] The delay of the memory device may vary according to the type of the memory device, and may also vary according to various access conditions of the memory device. As a first example, when the memory device is a DIMM, the delay may vary according to the address association between the previous request and the next request or whether a row hit is requested for data cached in the row buffer. The address association may refer to the physical proximity of the addresses indicated by the previous request and the next request, such as whether the addresses indicate the same memory row, or whether the addresses are included in the same memory bank. The higher the address association, the lower the delay. As a second example, when the memory device is a CXL memory module, the delay may vary according to the margin of the buffer space in the CXL memory module, which may be represented by a PCIe credit. For example, the delay may vary according to the available space in the buffer. In order to efficiently manage such communication, PCIe uses a flow control mechanism based on credits. PCIe credits may refer to the available credits that a device has for transmitting data. The credit system can help prevent data overload and ensure that communication between devices is balanced and efficient.
[0055] The input / output device may collect a plurality of delay values, and may generate a delay table based on the delay values measured for each memory device. For example, the delay table may store the delay value for each memory device. Since the delay value may change according to various access conditions even in one memory device, the input / output device may store a representative value determined for each memory device in the delay table based on the plurality of delay values measured for each memory device. For example, the representative value may be an average value of the plurality of delay values.
[0056] The inventive concept is not limited to the case where the delay table stores a delay value for each memory device. For example, the input / output device may group a plurality of memory devices into memory groups, and generate a delay table storing a delay value for each memory group. For example, a plurality of memory devices may be grouped into memory groups of the same type or memory groups having similar delay values.
[0057] In operation S23 , the I / O device may schedule I / O requests to the main memory with reference to the delay table to maintain uniform performance of the I / O requests and thus provide a predetermined QoS for each I / O request from the I / O device 130 .
[0058] In the following, reference will be made to Figures 4 to 7A method for scheduling input / output requests according to an embodiment is described in detail. Scheduling may refer to allocating credits to each memory group according to the delay of each memory group. If the credits of a memory group are exhausted, the input / output request to the one memory group will be delayed, and the input / output request to other memory groups whose credits are not exhausted can be processed in advance, so the input / output request can be scheduled.
[0059] Figure 4 is a diagram showing a delay table according to an embodiment.
[0060] Reference Figure 4 , the delay table may include a delay value per memory group and a unit data size per memory group. Hereinafter, the term “per memory group” may refer to “each memory group in a plurality of memory groups”. For example, the delay value per memory group may refer to the delay value of each memory group. For example, the unit data size of a memory group may refer to the unit data size of each memory group.
[0061] When the main memory includes heterogeneous memory devices, communication protocols applied to the heterogeneous memory devices may be different. Due to different protocols, the unit data size that can be transmitted through an input / output request may be different for each memory device.
[0062] For example, the first memory group may include a first type of memory devices, and the second memory group may include a second type of memory devices.The first type of memory devices may be DIMMs, and the second type of memory devices may be CXL memory modules.
[0063] According to an embodiment, a DIMM may perform an input / output operation in a burst mode. Specifically, in order to quickly access memory cells included in a row, an input / output operation may be performed on all memory cells indicated by a row address. Therefore, a unit data size of an input / output operation may be determined according to the size of a row of the DIMM, and the DIMM may perform an input / output operation of a fixed unit data size.
[0064] According to an embodiment, a data size for an input / output operation may be specified in an input / output request to a CXL memory module. For example, a minimum size of the data size of an input / output request may be 8B (bytes), and an input / output request may request data of a relatively large size, such as 1024B and 2048B. For example, the CXL memory module may process input / output requests having various data sizes as well as a unit data size of 8B. The delay of an input / output request to the CXL memory module may vary according to the data size of the input / output request.
[0065] The delay values stored in the delay table may represent the delay per unit data size. Figure 4 In the example of , the unit data size of the first type memory device may be 512B, and the delay value of the unit data size 512B may be 128ns. The unit data size of the second type memory device may be 8B, and the delay value of the unit data size 8B may be 8ns.
[0066] For CXL memory modules, the latency of an I / O request may not necessarily be proportional to the data size of the I / O request. This may be because the latency of an I / O request includes not only the time period used to send a data packet, but also the time period used to send a packet including the I / O request. For example, when it takes 2048ns to input / output 2048B data in a CXL memory module, it may take longer than 8ns to input / output 8B data.
[0067] When relatively large-sized data such as 2048B is mainly input / output in a CXL memory module supporting a high bandwidth, it can be assumed that a delay of an input / output request is proportional to the data size of the input / output request.
[0068] The input / output device may schedule the input / output request by referring to the delay table, allocating credits per memory group, and deducting the number of credits determined according to the data size of the input / output request to implement the input / output request.
[0069] Figure 5 and Figure 6 is a diagram illustrating a method for scheduling input / output requests according to an embodiment.
[0070] Reference Figure 5 In operation S31, the input / output device may set the size of the time window. The time window may refer to a period for allocating credits per memory group in the credit-based input / output request scheduling according to the embodiment.
[0071] In operation S32 , the input / output device may set a target processing size per memory group based on the size of the time window and the delay per memory group.
[0072] Reference Figure 6 , a first processing time period for processing an input / output request of a first memory group and a second processing time period for processing an input / output request of a second memory group can be respectively shown in a plurality of time windows. A time window refers to a time period in which a predetermined amount of data can be processed in the first memory group and the second memory group, respectively. The plurality of time windows can be continuous in terms of time reference.
[0073] The target processing size of each memory group may be determined in operation S32. The target processing size may refer to the size of data that can be processed within the time window. Figure 6 In the example, the target processing size of the first memory group and the second memory group can be determined so that the sum of a first processing time period for processing data of the target processing size in the first memory group and a second processing time period for processing data of the target processing size in the second memory group is no greater than the size of the time window.
[0074] The processing time period for each memory group can be determined based on the target processing size of each memory group and the delay per unit data size of each memory group. For example, the processing time for a memory group can be established by considering the target processing size within the memory group and the delay per unit data size. When assuming that the delay of an input / output request is proportional to the data size of the input / output request, the target processing size for each memory group can be determined so that the sum of the values obtained by multiplying the delay per unit data size of each memory group by its target processing size is not greater than the time window.
[0075] To determine the target processing size per memory group, a ratio of the target processing size per memory group may be determined. As a first example, the ratio of the target processing size may be determined to be equal to the performance ratio per memory group, which may be determined based on the latency per memory group. Figure 4 In the example of FIG. 1 , the delays required for the first memory group and the second memory group to process the same 512B data are 128ns and 512ns, respectively, and it can be determined that the performance of the first memory group is 4 times higher than that of the second memory group. In this case, the ratio of the target processing size of the first memory group and the target processing size of the second memory group can be determined to be 4:1.
[0076] The method for determining the ratio of the target processing size is not limited to the first example described above. As a second example, the target processing size of each memory group can be set to have the same value as each other. For example, the ratio of the target processing size of the first memory group and the target processing size of the second memory group can be determined to be 1:1.
[0077] Return to reference Figure 5 , in operation S33 , the input / output device may determine the number of credits to be allocated per memory group for each time window based on the target processing size per memory group.
[0078] For example, the number of credits to be allocated to each memory group can be determined by dividing the target processing size of each memory group by the unit data size. When the target processing sizes of the first memory group and the second memory group are 8192B and 2048B, respectively, Figure 4In the example of , the number of integrations of the first memory group may be determined as 8192B / 512B=16, and the number of integrations of the second memory group may be determined as 2048B / 8B=256.
[0079] In operation S34, the input / output device may process the input / output requests queued in the input / output queue within the credit range allocated per memory group. For example, the input / output device may process the input / output requests queued in the input / output queue using the credits allocated per memory group, and once the allocated credits are exhausted, the subsequent input / output requests may be postponed to the next time window when new credits are filled. Figure 7 and Figure 8 A method of scheduling input / output requests within an assigned credit range is described in more detail.
[0080] Figure 7 and Figure 8 is a diagram illustrating a method for scheduling input / output requests according to an embodiment.
[0081] The input / output device may access the main memory to perform operations for the host, and may generate an input / output request to access the main memory.
[0082] Figure 7 The address area of the main memory that can be accessed by the input / output device is shown. For example, the physical address corresponding to each area of the main memory can be mapped to the virtual address of the host, and the input / output device can access each area of the main memory based on the virtual address. Figure 7 A virtual address region of the main memory mapped to a corresponding physical address of the main memory may be shown.
[0083] The memory address to be accessed by the input / output request may be determined according to the operation performed by the input / output device, and the memory device may be determined according to the address. For example, when the host executes multiple virtual machines VM1 and VM2, the main memory area may be allocated to each virtual machine. When performing an operation requested by the first virtual machine VM1 and when performing an operation requested by the second virtual machine VM2, the input / output device may access different areas of the main memory.
[0084] In addition, the input / output device can access different areas of the main memory according to the operation performed by the input / output device. For example, when the input / output device executes a read command, as shown in FIG. Figure 2 As described, a read command may be obtained from a first region Region1 of the main memory, data may be written to a second region Region2 of the main memory, and a response may be written to a third region Region3 of the main memory.
[0085] Other areas of the main memory accessed by the input / output request may be specified by a memory address such as a virtual address. The virtual address may be converted into a physical address, and the input / output device may specify the memory device accessed by the input / output request based on the physical address.
[0086] In short, in order to perform an operation for the host, the input / output device can generate an input / output request to various memory devices constituting the main memory, and can specify a memory device within the main memory to be accessed by each input / output request.
[0087] Figure 8 An input / output device, a first type memory device included in a first memory group, and a second type memory device included in a second memory group are shown.
[0088] The input / output device may include at least one input / output queue. The input / output device may queue the generated input / output request to the at least one input / output queue. Figure 8 In the example of , the input / output device may include corresponding input / output queues IOQ1 and IOQ2 for the first memory group and the second memory group. In addition, the input / output device may queue the input / output request to one of the input / output queues IOQ1 and IOQ2 according to the memory device to be accessed by the input / output request. The inventive concept is not limited thereto, and an input / output queue may be provided for each memory device, or the input / output queues may be integrated into one input / output queue regardless of the memory group.
[0089] exist Figure 8 In the example of , in a time window, 16 credits may be allocated to the first input / output queue and 256 credits may be allocated to the second input / output queue. The input / output device may use the credits to execute the input / output requests (R11 to R15 and R21 to R25) queued in the first and second input / output queues. For example, each time the input / output device executes an input / output request to a memory group, the credits allocated to the memory group may be deducted. When all credits for the memory group are consumed in the time window, the input / output device may postpone the processing of the input / output request to the memory group until credits are allocated in the next time window.
[0090] The number of points deducted for an input / output request can be determined based on the data size to be processed by the input / output request. For example, the data size of the first input / output request R11 to R15 for the first memory group can be constant at 512B. The number of points deducted after completing each of the first input / output request R11 to R15 can be determined as "1". The data size of the second input / output request R21 to R25 for the second memory group can be changed for each input / output request. The number of points deducted after completing each of the second input / output request R21 to R25 can be determined based on the value of (data size) / (unit data size). Whenever the input / output device executes an input / output request to a memory group, points can be deducted.
[0091] According to an embodiment, in order to process input / output requests to a main memory of memory groups with different delays, a target processing size per memory group may be set based on the delay of each memory group. Credits per memory group may be allocated based on the target processing size per memory group, and input / output requests may be scheduled based on the credits.
[0092] According to the embodiment, since data corresponding to the target processing size of each memory group can be processed in the time window, the throughput of each memory group can be uniformly maintained. Based on the uniform throughput of each memory group, the input / output device can maintain the QoS of the operation involving the input / output request to the main memory.
[0093] Since the input / output request is scheduled based on the integral, a large amount of internal resources of the input / output device can be prevented from being occupied for a long time in order to process the input / output request of the memory group with a relatively long delay. As a result, the shortage of the internal resources of the input / output device can be prevented, and the operation delay caused by the shortage of the internal resources can be prevented. Therefore, the timeout of the input / output device to the host command can be prevented.
[0094] exist Figure 8In the example, read requests and write requests among input / output requests can be scheduled without being distinguished. The inventive concept is not limited to this. As a first example, points can be allocated to read requests and write requests among input / output requests of each memory group, and input / output requests can be scheduled based on the points. As a second example, points can be allocated only to read requests or write requests. When the input / output device has a distinguished read buffer and write buffer, the input / output device can allocate points to the read request and the write request, respectively, and can process the read request and the write request among the input / output requests, respectively, within the allocated range of points. The input / output device can schedule read requests and write requests separately to more effectively prevent the scarcity of resources of the read buffer and the write buffer.
[0095] Figures 4 to 8 An example has been described in which a main memory includes heterogeneous memory devices and scheduling is performed based on the delay of memory groups grouped according to the types of memory devices. The inventive concept is not limited thereto. Figures 9 to 13 A method for scheduling input / output requests according to an embodiment is described.
[0096] Fig. 9 and Fig.10 is a diagram illustrating an electronic system according to an embodiment.
[0097] Reference Fig. 9 , the electronic system 200 may include a host processor 210, a main memory 220, an input / output device 230, and a bus 201. The host processor 210, the main memory 220, the input / output device 230, and the bus 201 may correspond to the reference Figure 1 The host processor 110, the main memory 120, the input / output device 130 and the bus 101 are described below. Fig. 9 The electronic system 200 and Figure 1 The differences between the electronic systems 100.
[0098] The host processor 210 may include a plurality of cores 211 and 212. The plurality of cores 211 and 212 may each be a processing core including an arithmetic unit and a cache memory, and may operate simultaneously and independently.
[0099] The main memory 220 may include first to fourth memory device groups 221, 222, 223, and 224. The first memory device group 221 and the second memory device group 222 may include first type memory devices, and the third memory device group 223 and the fourth memory device group 224 may include second type memory devices. According to an embodiment, the first type memory device may be a DIMM, and the second type memory device may be a CXL memory module.
[0100] Even among homogeneous memory devices, the latency in the memory devices for processing input / output requests may vary depending on the location of the memory devices. For example, when the electronic system 200 is a non-uniform memory access (NUMA) system, the multiple cores 211 and 212 may each be designed to have a local memory.
[0101] For example, the first memory device group 221 may be a local memory of the first core 211, and the second memory device group 222 may be a local memory of the second core 212. The locations of the first memory device group 221 and the second memory device group 222 may be separated. The delay of the input / output request generated by the input / output device 230 may also be different according to the locations of the first memory device group 221 and the second memory device group 222.
[0102] When the third memory device group 223 and the fourth memory device group 224 also have physically separated locations, the delay for the input / output request generated by the input / output device 230 may be different.
[0103] The latency of an I / O request generated by I / O device 230 may vary depending on other factors as well as the physical location of the memory device, and may also vary, for example, depending on whether an address translation cache (ATC) hit occurs for the I / O request. Fig.10 ATC is described in more detail. In some embodiments, when there is an ATC hit for an input / output request, the latency of the input / output request for each memory device may be measured.
[0104] Fig.10 is a diagram showing how a host processor 210 and an input / output device 230 access a main memory 220 in an electronic system 200 . Fig.10 The host processor 210, the main memory 220 and the input / output device 230 may correspond to the reference Fig. 9 Those described.
[0105] Reference Fig.10, the electronic system 200 may further include a memory management unit (MMU) 240 and an input / output memory management unit (IOMMU) 250. The MMU 240 may manage access of the host processor 210 to the main memory 220. Specifically, the MMU 240 may convert a virtual address of the host processor 210 into a physical address of the main memory 220, and may perform various tasks such as memory protection, cache management, and bus arbitration.
[0106] Similar to MMU 240, IOMMU 250 can manage access of input / output device 230 to main memory 220. Specifically, IOMMU 250 can convert virtual addresses of input / output device 230 into physical addresses of main memory 220, and can provide a function of protecting memory from failure of input / output device 230.
[0107] The input / output device 230 may provide an input / output request including a virtual address to the IOMMU 250 to access the main memory 220. The IOMMU 250 may search for virtual address to physical address mapping information, may convert the virtual address into a physical address based on the mapping information, and may transmit the input / output request including the physical address to the main memory 220. Since the IOMMU 250 has to process each input / output request individually, a bottleneck phenomenon may occur.
[0108] In order to alleviate the bottleneck phenomenon, the IOMMU 250 can provide an address translation service (ATS). When using ATS, the IOMMU 250 can cache the most recently used mapping information in the address translation cache (ATC) 251. When the virtual address for the input / output request hits in the ATC 251, the virtual address can be converted to a physical address using the most recently used mapping notification, and the input / output request can be transmitted to the main memory 220. The delay of the input / output request can be changed according to whether the ATC is hit. For example, when the ATC hits, the delay of the input / output request will be lower than when the ATC misses.
[0109] According to an embodiment, the memory devices of the main memory may be grouped according to delay, and a delay table for each memory group may be generated. According to an embodiment, homogeneous memory devices may be grouped into two or more memory groups. In addition, according to an embodiment, for point-based scheduling, it may also be respectively referenced whether an input / output request has an ATC hit.
[0110] Fig.11 is a diagram showing a delay table according to an embodiment.
[0111] Reference Fig.11, the delay table may include the delay of the first memory group to the fourth memory group respectively per unit data size when the ATC hits. The present disclosure is not limited thereto. The delay table may include the delay of each of the first memory group to the fourth memory group when the ATC misses. When the ATC misses, the delay for searching the mapping information is increased, so compared with the ATC hit, when the ATC misses, the delay is increased.
[0112] The input / output device can classify the memory devices according to the memory type, and the memory devices classified as the same type can also be grouped into different groups according to the delay. Fig.11 In an example, a first memory group and a second memory group of a first type memory device may have different delays, and a third memory group and a fourth memory group of a second type memory device may have different delays.
[0113] As a first example, after measuring the latency of each memory device, the input / output device may group memory devices having latency in different ranges into different groups. As a second example, the input / output device may group memory devices based on their physical locations, etc., and may measure the latency of each memory device group.
[0114] According to an embodiment, the credits of each memory bank may be allocated based on the latency of each memory bank, and the input / output requests may be scheduled based on the credits. In addition, during scheduling, whether the input / output request ATC hits may also be considered.
[0115] Fig.12 is a flow chart illustrating in detail a method for scheduling input / output requests according to an embodiment.
[0116] In operation S41 , the input / output device may set a time window.
[0117] In operation S42 , the input / output device may determine a target processing size per memory type based on the size of the time window and the delay per memory type.
[0118] For example, the delay per memory type may be determined based on the delay per memory group included in the delay table.The target processing size per memory type may be set to a value capable of processing all target processing sizes per memory type within a time window according to the delay per memory type.
[0119] The target processing size per memory type may be set to have the same target processing size per memory type, or may be set to have a target processing size having a ratio determined based on the performance of each memory type.
[0120] In operation S43 , the input / output device may allocate points per memory type based on the target processing size per memory type in the time window.
[0121] For example, the integral may be determined as a value of (target processing size) / (unit data size) for each memory type.
[0122] In operation S44, the input / output device may distribute the points allocated to each memory type to each memory group.
[0123] As reference Fig.11 As described, the delay of homogeneous memory devices may also vary for various reasons. For different memory groups of the same memory type, credits may be allocated based on the delay. For example, more credits may be allocated to memory groups with lower delays.
[0124] In operation S45 , the input / output device may process the input / output request queued in the input / output queue within the allocated credit range in the time window.
[0125] Within the credit range allocated to each memory group, the input / output request of each memory group can be scheduled.
[0126] According to an embodiment, the input / output device may even perform scheduling of input / output requests between memory devices of the same type having different latencies to maintain a target processing size set for each memory type.
[0127] Fig.13 is a diagram illustrating a method for scheduling input / output requests according to an embodiment.
[0128] Fig.13 An input / output device and first to fourth memory groups are shown.
[0129] As described above, the input / output device can generate input / output requests to various memory devices constituting the main memory, and can specify a memory device to be accessed by each input / output request.
[0130] The input / output device may include one or more input / output queues. Fig.13 In an example of , the input / output device may include an input / output queue for each of the first to fourth memory groups. The inventive concept is not limited thereto.
[0131] exist Fig.13In the example of , in a time window, 16 credits may be allocated to the first type memory device and 256 credits may be allocated to the second type memory device. The 16 credits allocated to the first type memory device may be distributed to the first memory group and the second memory group, and the 256 credits allocated to the second type memory device may be distributed to the third memory group and the fourth memory group.
[0132] According to an embodiment, a larger number of credits may be allocated to a memory group with a lower latency, and a smaller number of credits may be allocated to a memory group with a higher latency. According to an embodiment, when a large number of requests to a memory group with a higher latency are executed, a target processing size per memory type may be provided while preventing a problem of internal resource starvation of an input / output device.
[0133] According to an embodiment, the amount of points deducted for each input / output request may be adjusted according to whether each input / output request misses the ATC. For example, when the ATC misses, an additional delay for searching for mapping information for converting a virtual address to a physical address occurs, so the delay increases compared to the delay when the ATC hits. Therefore, the amount of points deducted for an input / output request that may be an ATC miss may increase.
[0134] According to an embodiment, in an electronic system having a memory pool including memory devices having various delays, an input / output device can schedule an input / output request for accessing the memory pool based on the delay of the memory device, thereby maintaining constant input / output performance despite differences in delay between devices. Therefore, QoS of operations of the input / output device accompanying the input / output request can be guaranteed, and operation timeout can be prevented.
[0135] In the following, reference will be made to Figures 1 to 13 The electronic system according to the embodiment is described in detail by taking the case where the input / output device is a storage device as an example.
[0136] Fig.14 is a diagram showing a storage device according to an embodiment.
[0137] Reference Fig.14 , the electronic system 300 may include a host 310, a storage device 330, and a bus 301. The bus 301 may support communication between the host 310 and the storage device 330.
[0138] Host 310 may be a computing device such as a desktop computer, a laptop computer, a network server, and a mobile device. For example, host 310 may include a computer such as Figure 1 and Fig. 9 The host processor and main memory are shown.
[0139] The storage device 330 may store data in response to a request of the host 310. For example, the storage device 330 may include at least one of a solid state drive (SSD), an embedded memory, and a removable external memory.
[0140] The storage device 330 may include a storage controller 331 and a nonvolatile memory device 332. The nonvolatile memory device 332 may be a storage medium for storing data received from the host 310, and may include, for example, a flash memory. The storage controller 331 may control the nonvolatile memory device 332.
[0141] When the storage device 330 is an SSD, the storage device 330 may be a device operating according to the non-volatile memory express (NVMe) standard. When the storage device 330 is a device operating according to the NVMe standard, the bus 301 may be a peripheral component interconnect express (PCIe) bus.
[0142] The host 310 may execute at least one operating system using a processor. The operating system may manage and control the overall functions and operations of the host 310. For example, the host 310 may execute a hypervisor 311 under the host operating system. The hypervisor 311 may be a component of the host operating system and may be provided by an application running under the host operating system.
[0143] The hypervisor 311 can extract physical resources including processors, memories and input / output devices, and can provide the extracted physical resources to virtual machines VM1, VM2 and VM3 as virtual devices including virtual processors, virtual memories and virtual input / output devices. Each of the virtual machines VM1, VM2 and VM3 can use a virtual device to execute a client operating system. Under the client operating system, one or more applications can be run. The hypervisor 311 can also be referred to as a virtual machine monitor (VMM) and can be, for example, software that creates and runs virtual machines VM1, VM2 and VM3. In an embodiment, the hypervisor 311 can enable a host computer to support multiple virtual machines VM1 to VM3 by virtually sharing its resources (such as main memory and processing). For ease of description, reference is made to three virtual machines VM1 to VM3, but the present disclosure is not limited thereto. The hypervisor 311 can generate more than three or less than three virtual machines.
[0144] The storage controller 331 may provide multiple virtual storage devices to the host 310. For example, the storage controller 331 may provide multiple physical functions PF1, PF2, and PF3. A physical function may refer to a full-featured PCIe function that can be discovered, managed, and manipulated like any other PCIe device. The host 310 may access the physical functions PF1, PF2, and PF3 independently and individually. According to an embodiment, the storage controller 331 may also support a virtual function, which may be a lightweight PCIe function that shares one or more resources with a physical function.
[0145] The storage device 330 may allocate resources such as storage space to each of the physical functions PF1 , PF2 , and PF3 , and may need to ensure consistent target QoS for the physical functions PF1 , PF2 , and PF3 .
[0146] QoS may refer to the ability of the storage controller 331 to prioritize data flows between different physical functions PF1, PF2, and PF3 to ensure stable and consistent performance of data input / output operations of different physical functions PF1, PF2, and PF3. For example, performance indicators of QoS may include input / output operations per second (IOPS), response time, throughput, etc.
[0147] In order to ensure a stable QoS for each of the different physical functions PF1, PF2, and PF3, even when the workload is concentrated on the first physical function PF1, the storage device 330 can process the workload within the limit of the target QoS of the first physical function PF1. Processing of the workload concentrated on the first physical function PF1 by the storage device 330 does not affect the QoS of the second physical function PF2 and the third physical function PF3.
[0148] It becomes important for the electronic system 300 to ensure a fixed storage capacity and a stable QoS for each physical function PF1, PF2, and PF3. The storage device 330 may access the main memory of the host 310 to process commands for the physical functions PF1, PF2, and PF3. When the memory device included in the main memory of the host 310 has various delays, the command processing time may vary according to the delay, and it may be difficult to guarantee the QoS of each of the physical functions PF1, PF2, and PF3.
[0149] According to an embodiment, to guarantee QoS for physical functions PF1 , PF2 , and PF3 , storage controller 331 may allocate buffer memory resources of storage device 330 based on latency of a main memory allocated to each of virtual machines VM1 , VM2 , and VM3 .
[0150] Each of the physical functions PF1, PF2, and PF3 may be mapped to one or more virtual machines VM1, VM2, and VM3. The physical functions PF1, PF2, and PF3 may access a main memory area allocated to the virtual machines VM1, VM2, and VM3 for command processing. The storage controller 331 may identify a memory device included in the main memory of the host 310, and may determine a latency of an area accessed by each of the plurality of physical functions according to which memory device is included.
[0151] According to an embodiment, the storage controller 331 can schedule input / output requests to a region based on the latency of the region. Figures 1 to 13 As described, the storage controller 331 may perform credit-based scheduling. By performing credit-based scheduling by the storage controller 331, data throughput to the region may be maintained.
[0152] The storage controller 331 may allocate a buffer memory area included in the storage controller 331 based on a target QoS of each of the physical functions PF1, PF2, and PF3 and a data throughput of an area accessed by each of the physical functions PF1, PF2, and PF3. For example, when a target QoS of a specific physical function is higher and a data throughput of a main memory area accessed by the physical function is lower, the storage controller 331 may allocate a larger buffer memory area.
[0153] According to an embodiment, even if the latency of each main memory area of the main memory of the host 310 changes, the storage device 330 can maintain the data throughput to each main memory area, and can maintain the QoS provided by the physical functions PF1, PF2, and PF3. For example, regardless of how the latency of each area of the main memory of the host 310 changes, the data throughput of each main memory area can be maintained.
[0154] According to an embodiment, input / output requests to the main memory may be scheduled based on the latency of each region of the main memory to maintain a uniform throughput of the input / output requests.
[0155] According to an embodiment, the throughput of input / output requests to the main memory within a predetermined period of time may be adjusted to process operations accompanying the input / output requests to ensure QoS of the operations.
[0156] Problems to be solved by the present inventive concept are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood from the above description by those skilled in the art.
[0157] While example embodiments have been shown and described above, it will be apparent to those skilled in the art that modifications and variations may be made without departing from the scope of the inventive concept as defined by the appended claims.
Claims
1. A method for scheduling input / output requests to a main memory, the method include: acquiring information of a plurality of memory devices included in the main memory; respectively measuring a plurality of delays of the plurality of memory devices according to a unit data size of a type of each memory device of the plurality of memory devices; grouping the plurality of memory devices into a plurality of memory groups; Determine the size of the time window to be repeated periodically; setting a target processing size for each of the plurality of memory groups based on a size of the time window and a latency of each of the plurality of memory groups; assigning, for each time window, an integral value for each memory group based on the target processing size for each memory group in the plurality of memory groups; as well as A plurality of input / output requests queued in an input / output queue associated with a corresponding memory bank of the plurality of memory banks are processed using the allocated credits of each memory bank of the plurality of memory banks.
2. The method according to claim 1, in, Setting the target processing size of each memory group includes: setting the target processing size of each memory group so that the sum of values obtained by multiplying the delay per unit data size of each memory group in the multiple memory groups by its target processing size is not greater than the size of the time window.
3. The method according to claim 2, in, Setting the target processing size of each memory group includes: setting a ratio of the target processing size of each memory group equal to a performance ratio of each memory group, and The performance ratio of each memory group is determined according to the delay of each memory group.
4. The method according to claim 2, in, Setting the target processing size of each memory group includes setting the target processing size of each memory group to have the same value.
5. The method according to claim 1, in, The integrated value of each memory group corresponds to a number obtained by dividing the target processing size of each memory group by the unit data size of each memory group.
6. The method according to claim 1, in, Processing the plurality of input / output requests includes processing input / output requests to a memory bank and deducting a credit from the memory bank, and The size of the deducted points is determined based on the data size of the input / output request.
7. The method according to claim 1, in, Processing the plurality of input / output requests includes, when credits allocated to a memory group are exhausted, deferring processing of input / output requests for the memory group until credits are allocated to the memory group.
8. The method according to claim 1, further comprising: include: Distribute the credits allocated to each memory group to read requests and write requests, Wherein, processing the plurality of input / output requests includes using the distributed credits to process read requests and write requests among the plurality of input / output requests.
9. The method according to claim 1, in, Grouping the plurality of memory devices into the plurality of memory groups is performed based on a type of each of the plurality of memory devices such that memory devices of each of the plurality of memory groups have a same type.
10. The method according to claim 9, in, The type of each memory device of the plurality of memory devices corresponds to one of a dual in-line memory module and a computing express link memory module.
11. The method according to claim 9, in, Grouping the plurality of memory devices into the plurality of memory groups is further performed based on a latency of each of the plurality of memory devices.
12. The method according to claim 1, in, Measuring the plurality of delays includes measuring a turnaround time until each memory device completes the input / output request.
13. A method for scheduling input / output requests to a main memory, the method include: acquiring information of a plurality of memory devices included in the main memory; respectively measuring a plurality of delays of the plurality of memory devices according to a unit data size of a type of each memory device of the plurality of memory devices; grouping the plurality of memory devices into a plurality of memory groups according to a memory type and a delay of each of the plurality of memory devices, wherein each memory type includes at least two or more memory groups; Determine the size of the time window to be repeated periodically; setting a target processing size for each memory type of the plurality of memory devices based on the size of the time window and the latency of each memory type; allocating a credit value to each memory type based on the target processing size of each memory type; distributing the integrated value of each memory type to the at least two or more memory groups of each memory type based on a delay of each memory group among the at least two or more memory groups of each memory type; and A plurality of input / output requests queued in an input / output queue associated with a corresponding memory bank of the plurality of memory banks are processed using the allocated credits of each memory bank.
14. The method according to claim 13, in, Measuring the plurality of delays is performed when there is an address translation cache hit for the input / output request.
15. The method according to claim 13, in, Processing the multiple input / output requests includes: processing the input / output requests to the memory group; and deducting the points of the memory group according to the number of points determined based on the data size of the input / output request and whether the input / output request hits the address translation cache.
16. The method according to claim 13, in, Grouping the plurality of memory devices into the plurality of memory groups comprises: classifying the plurality of memory devices according to a memory type of each of the plurality of memory devices; and The memory devices classified into the same memory type are grouped into at least two memory groups according to a delay of each memory device classified into the same memory type.
17. The method according to claim 13, in, Grouping the plurality of memory devices into the plurality of memory groups comprises: classifying the plurality of memory devices according to a memory type of each of the plurality of memory devices; and Memory devices classified into the same memory type are grouped into at least two memory groups according to a physical location of each memory device classified into the same memory type.
18. A storage device, the storage device include: a non-volatile memory device; as well as A memory controller comprising a buffer memory and configured to: virtualizing resources of the non-volatile memory device to provide multiple physical functions to a host; as well as Each of the plurality of physical functions is assigned to a corresponding region of the buffer memory based on target quality of service of the plurality of physical functions and latency of a main memory region of a main memory associated with the host, the main memory region being accessed by the plurality of physical functions.
19. The storage device according to claim 18, in, The memory controller is configured to perform integration-based scheduling based on the delays of the main memory regions to maintain data throughput of the main memory regions as the delay of each of the main memory regions changes, and The size of the corresponding area of the buffer memory is determined based on the data throughput and the target quality of service.
20. The storage device according to claim 18, in, The storage controller is configured to: Acquire information of a plurality of memory devices included in a main memory of the host; as well as A latency of the main memory region of the main memory accessed by each of the plurality of physical functions is determined.
Citation Information
Cited By
Flow table unloading system, equipment and cluster
CN119211121A
Flow table offloading system, device and cluster
CN119211121B
