Intelligent data distribution and flow tracking of storage devices

Abstract

The present application discloses intelligent data diversion and flow tracking of a storage device. The disclosed intelligent data diversion includes the following steps: obtaining an IO command; obtaining an update parameter according to the IO command; and allocating a flow to the IO command according to the update parameter.

Description

Technical Field

[0001] The present application relates to storage technology, and more particularly to identifying data accessing a storage device as multiple flows, and adaptively tracking the identified multiple data flows. Background Art

[0002] Figure 1 The block diagram of the storage device is shown. The storage device 102 is coupled to the host to provide storage capacity for the host. The host and the storage device 102 can be coupled in a variety of ways, including but not limited to connecting the host and the solid-state storage device 102 through, for example, SATA (Serial Advanced Technology Attachment), SCSI (Small Computer System Interface), SAS (Serial Attached SCSI), IDE (Integrated Drive Electronics), USB (Universal Serial Bus), PCIE (Peripheral Component Interconnect Express, PCIe), NVMe (NVM Express), Ethernet, Fiber Channel, wireless communication network, etc. The host can be an information processing device that can communicate with the storage device in the above manner, such as a personal computer, a tablet computer, a server, a portable computer, a network switch, a router, a cellular phone, a personal digital assistant, etc. The storage device 102 includes an interface 103 , a control component 104 , one or more NVM chips 105 , and a DRAM (Dynamic Random Access Memory) 110 .

[0003] NAND flash memory, phase change memory, FeRAM (Ferroelectric RAM), MRAM (Magnetic Random Access Memory), RRAM (Resistive Random Access Memory), etc. are common NVMs.

[0004] The interface 103 may be adapted to exchange data with the host via, for example, SATA, IDE, USB, PCIE, NVMe, SAS, Ethernet, Fibre Channel, etc.

[0005] The control component 104 is used to control data transmission between the interface 103, the NVM chip 105 and the DRAM 110, and is also used for storage management, mapping of host logical addresses to flash physical addresses, erase leveling, bad block management, etc. The control component 104 can be implemented in a variety of ways, such as software, hardware, firmware, or a combination thereof. For example, the control component 104 can be in the form of an FPGA (Field-programmable gate array), an ASIC (Application Specific Integrated Circuit), or a combination thereof. The control component 104 can also include a processor or a controller, in which software is executed to manipulate the hardware of the control component 104 to process IO (Input / Output) commands. The control component 104 can also be coupled to the DRAM 110 and can access the data of the DRAM 110. The data of the FTL table and / or cached IO commands can be stored in the DRAM.

[0006] The control component 104 includes a flash memory interface controller (or medium interface controller, flash memory channel controller), which is coupled to the NVM chip 105 and issues commands to the NVM chip 105 in a manner that complies with the interface protocol of the NVM chip 105 to operate the NVM chip 105, and receives command execution results output from the NVM chip 105. Known NVM chip interface protocols include "Toggle", "ONFI" and the like.

[0007] In a solid-state storage device, the FTL (Flash Translation Layer) is used to maintain the mapping information from the logical address to the physical address. The logical address constitutes the storage space of the solid-state storage device perceived by the upper-layer software such as the operating system. The physical address is the address used to access the physical storage unit of the solid-state storage device. In the related art, the address mapping can also be implemented using the intermediate address form. For example, the logical address is mapped to the intermediate address, and then the intermediate address is further mapped to the physical address.

[0008] The table structure that stores the mapping information from logical addresses to physical addresses is called an FTL table. The FTL table is an important metadata in a solid-state storage device. Usually, the data items of the FTL table record the address mapping relationship in units of data pages in the solid-state storage device.

[0009] Figure 2A schematic diagram of a large block is shown. A large block includes physical blocks from each of a plurality of logical units (called logical unit groups). Optionally, each logical unit provides a physical block for a large block. As an example, a large block is constructed on every 16 logical units (LUNs). Each large block includes 16 physical blocks from each of the 16 logical units (LUNs). Figure 2 In the example of , chunk 0 includes physical chunk 0 from each of the 16 logical units (LUNs), and chunk 1 includes physical chunk 1 from each logical unit (LUN). There are many other ways to structure chunks.

[0010] As an optional way, page stripes are constructed in large blocks, and physical pages with the same physical address in each logical unit (LUN) constitute a "page stripe". Figure 2 In , physical pages 0-0, 0-1, ... and 0-x constitute page stripe 0, where physical pages 0-0, 0-1, ... and 0-14 are used to store user data, and physical pages 0-15 are used to store check data calculated based on all user data in the stripe. Similarly, Figure 2 In FIG. 2 , physical page 2-0, physical page 2-1, ... and physical page 2-x constitute page stripe 2. Optionally, the physical page for storing the check data may be located at any position in the page stripe.

[0011] When data is repeatedly written to a logical page, the corresponding relationship between the logical page address and the latest physical page address is recorded in the FTL table entry, and the data recorded in the physical page address that has been written but no longer referenced (for example, there is no record in the FTL table) becomes "garbage" (data). Data that has been written and referenced (for example, there is a record in the FTL table) is called valid data, and "garbage" is called dirty data. The physical block containing dirty data is called a "dirty physical block", and the physical block that has not been written with data is called a "free physical block". Summary of the invention

[0012] The data recorded in the storage device is updated frequently. The time interval between two updates of data is called the life cycle of the data. Various data have different life cycles. For example, the life cycle of video data is long or close to read-only, while the life cycle of data cached as an Internet web page is relatively short, perhaps a few hours, days or months. As a result, the data recorded in the storage device gradually becomes invalid due to updates as it is used. The space occupied by invalid data is released through operations such as garbage collection.

[0013] Since valid data and invalid data are mixed and recorded in the storage medium, the garbage collection process needs to move valid data. The movement of valid data causes the write amplification effect, which affects the performance and life of the storage device. The garbage collection operation of the existing technology selects storage media with a higher proportion of invalid data for garbage collection in an effort to reduce write amplification.

[0014] According to an embodiment of the present application, data written to a storage device is divided into multiple streams, and the data of each stream has the same or similar life cycle. And according to the change of the life cycle of the data of each stream, the data belonging to the stream is still divided into the same stream even if the life cycle fluctuates. The stream also has the opportunity to be split or merged so that the stream better indicates the life cycle characteristics of the data to which it belongs.

[0015] The same stream is stored in the same or adjacent physical blocks or large blocks of the storage medium as much as possible, and different streams are stored in different physical blocks or large blocks of the storage medium as much as possible. The physical blocks or large blocks are recycled as a whole during the garbage collection process. Since data belonging to the same stream has a similar life cycle, it is easier to obtain physical blocks or large blocks with a larger proportion of invalid data during garbage collection, thereby reducing the write amplification caused by the garbage collection process.

[0016] In addition to being used in the garbage collection process, data streaming also provides an opportunity for storage devices to optimize their IO processing.

[0017] According to the first aspect of the present application, a first method for allocating data streams according to the first aspect of the present application is provided, which includes the following steps: obtaining an IO command; obtaining an update parameter according to the IO command; and allocating a stream to the IO command according to the update parameter.

[0018] According to the first method for allocating data streams of the first aspect of the present application, a second method for allocating data streams of the first aspect of the present application is provided, wherein the IO command carries update parameters.

[0019] According to the first method for allocating data streams of the first aspect of the present application, a third method for allocating data streams of the first aspect of the present application is provided, wherein a data unit accessed by an IO command is obtained, and an update time interval of the data unit is obtained as an update parameter.

[0020] According to the first method for allocating data streams of the first aspect of the present application, a fourth method for allocating data streams of the first aspect of the present application is provided, wherein a data unit accessed by an IO command is obtained, an update time interval of the data unit is obtained, and an average value, a median value, and a mode value of one or more update time intervals of the data unit are used as update parameters.

[0021] According to the first method for allocating data streams in the first aspect of the present application, a fifth method for allocating data streams in the first aspect of the present application is provided, wherein a data unit accessed by an IO command is obtained, an update time interval of the data unit is obtained, and an update parameter is calculated according to g'=(1-k)*g+k*(dt), wherein g' is the calculated update parameter, g is the old update parameter, dt is the update time interval, 0 <k<1。

[0022] According to the method for allocating data streams of any one of items 1 to 5 of the first aspect of the present application, a sixth method for allocating data streams of the first aspect of the present application is provided, wherein the data unit represents a logical address space or a physical address space of a specified size.

[0023] According to the sixth method for allocating data streams of the first aspect of the present application, a seventh method for allocating data streams of the first aspect of the present application is provided, wherein a data unit accessed by an IO command is obtained, and a stream is allocated to the data unit according to update parameters of the obtained data unit.

[0024] According to the seventh method for allocating data streams of the first aspect of the present application, an eighth method for allocating data streams of the first aspect of the present application is provided, wherein data units with the same update parameters or update parameter distances less than a threshold are allocated to the same stream.

[0025] According to the seventh method for allocating data streams of the first aspect of the present application, a ninth method for allocating data streams of the first aspect of the present application is provided, which also includes allocating the first stream to the IO command accessing the first data unit if the distance between the update parameter of the first data unit and the update parameter of the first stream is less than a threshold.

[0026] According to the seventh method for allocating data streams of the first aspect of the present application, the tenth method for allocating data streams of the first aspect of the present application is provided, wherein the first stream is allocated to data units with the same update parameters; the second stream is allocated to data units whose update parameter distances are within a threshold, and the third stream is allocated to other data units.

[0027] According to the method for allocating data streams of any one of the third to eighth items of the first aspect of the present application, the eleventh method for allocating data streams of the first aspect of the present application is provided, wherein the update parameters of the acquired data unit are compared with the update parameters of one or more streams, and the IO command is allocated to the stream whose update parameters are within a threshold value from the acquired update parameters.

[0028] According to the method for allocating data streams of any one of items 8 to 11 of the first aspect of the present application, a twelfth method for allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating streams for data units accessed by IO commands, update parameters of the data streams are calculated based on update parameters of the data units.

[0029] According to the twelfth method for allocating data streams of the first aspect of the present application, a thirteenth method for allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating a first stream to a data unit accessed by an IO command, the update parameters of the data units of the mean, median, or mode of the update parameters of multiple or all data units allocated to the first stream are used as the update parameters of the first stream.

[0030] According to the twelfth method for allocating data streams of the first aspect of the present application, a fourteenth method for allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating a first stream to a data unit accessed by an IO command, update parameters of multiple or all data units allocated to the first stream are weighted averaged, or the result of low-pass filtering is used as the update parameter of the first stream.

[0031] According to the twelfth method for allocating data streams of the first aspect of the present application, a fifteenth method for allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating a first stream to a data unit accessed by an IO command, new update parameters of the first stream are calculated using update parameters of the data unit and old update parameters of the first stream.

[0032] According to the fifteenth method for allocating data streams of the first aspect of the present application, a sixteenth method for allocating data streams of the first aspect of the present application is provided, wherein the new update parameter is a weighted sum of the old update parameter of the stream and the update parameter of the data unit.

[0033] According to the method for allocating data streams of any one of items 10 to 16 of the first aspect of the present application, a method for allocating data streams of the seventeenth aspect of the first aspect of the present application is provided, wherein, if the distance between the acquired data unit update parameters and the update parameters of one or more streams is greater than a threshold, a new stream is created, and a new stream or a specified stream is allocated to the IO command.

[0034] According to the method for allocating data streams of any one of items 10 to 16 of the first aspect of the present application, an eighteenth method for allocating data streams of the first aspect of the present application is provided, wherein update parameters of one or more new streams are created based on the update parameters of the streams, and the update parameters of each stream represent a stream.

[0035] According to the seventeenth or eighteenth method for allocating data streams of the first aspect of the present application, a nineteenth method for allocating data streams of the first aspect of the present application is provided, wherein second update parameters and third update parameters are generated based on the update parameters of the first stream, the second update parameters represent the second stream, and the third update parameters represent the third stream.

[0036] According to the eighteenth method for allocating data streams of the first aspect of the present application, a second method for allocating data streams of the first aspect of the present application is provided, wherein the average value or statistical value of the second update parameter and the third update parameter is the first update parameter, and the first update parameter is greater than the second update parameter.

[0037] According to the method for allocating data streams of any one of items 11 to 20 of the first aspect of the present application, a twenty-first method for allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating the first stream to the IO command, new update parameters of the first stream are calculated.

[0038] According to the method for allocating data streams of any one of items 1 to 21 of the first aspect of the present application, a twenty-second method for allocating data streams of the first aspect of the present application is provided, wherein the IO command is allocated to the initial stream.

[0039] According to the method for allocating data streams of any one of items 1 to 22 of the first aspect of the present application, a method for allocating data streams of the twenty-third aspect of the present application is provided, wherein the update parameters are compared with the update parameters of one or more existing streams, and the IO command is allocated a stream whose update parameters are within a threshold distance from the obtained update parameters.

[0040] According to the method of allocating data streams of any one of items 1 to 23 of the first aspect of the present application, a twenty-fourth method of allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating a stream to an IO command, new update parameters of the allocated stream are calculated.

[0041] According to the method of allocating data streams of any one of items 1 to 24 of the first aspect of the present application, a twenty-fifth method of allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating streams to IO commands, the discreteness of the streams to which the IO commands are allocated is calculated.

[0042] According to the twenty-fifth method for allocating data streams of the first aspect of the present application, a twenty-sixth method for allocating data streams of the first aspect of the present application is provided, wherein the discreteness of the stream is the discreteness of update parameters of multiple data units associated with the same stream.

[0043] According to the twenty-fifth or twenty-sixth method for allocating data streams of the first aspect of the present application, a twenty-seventh method for allocating data streams of the first aspect of the present application is provided, wherein the discreteness of the stream is the discreteness of the update parameter of the stream.

[0044] According to the twenty-fifth or twenty-sixth method for allocating data streams of the first aspect of the present application, a twenty-eighth method for allocating data streams of the first aspect of the present application is provided, wherein the discreteness of the stream is the ratio of the discreteness of the update parameters of the stream to the update time interval of the stream.

[0045] According to the method for allocating data streams of any one of items 25 to 28 of the first aspect of the present application, a 29th method for allocating data streams of the first aspect of the present application is provided, wherein the streams are split and / or merged according to discreteness.

[0046] According to the twenty-ninth method for allocating data streams of the first aspect of the present application, the third method for allocating data streams of the first aspect of the present application is provided, wherein two new update parameters are generated according to the update parameters of the stream to split the stream into two streams.

[0047] According to the thirty-first method for allocating data streams of the first aspect of the present application, a thirty-first method for allocating data streams of the first aspect of the present application is provided, wherein the average value or statistical value of the updated parameters of the two streams is the same as the updated parameters of the streams before splitting, and one of the updated parameters of the two streams is greater than the other.

[0048] According to the method for allocating data streams of any one of items 25 to 31 of the first aspect of the present application, a method for allocating data streams of item 32 of the first aspect of the present application is provided, wherein, in response to the discreteness of the first stream being greater than a threshold or the increment of the discreteness of the first stream being greater than a threshold, the first stream is split into a first stream and a second stream.

[0049] According to the thirty-fifth method for allocating data streams of the first aspect of the present application, a thirty-third method for allocating data streams of the first aspect of the present application is provided, wherein a specified part of the discreteness of the first stream is used as the discreteness of the first stream and the second stream after splitting.

[0050] According to the thirty-second method for allocating data streams of the first aspect of the present application, a thirty-fourth method for allocating data streams of the first aspect of the present application is provided, wherein the discreteness of the first stream after splitting is half of the discreteness of the first stream before splitting.

[0051] According to the method for allocating data streams of any one of items 25 to 31 of the first aspect of the present application, there is provided a method for allocating data streams of item 35 of the first aspect of the present application, wherein multiple streams are sorted according to update time intervals or update parameters, the discreteness after merging two or more adjacent streams is calculated, and two or more streams whose discreteness is less than a specified threshold are merged.

[0052] According to the method of allocating data streams of any one of items 11 to 35 of the first aspect of the present application, a thirty-sixth method of allocating data streams of the first aspect of the present application is provided, wherein, in response to allocating a stream to an IO command, an update time interval of the stream to which the IO command is allocated is calculated.

[0053] According to the thirty-fifth method for distributing data streams of the first aspect of the present application, a thirty-seventh method for distributing data streams of the first aspect of the present application is provided, wherein low-pass filtering is performed on multiple update time intervals of the stream.

[0054] According to the method for allocating data streams of any one of items 36 to 37 of the first aspect of the present application, a 38th method for allocating data streams of the first aspect of the present application is provided, wherein the streams are split and / or merged by updating time intervals.

[0055] According to the thirty-eighth method for allocating data streams of the first aspect of the present application, a thirty-ninth method for allocating data streams of the first aspect of the present application is provided, wherein two or more streams whose update time interval is greater than a specified threshold are merged.

[0056] According to the method of allocating data streams of any one of items 36 to 39 of the first aspect of the present application, a method of allocating data streams of the fortieth aspect of the first aspect of the present application is provided, wherein the update time interval of the stream is used as the update time interval of each split stream.

[0057] According to the method of allocating data streams of any one of items 29 to 39 of the first aspect of the present application, a method of allocating data streams of the 41st aspect of the present application is provided, wherein the average value or statistical value of the update parameters of two or more merged streams is used as the update parameter of the merged stream.

[0058] According to the second aspect of the present application, a first storage device according to the second aspect of the present application is provided, which includes a control component and a non-volatile storage medium, and the control component executes the method of allocating data streams of any one of items 1 to 41 of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For those skilled in the art, other drawings can also be obtained based on these drawings.

[0061] Figure 1 A block diagram of a storage device in the prior art;

[0062] Figure 2 It is a schematic diagram of a large block in the prior art;

[0063] Figure 3 It is a schematic diagram of the state change of a large block of data in the prior art;

[0064] Figure 4 A schematic diagram of the flow diversion provided in an embodiment of the present application;

[0065] Figure 5A flow chart for allocating flows to data units provided in an embodiment of the present application;

[0066] Fig. 6A and Figure 6B It is a schematic diagram of splitting and merging of streams provided in an embodiment of the present application;

[0067] Figure 7 A flowchart of splitting and / or merging streams provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0069] Figure 3 It is a schematic diagram of the state changes of large blocks of data.

[0070] Figure 3 The data on the block 1 and block 2 where data is written is shown. As time goes by, invalid data gradually increases. Figure 3 In the figure, T1 indicates the earliest time, T2 indicates the subsequent time, and T3 indicates the latest time. Large blocks of blank squares indicate storage units where data is written and the data is valid, such as physical pages or physical blocks of flash memory, and large blocks of shaded squares indicate storage units where data is written and becomes invalid because the data is updated.

[0071] As time passes from T1 to T2 and T3, the part of valid data on chunk 1 and chunk 2 gradually becomes invalid. And at each time, chunk 1 and chunk 2 each have a proportion of invalid data. At T1, the proportion of invalid data in chunk 1 is greater than that in chunk 2, and at T3, the proportion of invalid data in chunk 1 is still greater than that in chunk 2.

[0072] At time T3, if garbage collection is performed, it is advantageous to select block 1 and not block 2 for the time being, because block 1 stores less valid data. And it is expected that block 2 will be garbage collected at a later time.

[0073] In order to reduce the number of valid data in a large block selected for garbage collection, the garbage collection time of the large block can be postponed. However, in some cases, the demand for storage space on the storage device makes it difficult to implement the strategy of postponing garbage collection. In addition, the valid data on the large block may have a very long life cycle, which makes the benefits of postponing garbage collection strategy negligible.

[0074] Figure 4 A schematic diagram of flow diversion according to an embodiment of the present application is shown.

[0075] The storage device provides, for example, a continuous logical address space. The rightward arrow indicates the direction in which the logical address space increases. The logical address space is divided into a plurality of data units (410, 411, 412, 413, ... 418). Each data unit occupies a logical address space of a specified size or a fixed size. For example, the size of the data unit is 512 bytes, 4K, 16K, or other values.

[0076] Each data unit has a life cycle. For example, data unit 410 is generally updated once a day, and its life cycle is 1 day; data unit 411 is generally updated once a month, and its life cycle is 1 month; data unit 412 is generally updated once an hour, and its life cycle is 1 hour; data unit 415 is generally updated once a year, and its life cycle is 1 year.

[0077] The FTL table of the storage device maps the address of the logical address space to the physical address of the storage medium. In response to the data unit of the logical address space being updated, a new physical address is allocated to the data unit to carry the updated data, so that the data at the old physical address becomes dirty data.

[0078] The data units of the logical address space are divided into multiple streams according to their life cycles. According to an embodiment of the present application, data units with the same or similar life cycles are divided into the same stream. Different data units with greatly different life cycles are divided into different streams. For example, see Figure 4 , the life cycles of data unit 410, data unit 414 and data unit 418 are all 1 day, and these data units are divided into stream S1; the life cycles of data unit 411 and data unit 413 are close (1 month and 1 week respectively), and they are divided into stream S3; and data unit 412 and data unit 416 are divided into stream S2; data unit 415 is divided into stream Sn. Optionally, other data units that are not suitable for being divided into stream S1, stream S2 or S3 are also divided into stream Sn.

[0079] In an optional implementation, the stream has a standard life cycle. For example, the standard life cycle of stream S1 is 1 day, and data units with a life cycle of 0.5 days to 5 days are all classified as stream S1.

[0080] Still optionally or further, the standard life cycle and / or life cycle range of the flow is adjusted to adapt to the change of the life cycle of the data unit. For example, the standard life cycle of the flow is updated using the mean, median or other statistical value of the life cycle of each data unit assigned to the flow.

[0081] Optionally, the time interval between two updates of the data unit history, or the average of multiple updated time intervals, is used as the update parameter of the data unit to replace the life cycle. The data unit is divided into flows according to the update parameter of the data unit. Still optionally, the weighted average of the multiple updated time intervals of the data unit history, or the result obtained by low-pass filtering, is used as the update parameter of the data unit.

[0082] Optionally or further, the update parameters of the stream are calculated based on the update parameters of multiple or all data units assigned to the stream to replace the standard life cycle of the stream. For example, the update parameters of the data units of the mean, median, and mode of the update parameters of multiple or all data units assigned to the stream are used as the update parameters of the stream. Still optionally, the update parameters of multiple or all data units assigned to the stream are weighted averaged, or the result of low-pass filtering is used as the update parameters of the stream.

[0083] According to an embodiment of the present application, the stream is dynamic. In some cases, a data unit originally divided into one stream is changed to be divided into another stream; in other cases, a single stream is split into multiple streams; and in other cases, two or more streams are merged into a single stream. This reflects the change in the way data units are accessed. For example, the life cycle of a storage unit of a document being edited is very short; and when the document is created, the life cycle of its storage unit will become very long. As the life cycle of a storage unit changes greatly, it also changes to the stream into which the storage unit is divided. As another example, the update parameters of a stream show greater volatility, and the stream is split into two or more streams to reduce the volatility of the update parameters of each stream.

[0084] Still optionally, the life cycles of some storage units are difficult to identify, and the storage units whose life cycles are difficult to identify are all divided into designated data streams.

[0085] exist Figure 4 In the example, the logical address space provided by the storage device is divided into a plurality of data units. In an optional embodiment, the storage device provides a physical address space, and an application program of a host accessing the storage device accesses the logical address space, and a mapping from a logical address to a physical address is completed in the host. Also in the host, the logical address space is divided into a plurality of data units, and the host counts update parameters of each data unit, divides the data units into streams, and calculates update parameters of the streams.

[0086] Still optionally, the data units are divided on other address spaces. The other address spaces are physical addresses or logical addresses to be mapped to physical addresses. The data units of the other address spaces can be updated. Update parameters are calculated for each data unit to divide the data unit into streams, and update parameters for the streams are calculated.

[0087] Figure 5 This is a flowchart of allocating flows to data units according to an embodiment of the present application.

[0088] Allocating streams to data units also allocates streams to IO commands that write data to data units.

[0089] Taking a storage device as an example, an IO command (510) is received from a host or other device, and the IO command indicates, for example, a logical address to which data is to be written, and the logical address corresponds to one or more data units in the logical address space. For the purpose of simplicity, as an example, the IO command accesses a single data unit (referred to as DU1).

[0090] Calculate update parameters (520) for the data unit DU1 accessed by the IO command. The update parameters indicate the life cycle of the data unit, or the time when it is expected to be updated. As an example, the update parameters of the accessed data unit are carried in the IO hit. As another example, the time when it was last updated or updated is recorded for each data unit, and the difference between the current time and the time when it was last updated or updated is used as or calculated as the update parameter. As still another example, the time intervals according to the previous one or more updates of each data unit are recorded, and the average, median, mode or other statistical value of one or more time intervals is used as the update parameter. In another example, the time when it was last updated or updated is recorded for each data unit, and the update parameters are recorded, and based on the difference dt between the current time and the previous or last update time, and the recorded update parameter g, a new update parameter is calculated (for example, g'=(1-k)*g+k*(dt), where g' is the new update parameter obtained after calculation, 0 <k<1)。

[0091] The obtained update parameter of the data unit is compared with the update parameters of one or more streams, and a stream is allocated to the data unit (530). For example, the stream having the stream update parameter closest to the data unit update parameter is allocated to the data unit.

[0092] As another example, if there is no stream, or the update parameter of no stream is close enough to the update parameter of the data unit, a new stream is created, and the data unit is assigned to the newly created stream. As another example, if the update parameter of no stream is close enough to the update parameter of the data unit, the data unit is assigned to a designated stream. The stream is represented by, for example, a stream identifier, and the stream is assigned to the data unit by, for example, attaching the stream identifier to the data of the updated data unit.

[0093] Optionally, as the data units are assigned to the stream, the update parameters of the stream are also recalculated (540). As an example, the average, median, mode or other statistical value of the update parameters of one, more or all data units assigned to the stream is used as the update parameter of the stream. As another example, the update parameters of each stream are recorded, and in response to the stream being assigned to a new data unit, the new update parameters of the stream are calculated using the update parameters of the new data unit and the current update parameters of the stream. Still as an example, the new update parameters of the stream are the weighted sum of the current update parameters of the stream and the update parameters of the new data unit.

[0094] Fig. 6A and Figure 6B It is a schematic diagram of splitting and merging of flows according to an embodiment of the present application.

[0095] Fig. 6A and Figure 6B In the above diagram, the coordinate axes represent update parameters. For the purpose of description, the update parameters on the coordinate axes represent both the update parameters of the data unit and the update parameters of the stream. The update parameters of the data unit are directly comparable to the update parameters of the stream. In some cases, the update parameters of the stream are calculated from the update parameters of the data unit, but the update parameters of the stream are still directly comparable to the update parameters of the data.

[0096] See also Fig. 6A , there are already two streams, stream S1 and stream S2. The rectangular boxes where streams S1 and S2 are located represent the update parameters of the streams. For example, the starting point or midpoint of the coordinate axis covered by the rectangular box is used as the update parameter of the stream. The data units whose update parameters fall within the range covered by the rectangular box will be assigned to the stream represented by the rectangular box. The circles on the coordinate axis represent data units, and the positions of the circles on the coordinate axis represent the update parameters of the data units. Therefore, the data units represented by the circles in the rectangular box are assigned to the stream represented by the rectangular box.

[0097] Continue to view Fig. 6A In the rectangular box where stream S1 is located, multiple data units have formed three groups (G1, G2 and G3) according to their updated parameters. In the rectangular box where stream S2 is located, multiple data units have formed two groups (G4 and G5) according to their updated parameters. Among them, data unit group G3 and data unit group G4 are close to each other. Data unit group G4 and data unit group G5 are far apart.

[0098] According to the embodiment of the present application, stream S1 and stream S2 are split to obtain new streams S1', S2 and S3. New stream S1' contains data unit group G1 and data unit group G2; new stream S2' contains data unit group G3 and data unit group G4, and stream S3 contains data unit group G5. Figure 6B shown.

[0099] Optionally, two or more streams are merged, and the merged stream contains all data unit groups of the streams participating in the merge.

[0100] For example, streams can be split or merged by changing the update parameters of the streams. Fig. 6A ,The position of the midpoint of the rectangular box representing the flow on the coordinate axis indicates the update parameters of the flow. The flow can be split or merged by changing or creating the update parameters of the flow.

[0101] Although, Fig. 6A and Figure 6B The rectangular box corresponding to the stream is shown to contain multiple data units. Optionally, in response to a change in the stream (e.g., splitting or merging), the data units already assigned to the stream do not need to be reattached with a new stream identifier, but for a newly received IO request, the changed stream is assigned to the data units accessed by it.

[0102] See also Fig. 6A Optionally, the parameters of the stream may be variable. For example, in response to more data units whose update parameters are close to the data unit group G3 being allocated to stream S1, the update parameters of stream S1 increase, so that the rectangular box corresponding to stream S1 moves rightward on the coordinate axis.

[0103] exist Fig. 6A and Figure 6B In the example of , splitting or merging streams according to the discreteness of the update parameter is shown. Still optionally, streams are also merged according to the update time interval of the streams. Updates to the task data units assigned to the streams are regarded as updates to the streams. For two or more streams with a large (e.g., greater than a specified threshold) update time interval, it is advantageous to merge them into the same stream.

[0104] Figure 7 A flowchart of splitting and / or merging streams according to an embodiment of the present application is shown.

[0105] According to an embodiment of the present application, the identification of the stream is dynamic. After initialization, an initial stream is provided, and any IO command for accessing any data unit is assigned to the initial stream (710). The initial stream has initial stream update parameters.

[0106] During operation of the storage device, an IO command is received (720).

[0107] Update parameters are calculated for the data unit accessed by the IO command (730).

[0108] The obtained updated parameters of the data unit are compared with the updated parameters of one or more existing streams to allocate a stream to the data unit (740). For example, the stream with the stream updated parameters closest to the updated parameters of the data unit is allocated to the data unit.

[0109] According to the update parameters of the data unit, the update parameters of the stream to which the data unit is assigned are recalculated (750). Optionally or further, other parameters are also calculated for the stream to which the data unit is assigned. For example, the update time interval of the stream, the discreteness of the update parameters of the stream. The update time interval of the stream refers to the time interval between the current update of the stream and the last update. Optionally, the update time interval of the stream is also low-pass filtered to reduce the large fluctuations in the update time interval parameters of the stream caused by occasional short time interval updates. The discreteness of the update parameters of the stream represents the degree of deviation between the update parameters of the data unit assigned to the stream and the update parameters of the stream, for example, each time the data unit is assigned to the stream, the sum of the differences between the update parameters of the data unit and the update parameters of the stream, the weighted sum of the differences between the update parameters of the data unit and the update parameters of the stream, or the statistical value of the differences between the update parameters of the data unit and the update parameters of the stream.

[0110] In response to allocating data units to the flow, periodically or when necessary, obtaining the updated discreteness of the flow (760). Optionally, the discreteness of the updated parameters of the flow is used as the discreteness of the flow. Still optionally, the ratio of the discreteness of the updated parameters of the flow to the update time interval of the flow is used as the discreteness of the flow.

[0111] Split the stream and / or merge the stream (770) according to the discreteness of the stream. The stream is represented by parameters including the update parameter, update time interval and / or discreteness of the stream. The stream is created by creating one or more new stream parameters according to the parameters of the stream. For example, the stream is split into two streams by generating two update parameters according to the update parameters of the stream, wherein the average value or statistical value of the two new update parameters is the original update parameter, and one of the new update parameters is greater than the other new update parameter. Still as an example, a specified part (such as 1 / 2) of the discreteness of the stream is used as the discreteness of each split stream. The update time interval of the stream is used as the update time interval of each split stream.

[0112] Similarly, the streams are merged by generating parameters of a new single stream based on the parameters of two or more streams, for example, taking the average or statistical value of the updated parameters of the two or more streams to be merged as the updated parameters of the merged stream.

[0113] Optionally, for a single stream, in response to its discreteness being too large or increasing rapidly, the stream is split. For multiple streams, in response to having a smaller discreteness after merging them, the streams are merged. Still optionally, the multiple streams are sorted according to their discreteness, the discreteness of two or more adjacent streams after merging is calculated, and the two or more streams corresponding to the calculated discreteness having the smallest discreteness or less than a specified threshold are merged.

[0114] Therefore, during the operation of the storage device, the changes in the stream caused by the IO request are tracked by updating the parameters of the stream, so that the same stream identifier is attached to the data units with the same or similar life cycle.

[0115] Furthermore, the storage device also places data units belonging to the same stream in the same or adjacent physical blocks or large blocks, so that during garbage collection, there is an opportunity to reasonably select the physical blocks or large blocks to be recycled, thereby reducing write amplification and improving the performance of the storage device.

[0116] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application. Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for allocating data streams, characterized in that: The following steps are involved: Get IO commands; Get update parameters according to IO commands; as well as Allocate streams for IO commands based on update parameters; Among them, obtaining update parameters according to IO commands includes: Obtain the data unit accessed by the IO command, obtain the update time interval of the data unit, and use the average value, median value, mode value, weighted average value or low-pass filtering result of one or more update time intervals of the data unit as an update parameter.

2. A method for distributing data streams according to claim 1, characterized in that Get the data unit accessed by the IO command, get the update time interval of the data unit, and calculate the update parameter according to g'=(1-k)*g+k*(dt), where g' is the calculated update parameter, g is the old update parameter, dt is the update time interval, 0 <k<1。 3. The method for distributing data streams according to claim 1, characterized in that A data unit represents a logical address space or a physical address space of a specified size.

4. A method for distributing data streams according to claim 1, characterized in that The data unit accessed by the IO command is obtained, and a stream is allocated to the data unit according to the update parameters of the obtained data unit.

5. A method for distributing data streams according to claim 4, characterized in that Data units with the same update parameter or with a distance of update parameters less than a threshold are allocated to the same flow.

6. A method for distributing data streams according to claim 5, characterized in that The method further includes allocating the first stream to the IO command accessing the first data unit if the distance between the update parameter of the first data unit and the update parameter of the first stream is less than a threshold.

7. A method for distributing data streams according to claim 6, characterized in that The first stream is allocated to data units with the same update parameters; the second stream is allocated to data units whose update parameters are within a threshold; and the third stream is allocated to other data units.

8. The method for distributing data streams according to claim 1, characterized in that The acquired update parameter of the data unit is compared with the update parameter of one or more streams, and a stream whose update parameter is within a threshold value from the acquired update parameter is allocated to the IO command.

9. A method for distributing data streams according to claim 8, characterized in that In response to allocating a stream for a data unit accessed by the IO command, an update parameter of the data stream is calculated based on the update parameter of the data unit.

10. Method for distributing data streams according to claim 9, characterized in that In response to allocating a first stream to data units accessed by an IO command, an update parameter of a data unit having a mean, a median or a mode of update parameters of multiple or all data units allocated to the first stream is used as an update parameter of the first stream.

11. Method for distributing data streams according to claim 10, characterized in that In response to allocating the first stream to the data unit accessed by the IO command, weighted averaging or low-pass filtering the update parameters of multiple or all data units allocated to the first stream are used as the update parameters of the first stream.

12. Method for distributing data streams according to claim 11, characterized in that In response to allocating the first stream for the data unit accessed by the IO command, new update parameters of the first stream are calculated using update parameters of the data unit and old update parameters of the first stream.

13. Method for distributing data streams according to claim 12, characterized in that The new update parameter is the weighted sum of the old update parameter of the stream and the update parameter of the data unit.

14. The method for distributing data streams according to claim 2, characterized in that If the distances between the acquired data unit update parameters and the update parameters of one or more streams are greater than a threshold, a new stream is created, and the new stream or the designated stream is allocated to the IO command.

15. Method for distributing data streams according to claim 14, characterized in that Create one or more new stream update parameters according to the stream update parameters, each stream update parameter represents a stream.

16. Method for distributing data streams according to claim 15, characterized in that According to the update parameters of the first stream, second update parameters and third update parameters are generated, the second update parameters represent the second stream, and the third update parameters represent the third stream.

17. Method for distributing data streams according to claim 16, characterized in that An average value or a statistical value of the second update parameter and the third update parameter is the first update parameter, and the first update parameter is greater than the second update parameter.

18. Method for distributing data streams according to claim 17, characterized in that In response to the first stream being assigned to the IO command, new update parameters for the first stream are calculated.

19. A method for allocating data streams according to any one of claims 1 to 18, characterized in that: in, Assign IO commands to the initial stream.

20. Method for distributing data streams according to claim 19, characterized in that The update parameter obtained according to the IO command is compared with the update parameters of one or more existing flows, and the flow whose update parameter is within a threshold value from the obtained update parameter is allocated to the IO command.

21. Method for distributing data streams according to claim 20, characterized in that In response to allocating a stream to the IO command, new update parameters of the allocated stream are calculated.

22. A method for allocating data streams according to any one of claims 1 to 18, characterized in that: In response to the stream being assigned to the IO command, a dispersion of the streams to which the IO command is assigned is calculated.

23. Method for distributing data streams according to claim 22, characterized in that The discreteness of a flow is the discreteness of update parameters of multiple data units associated with the same flow.

24. Method for distributing data streams according to claim 23, characterized in that The discreteness of the flow is the discreteness of the update parameters of the flow.

25. Method for distributing data streams according to claim 24, characterized in that The discreteness of a flow is the ratio of the discreteness of the update parameters of the flow to the update time interval of the flow.

26. Method for distributing data streams according to claim 25, characterized in that Split and / or merge streams based on discreteness.

27. Method for distributing data streams according to claim 26, characterized in that Generate two new update parameters based on the update parameters of the stream to split the stream into two streams.

28. Method for distributing data streams according to claim 27, characterized in that The average values ​​or statistical values ​​of the updated parameters of the two streams are the same as the updated parameters of the stream before the split, and one of the updated parameters of the two streams is greater than the other.

29. Method for distributing data streams according to claim 28, characterized in that In response to the first stream discreteness being greater than a threshold or the increment of the first stream discreteness being greater than a threshold, the first stream is split into the first stream and the second stream.

30. Method for distributing data streams according to claim 29, characterized in that The specified part of the discreteness of the first stream is used as the discreteness of the first stream and the second stream after the splitting.

31. A method for distributing data streams according to claim 30, characterized in that The discreteness of the first stream after the split is half of the discreteness of the first stream before the split.

32. Method for distributing data streams according to claim 31, characterized in that Sort multiple streams by update time interval or update parameter, calculate the discreteness of two or more adjacent streams after merging, and merge two or more streams whose discreteness is less than a specified threshold.

33. A method for distributing data streams according to claim 31, characterized in that In response to the stream being assigned to the IO command, an update time interval of the stream to which the IO command is assigned is calculated.

34. Method for distributing data streams according to claim 33, characterized in that Low pass filter the stream over multiple update intervals.

35. Method for distributing data streams according to claim 34, characterized in that Split and / or merge streams by update time interval.

36. Method for distributing data streams according to claim 35, characterized in that Merge two or more streams whose update interval is greater than the specified threshold.

37. Method for distributing data streams according to claim 36, characterized in that The update time interval of the stream is used as the update time interval of each split stream.

38. A method for distributing data streams according to claim 36, characterized in that The average value or statistical value of the update parameters of the merged two or more streams is used as the update parameter of the merged stream.

39. A storage device, characterized in that: The device comprises a control component and a non-volatile storage medium, wherein the control component executes the method according to any one of claims 1-38.

Images