A data processing method and device

Abstract

This application provides a data processing method and apparatus, relating to the field of storage technology. Based on the characteristics of read and / or write requests corresponding to different volumes of a storage device, a personalized flushing method is used to flush different volumes to disk, thereby improving the flushing performance of the storage device. Specifically, the storage device receives multiple operation requests sent by the host, including read and / or write requests. These multiple operation requests are stored in a cache. Model features are obtained, which represent the characteristics of the multiple operation requests corresponding to a first volume of non-volatile storage media in the cache. The first volume is flushed to disk according to the model features, so that dirty data corresponding to the first volume in the cache is written to the first volume. This application embodiment is used for data processing.

Description

[0001] This application is a divisional application of patent application number 201711107599.9, filed on November 10, 2017, entitled "A Data Processing Method and Apparatus". Technical Field

[0002] This application relates to the field of storage technology, and in particular to a data processing method and device. Background Technology

[0003] Since the read and write speeds of cache in storage devices are usually higher than those of non-volatile storage media such as physical disks, meaning that the read and write speeds of non-volatile storage media and cache cannot be matched, storage devices usually employ a write-back caching technique. This technique involves temporarily storing the data to be written received from the host into the cache and sending a write request completion signal to the host. The operation of writing the data in the cache to the non-volatile storage media will be delayed.

[0004] In the prior art, see Figure 1 The storage device flushes data to disk based on the cache water level, which is the percentage of dirty data in the cache space. For example, when the water level is greater than or equal to a preset water level, a preset number of data are selected from the cache according to an Least Recently Used (LRU) algorithm or a sorting algorithm, and each volume in the non-volatile storage medium is flushed to disk using a round-robin method. Specifically, when a volume in the non-volatile storage medium is selected, the dirty data corresponding to that volume is flushed to that volume in the selected dirty data according to a preset concurrency threshold. In other words, the storage device uses a fair flushing method to flush each volume in the non-volatile storage medium.

[0005] Since the read or write requests for each volume are of different business types, when using the existing fair flushing method, the flushing performance of volumes with high flushing demand cannot meet the flushing demand, while the flushing performance of volumes with low flushing demand cannot be fully utilized, thus failing to make reasonable use of the flushing performance of the storage device. Summary of the Invention

[0006] This application provides a data processing method and apparatus that can flush different volumes to disk using a personalized method based on the characteristics of read and / or write requests corresponding to different volumes of the storage device, thereby improving the disk flushing performance of the storage device.

[0007] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a data processing method applied to a storage device. The storage device includes a cache and a non-volatile storage medium, the non-volatile storage medium comprising multiple volumes. The method includes: the storage device receiving multiple operation requests sent by a host, the multiple operation requests including multiple read requests and / or write requests; then, the storage device storing the multiple operation requests in a cache; subsequently, the storage device obtaining model features, the model features representing the characteristics of the multiple operation requests in the cache corresponding to a first volume of the non-volatile storage medium; and then, the storage device flushing the first volume to disk according to the model features, to write dirty data corresponding to the first volume in the cache into the first volume.

[0009] In this way, the storage device can flush different volumes according to the characteristics of the read and / or write requests corresponding to different volumes in the non-volatile storage medium. This allows for the adoption of a personalized flushing strategy that matches the business type of each volume, thus fully leveraging the flushing performance of each volume and improving the flushing characteristics of the entire storage device.

[0010] In conjunction with the first aspect, in one possible implementation, the multiple operation requests include multiple write requests, and the model features include write request randomness, which represents the degree of randomness of the pending operation position of the write request corresponding to the first volume. The storage device flushes the first volume to disk according to the model features as follows: when the write request randomness is less than a first preset value, the storage device flushes the first volume to disk according to a first flush concurrency threshold. When the write request randomness is greater than or equal to the first preset value, the storage device flushes the first volume to disk according to a second flush concurrency threshold. Wherein, the first flush concurrency threshold is greater than the second flush concurrency threshold.

[0011] In this way, the storage device can determine the business type and storage content characteristics of each volume based on the randomness of the write requests corresponding to each volume, and then use a personalized approach to flush different volumes to make reasonable use of the flushing performance of each volume, thereby improving the flushing performance of the entire storage device.

[0012] Combining the first aspect and the above possible implementations, in another possible implementation, the randomness of write requests is the ratio of the number of chunks mapped to the dirty data corresponding to multiple write requests to the number of pages.

[0013] In this way, the storage device can determine the degree of randomness of a write request based on the ratio of the number of chunks mapped to the number of pages corresponding to the dirty data of the write request.

[0014] Combining the first aspect and the aforementioned possible implementations, in another possible implementation, the multiple operation requests also include multiple read requests, and the model features also include a read request ratio, which is the ratio of the number of read requests to the number of operation requests. The storage device flushing the first volume according to the first flush concurrency threshold includes: when the read request ratio is less than a second preset value, the storage device flushes the first volume according to the first flush concurrency threshold. When the read request ratio is greater than or equal to the second preset value, and dirty data has filled a stripe, the storage device flushes the first volume according to the first flush concurrency threshold. The storage device flushing the first volume according to the second flush concurrency threshold includes: when the read request ratio is greater than or equal to the second preset value, if the chunk to be flushed corresponding to the first write request and the chunk to be read corresponding to the first read request are the same chunk, then the storage device prioritizes executing the read operation corresponding to the first read request. After the read operation is completed, the storage device flushes the first volume to disk according to the second disk flushing concurrency threshold. When the read request ratio is less than the second preset value, the storage device flushes the first volume to disk according to the second disk flushing concurrency threshold.

[0015] In this way, the storage device can also learn about the business type and storage content characteristics of each volume based on the read request ratio corresponding to each volume, and thus adopt a personalized approach to flush different volumes to make reasonable use of the flush performance of each volume, thereby improving the flush performance of the entire storage device.

[0016] In combination with the first aspect and the above possible implementations, in another possible implementation, the storage device flushes the first volume according to the model characteristics as follows: when the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data in the first volume in the cache is greater than or equal to a third preset value, the storage device flushes the first volume according to the model characteristics.

[0017] Thus, when the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the dirty data corresponding to the first volume in the cache is greater than or equal to the third preset value, it can be said that the amount of dirty data corresponding to the first volume in the cache is large and occupies a lot of cache space. Therefore, it needs to be flushed to the first volume of non-volatile storage medium to release cache space to store newly received operation requests and data to be operated from the host.

[0018] In conjunction with the first aspect and the aforementioned possible implementations, in another possible implementation, the storage device flushing the first volume based on model characteristics includes: the storage device determining the amount of data to be flushed from the first volume based on the ratio of the amount of dirty data corresponding to the first volume in the cache to a preset threshold corresponding to the dirty data in the first volume in the cache, and a third preset value. The storage device flushes the first volume based on the amount of data to be flushed and the model characteristics. If the amount of data to be flushed from the first volume is greater than or equal to the amount of data to be flushed, and the first duration of flushing the first volume is less than the preset duration, then the storage device flushes the second volume based on the model characteristics of multiple operation requests corresponding to the second volume within a second duration, where the second duration is the difference between the preset duration and the first duration.

[0019] This allows the storage device to fully utilize the time period corresponding to the preset duration to flush dirty data in at least one volume of the storage device each time it flushes, thereby improving the overall flushing performance of the storage device.

[0020] Combining the first aspect and the above possible implementations, in another possible implementation, the amount of data to be flushed to the disk corresponding to the first volume can be the difference obtained by subtracting the product of the cache space size and the third preset value from the amount of dirty data corresponding to the first volume.

[0021] In this way, for each volume in the storage device, the storage device can accurately calculate the amount of data to be flushed to disk corresponding to that volume in a similar manner to the first volume, and then perform personalized flushing of the volume based on the amount of data to be flushed to disk corresponding to that volume.

[0022] In combination with the first aspect and the above possible implementations, in another possible implementation, the operation request carries the address to be operated on. Before the storage device obtains the model features of multiple operation requests corresponding to the first volume of the non-volatile storage medium of the storage device, the method further includes: the storage device determining that the operation request is the operation request corresponding to the first volume based on the address to be operated on.

[0023] Using a method similar to that used for the first volume, the storage device can determine the operation request corresponding to each volume based on the address to be operated on.

[0024] Secondly, embodiments of this application provide a storage device, including: a cache, a non-volatile storage medium, a receiving unit, an acquiring unit, and a disk flushing unit. The non-volatile storage medium includes multiple volumes. The receiving unit is used to receive multiple operation requests sent by a host, the multiple operation requests including multiple read requests and / or write requests. The cache is used to store the multiple operation requests in the cache. The acquiring unit is used to acquire model features, the model features representing the characteristics of the multiple operation requests in the cache corresponding to a first volume of the non-volatile storage medium. The disk flushing unit is used to flush the first volume to disk according to the model features, so as to write dirty data corresponding to the first volume in the cache into the first volume.

[0025] In conjunction with the second aspect, in one possible implementation, the multiple operation requests include multiple write requests, and the model features include write request randomness, which represents the degree of randomness of the pending operation position of the write request corresponding to the first volume. The disk flushing unit is specifically used to: flush the first volume to disk according to a first disk flushing concurrency threshold when the write request randomness is less than a first preset value; and flush the first volume to disk according to a second disk flushing concurrency threshold when the write request randomness is greater than or equal to the first preset value; wherein the first disk flushing concurrency threshold is greater than the second disk flushing concurrency threshold.

[0026] Combining the second aspect and the above possible implementations, in another possible implementation, the randomness of write requests is the ratio of the number of chunks mapped to the dirty data corresponding to multiple write requests to the number of pages.

[0027] Combining the second aspect and the aforementioned possible implementations, in another possible implementation, the multiple operation requests also include multiple read requests, and the model features also include a read request ratio, which is the ratio of the number of read requests to the number of operation requests. The disk flushing unit is specifically used for: flushing the first volume to disk when the read request ratio is less than a second preset value, based on a first disk flushing concurrency threshold; and flushing the first volume to disk when the read request ratio is greater than or equal to the second preset value, and when dirty data has aggregated to fill a stripe, based on the first disk flushing concurrency threshold.

[0028] Combining the second aspect and the aforementioned possible implementations, in another possible implementation, the multiple operation requests also include multiple read requests, and the model features also include a read request ratio, which is the ratio of the number of read requests to the number of operation requests. The disk flushing unit is specifically used as follows: when the read request ratio is greater than or equal to a second preset value, if the chunk to be flushed corresponding to the first write request and the chunk to be read corresponding to the first read request are the same chunk, then the read operation corresponding to the first read request is executed first. After the read operation is completed, the first volume is flushed according to the second disk flushing concurrency threshold. When the read request ratio is less than the second preset value, the first volume is flushed according to the second disk flushing concurrency threshold.

[0029] Combining the second aspect and the above possible implementations, in another possible implementation, the disk flushing unit is specifically used to flush the first volume to disk based on the model characteristics when the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data in the first volume in the cache is greater than or equal to a third preset value.

[0030] Combining the second aspect and the aforementioned possible implementations, in another possible implementation, the disk flushing unit is specifically used to: determine the amount of data to be flushed to the first volume based on the ratio of the amount of dirty data corresponding to the first volume in the cache to a preset threshold corresponding to the dirty data in the first volume in the cache, and a third preset value. The first volume is flushed to disk based on the amount of data to be flushed and the model characteristics. If the amount of data to be flushed to the first volume is greater than or equal to the amount of data to be flushed, and the first duration of flushing the first volume is less than a preset duration, then the second volume is flushed to disk within a second duration based on the model characteristics of the multiple operation requests corresponding to the second volume. The second duration is the difference between the preset duration and the first duration.

[0031] In conjunction with the second aspect and the above possible implementations, in another possible implementation, the storage device further includes: a determining unit, configured to determine, based on the address to be operated, that the operation request is an operation request corresponding to the first volume.

[0032] Thirdly, embodiments of this application provide a storage device, including: one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the storage device performs a data processing method as described in any of the first aspects.

[0033] Fourthly, embodiments of this application provide a computer storage medium including computer instructions that, when executed on a storage device, cause the storage device to perform a data processing method as described in any of the first aspects.

[0034] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform the data processing method as described in any of the first aspects.

[0035] The beneficial effects of the second to fifth aspects mentioned above can be found in the description of the first aspect above, and will not be repeated here. Attached Figure Description

[0036] Figure 1 A schematic diagram of a disk flushing strategy provided in the prior art;

[0037] Figure 2 A schematic diagram of a storage system provided in an embodiment of this application;

[0038] Figure 3a This is a schematic diagram of the structure of a storage device provided in an embodiment of this application;

[0039] Figure 3b A schematic diagram of a non-volatile storage medium being divided into volumes, provided for an embodiment of this application;

[0040] Figure 3c This is a schematic diagram of the structure of another storage device provided in an embodiment of this application;

[0041] Figure 4 A flowchart of a method provided in an embodiment of this application;

[0042] Figure 5a This is a schematic diagram illustrating a writing method provided in an embodiment of this application;

[0043] Figure 5b This is a schematic diagram illustrating another writing method provided in an embodiment of this application;

[0044] Figure 6 Another method flowchart provided for embodiments of this application;

[0045] Figure 7 Another method flowchart provided for embodiments of this application;

[0046] Figure 8 This is a schematic diagram of a disk flushing strategy provided in an embodiment of this application;

[0047] Figure 9 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application;

[0048] Figure 10 This is a schematic diagram of another storage device provided in an embodiment of this application. Detailed Implementation

[0049] For ease of understanding, the examples provide explanations of some concepts related to the embodiments of this application for reference. As shown below:

[0050] Storage devices: Devices used to store information, usually by digitizing the information and then storing it using media such as electricity, magnetism or optics.

[0051] Cache: A buffer for data exchange.

[0052] Non-volatile storage media: storage media whose data will not be lost after power failure, including mechanical hard disks, flash drives, magnetic tapes, optical discs, etc.

[0053] Cache object: A data structure corresponding to a volume in non-volatile storage media, used to store volume-related management data, metadata, etc.

[0054] Write-back: A caching technique in which a write request is sent as soon as data is written to the cache, while the actual operation of writing the data to the non-volatile storage medium is delayed.

[0055] Destage: Writes data from the cache to a non-volatile storage medium.

[0056] Dirty data: Data in the cache that is not written to non-volatile storage media is called dirty data.

[0057] Disk flush concurrency threshold: refers to the upper limit of the number of write requests that the cache has sent to the non-volatile storage medium but has not received a response from the non-volatile storage medium during the disk flush process.

[0058] Watermark: The percentage of dirty data in the array cache.

[0059] Redundant Arrays of Independent Disks (RAID): This technology combines many inexpensive disks into a large-capacity disk group, leveraging the additive effect of individual disks to improve the overall performance of the disk system. Data is divided into many segments and stored on separate disks. RAID also utilizes the concept of peer checking, ensuring that data can still be read even if one disk in the array fails. During data reconstruction, the data is recalculated and re-placed onto new disks.

[0060] Striping: A method of dividing continuous data into blocks of equal size and writing each segment of data to different disks in a disk array. For example, RAID5 divides the data into four blocks of equal size, and the stripe width is said to be 4.

[0061] Write penalty: In RAID, write operations are accompanied by read operations. For example, if the non-volatile storage medium is configured in RAID 5, it includes one parity disk and four data disks. Assuming a write operation only modifies one data disk, in the optimal scenario, it is necessary to first read data from one data disk and one parity disk, recalculate the parity data, and then write the modified data from the data and parity disks to the non-volatile storage medium. Therefore, in this case, one write request is amplified into two read requests and two write requests, resulting in additional read / write overhead. This phenomenon is called write penalty.

[0062] Full stripe flush: Because the stripes are not aligned during write operations, the additional overhead caused by write penalties will increase and reduce processing performance. If the write operation is aligned with the stripe, the check value can be calculated directly, and then the data and check value can be written to the non-volatile storage medium, reducing the additional read and write overhead caused by write penalties. This aligned write method is called full stripe flush.

[0063] Partition: A portion of a physical disk, equivalent to a physical partition.

[0064] Volume: A partition or collection of partitions used by a file system.

[0065] Bandwidth-based services: These services require write requests to be executed in the order of storage space addresses (e.g., the order of storage space addresses in a stripe), and each write request corresponds to a large amount of data.

[0066] In existing disk flushing solutions, storage devices use a fair flushing method for volumes with different service types, which fails to make reasonable use of the flushing performance of each volume and reduces the overall flushing performance of the storage device. The method provided in this application can use a personalized flushing strategy to flush different volumes according to the service characteristics corresponding to each volume, thereby making reasonable use of the flushing performance of each volume and improving the overall flushing performance of the storage device.

[0067] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0068] Figure 2 A simplified schematic diagram of a storage system architecture applicable to embodiments of this application is shown. The storage system may include a host 10 and a storage device 20. The host 10 can be used to send operation requests to the storage device 20. These operation requests may include read requests and / or write requests; therefore, operation requests can also be referred to as I / O requests. The host 10 can be a server providing services, or a computer requesting services, etc. The storage device 20 can be used to receive operation requests sent by the host 10 to read or write data.

[0069] For details, see Figure 3aThe storage device 20 may include a controller 21, a transceiver 22, a cache 23, and a non-volatile storage medium 24. The controller 21 can manage and control the various components of the storage device 20 to perform storage-related operations. Specifically, the controller 21 can control the transceiver 21 to receive read or write requests from the host 10, or to send response data to the host 10, such as a response signal corresponding to a write request or data read from a read request; the controller 21 can control the cache 23 to temporarily store operation requests received by the transceiver 21 from the host 10; the controller 21 can also, in response to a write request from the host 10, temporarily store the data to be operated corresponding to the write request in the cache 23, and write the data to be operated to the non-volatile storage medium 24 according to the disk flushing mechanism; the controller 21 can also, in response to a read request from the host 10, read the data corresponding to the read request from the non-volatile storage medium 24. In addition, the storage device 20 may include other components, which will not be described in detail here.

[0070] Among them, see Figure 3b The non-volatile storage medium 24 can be logically divided into multiple volumes, such as volume A, volume B, volume C, and volume D. The storage device 20 can flush the data corresponding to the write request in the cache 23 to the corresponding volume according to the specific content of the write request. The storage device 20 can also read the corresponding data from the corresponding volume according to the specific content of the read request and return it to the host 10.

[0071] The non-volatile storage medium 24 in storage device 20 can be a physical storage medium with persistent storage characteristics. When the non-volatile storage medium 24 uses RAID, it can improve the read and write speed of the storage device, provide fault tolerance, and continue to work even if any disk fails, without being affected by the failed disk. For example, when the non-volatile storage medium 24 is a RAID5 disk array, the disk array can include 1 parity disk and 4 data disks; when the non-volatile storage medium 24 is a RAID6 disk array, the disk array can include 2 parity disks and 4 data disks, etc.

[0072] Figure 3c This illustration shows another schematic diagram of the composition of the storage device provided in an embodiment of this application. For example... Figure 3c As shown, the storage device may include at least one processor 31, a memory 32, a communication interface 33, and a communication bus 34. The following describes the process in conjunction with... Figure 3c A detailed introduction to each component of a storage device:

[0073] Processor 31 is the control center of the storage device and can be a single processor or a collective term for multiple processing elements. For example, processor 31 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). For example, in... Figure 3c In this context, processor 31 may include CPU1 and CPU2.

[0074] The processor 31 can perform various functions of the storage device by running or executing software programs stored in the memory 32 and calling data stored in the memory 32.

[0075] In a specific implementation, as one example, the storage device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0076] The memory 32 may include cache and non-volatile storage media. Specifically, the memory 32 may include read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory 32 may exist independently and be connected to the processor 31 via the communication bus 34. The memory 32 may also be integrated with the processor 31.

[0077] The memory 32 is used to store the operation data corresponding to the operation request and the software program for executing the solution of this application, and is controlled by the processor 31 to execute.

[0078] Communication interface 33 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Communication interface 33 may include a receiving unit to implement receiving functions and a transmitting unit to implement transmitting functions.

[0079] The communication bus 34 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3c The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0080] Figure 3c The device structure shown does not constitute a limitation on the storage device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] The following will combine Figure 2 The storage system shown and Figure 3a or Figure 3c The storage device shown, taking RAID as an example of a non-volatile storage medium, will be described in detail below to illustrate the implementation of the embodiments of this application.

[0082] See Figure 4 This application provides a data processing method, which may include:

[0083] 401. The storage device receives multiple operation requests from the host, including multiple read requests and / or write requests.

[0084] 402. The storage device stores multiple operation requests in the cache.

[0085] In steps 401 and 402, the storage device may temporarily store the operation request received from the host in a cache.

[0086] 403. Storage device acquires model features, which are used to represent the characteristics of multiple operation requests in the cache corresponding to the first volume of non-volatile storage media.

[0087] The non-volatile storage medium can include multiple volumes, each corresponding to services implemented through read and / or write operations. Therefore, the characteristics of read and / or write requests for each volume are related to the service type of that volume. For example, when the data volume of the read operation corresponding to a write request is large, and the addresses to be operated on by multiple write requests are written sequentially according to the storage address position, the volume may correspond to a bandwidth-intensive service, and this volume may be used to store video data. The first volume can be any volume included in the non-volatile storage medium.

[0088] In this embodiment, each volume in the non-volatile storage medium can correspond to a cache object in the cache. The storage device flushes dirty data in the cache to the volume of the non-volatile storage medium through the mapping between cache objects and volumes.

[0089] 404. The storage device flushes the first volume to disk based on the model characteristics, so as to write the dirty data corresponding to the first volume in the cache into the first volume.

[0090] Since the characteristics of the read and / or write requests corresponding to the first volume in the storage device are related to the business type of the first volume, the storage device can flush the first volume to disk according to the characteristics of the read and / or write requests corresponding to the first volume, thereby writing the dirty data corresponding to the first volume in the cache into the first volume.

[0091] Similar to the first volume, in this embodiment of the application, the storage device can flush different volumes to disk according to the characteristics of the read requests and / or write requests corresponding to different volumes in the non-volatile storage medium. This allows for the adoption of a personalized flushing strategy that matches the service type of each volume, thereby fully utilizing the flushing performance of each volume and improving the flushing characteristics of the entire storage device.

[0092] Specifically, in this embodiment, multiple operation requests may include multiple write requests, and model features may include the randomness of the write requests corresponding to the multiple write requests. The write request randomness can be used to represent the degree of randomness of the pending operation position of the write request corresponding to the first volume in the cache. The pending operation position of the write request refers to the location where the write request requests writing data to the non-volatile storage medium. The write request randomness can be used to represent the degree of randomness of the pending operation positions of multiple write requests corresponding to the first volume stored in the cache.

[0093] Specifically, in one writing method, see [link to documentation]. Figure 5aThe first volume's corresponding n (positive integers greater than 1, such as 2) write requests can request that data be written sequentially to the non-volatile storage medium according to the address order of the storage space corresponding to the stripes. In this case, the randomness of the write requests will be low. For example, when the first volume corresponds to a video service, multiple write requests corresponding to the first volume will typically use this writing method to request that contiguous video data be written to the first volume.

[0094] In another writing method, see Figure 5b The n (e.g., 2) write requests corresponding to the first volume can request to write data randomly to any location on the non-volatile storage medium, which will result in a high degree of randomness in the write requests. For example, when the business type corresponding to the first volume makes the first volume used to store different data information of different users, the multiple write requests corresponding to the first volume will usually adopt this writing method to request to write unrelated or poorly related data to the first volume.

[0095] As can be seen, the storage device can determine the relevant characteristics of the first volume, such as the business type and storage content, based on the randomness of write requests. Therefore, when the model characteristics of multiple operation requests corresponding to the first volume include write request randomness, the storage device can write the dirty data corresponding to the first volume in the cache into the first volume based on the write request randomness of the multiple write requests corresponding to the first volume. Here, the dirty data corresponding to the first volume in the cache refers to the data stored in the cache that is to be written to the first volume on the non-volatile storage medium.

[0096] In this way, the storage device can determine the business type and storage content characteristics of each volume based on the randomness of the write requests corresponding to each volume, and then use a personalized approach to flush different volumes to make reasonable use of the flushing performance of each volume, thereby improving the flushing performance of the entire storage device.

[0097] Specifically, in this embodiment, the operation request may carry the address to be operated on. See also Figure 6 Before step 403, the method may further include:

[0098] 405. The storage device determines that the operation request is the operation request corresponding to the first volume based on the address to be operated on.

[0099] When the operation request is a write request, the address to be operated on can be the address to be written, that is, the address in the non-volatile storage medium corresponding to the data to be written, specifically the starting address in the non-volatile storage medium to be written. Therefore, when the address to be written in the write request received by the storage device from the host is within the storage space range corresponding to the first volume, the storage device can determine that the write request is a write request corresponding to the first volume.

[0100] Specifically, since each cached object corresponds to a volume, the storage device can count write requests in units of cached objects based on the address to be written in the write requests in the cache. These write requests also correspond to the volumes corresponding to the cached objects.

[0101] In addition, when the operation request is a write request, the write request can also include the specific content of the data to be written.

[0102] Specifically, each volume of non-volatile storage media can include multiple chunks, and each chunk can include multiple pages. For example, a chunk can be 256KB (bytes), and a page can be 4KB.

[0103] In one possible implementation, the write request randomness is the ratio of the number of chunks mapped to the dirty data corresponding to multiple write requests for the first volume to the number of pages. Here, the number of chunks mapped to the dirty data corresponding to the multiple write requests for the first volume refers to the number of chunks containing data to be written to the first volume for those multiple write requests; the number of pages mapped to the dirty data corresponding to the multiple write requests for the first volume refers to the number of pages containing data to be written to the first volume for those multiple write requests. When a write request includes both the address to be written and the specific content of the data to be written, the storage device can determine the number of pages and chunks mapped to the dirty data corresponding to the write request based on the size of the address to be written and the specific content of the data to be written, thereby determining the ratio of the number of chunks to the number of pages, and thus the write request randomness.

[0104] It should be noted that since a chunk consists of multiple pages, the ratio of the number of chunks mapped to the dirty data corresponding to multiple write requests in the first volume to the number of pages is less than or equal to 1.

[0105] When the model features include write request randomness, step 404 above may specifically include:

[0106] 4041. When the randomness of a write request is less than a first preset value, the storage device flushes the first volume according to the first flush concurrency threshold.

[0107] The first preset value can be less than 1. The specific size of the first preset value can be set according to actual needs and will not be limited here. For example, the first preset value can be 1 / 4. The first disk flushing concurrency threshold can be set according to actual needs, for example, it can be 200 (write requests).

[0108] When the randomness of write requests is less than a first preset value, the ratio of the number of chunks mapped to the number of pages corresponding to the dirty data of multiple write requests for the first volume is small. The pages mapped to the dirty data of multiple write requests for the first volume are distributed across a smaller number of chunks. The probability that the dirty data of multiple write requests for the first volume is written to the first volume in striped order is high, and the probability that the dirty data of multiple write requests for the first volume is written randomly to any position in the first volume is low. Therefore, the randomness of write requests is low. At this time, the storage device can flush the first volume according to the first flush concurrency threshold, thereby writing a portion of the dirty data corresponding to the first volume in the cache into the first volume.

[0109] 4042. When the randomness of a write request is greater than or equal to the first preset value, the storage device flushes the first volume according to the second flush concurrency threshold.

[0110] When the randomness of write requests is greater than or equal to a first preset value, the ratio of the number of chunks mapped to the number of pages corresponding to the dirty data of multiple write requests for the first volume is relatively large. The pages mapped to the dirty data of multiple write requests for the first volume are distributed across a large number of chunks. Therefore, the probability that the dirty data of multiple write requests for the first volume is randomly written to any location in the first volume is relatively high, resulting in a high randomness of write requests. At this time, the storage device can flush the first volume according to the second flush concurrency threshold, thereby writing a portion of the dirty data corresponding to the first volume in the cache into the first volume.

[0111] The second disk flushing concurrency threshold can be set according to actual needs, for example, it can be 100. When the second disk flushing concurrency threshold is 100, if the number of write requests that the cache has sent to the non-volatile storage medium but has not received a response from the non-volatile storage medium is greater than or equal to the second disk flushing concurrency threshold of 100, the cache cannot continue to send write requests to the first volume of the non-volatile storage medium; when the number of write requests that the cache has sent to the non-volatile storage medium but has not received a response from the non-volatile storage medium is less than the second disk flushing concurrency threshold of 100, the cache can continue to send write requests to the first volume.

[0112] The first disk flushing concurrency threshold is greater than the second disk flushing concurrency threshold. In other words, when the randomness of write requests is low, the storage device can use a larger disk flushing concurrency threshold compared to when the randomness of write requests is high.

[0113] Specifically, when the randomness of write requests is low, the probability that dirty data corresponding to multiple write requests for the first volume will be randomly written to the first volume is low, and the probability that it will be written sequentially to the first volume is high. The probability that the dirty data corresponding to multiple write requests for the first volume is related content (such as video data) is high. The probability that the business of multiple write requests for the first volume is a bandwidth-intensive business is also high. At this time, the amount of data to be written to the first volume may be large and the duration may be long, and the write task pressure on the storage device may be high. Therefore, a larger flushing concurrency threshold can be used to flush the disk, so as to flush the dirty data corresponding to the first volume to the non-volatile storage medium as soon as possible.

[0114] When the randomness of write requests is high, it is more likely that the dirty data corresponding to multiple write requests for the first volume will be randomly written to the first volume, and less likely that it will be written sequentially to the first volume. It is less likely that the dirty data corresponding to multiple write requests for the first volume is related content (such as video data), and less likely that the business of multiple write requests for the first volume is a bandwidth-intensive business. In this case, the amount of data to be written to the first volume and the duration may not be very long, and the write task pressure on the storage device is not particularly high. Therefore, a smaller flushing concurrency threshold can be used for flushing.

[0115] As can be seen from steps 4041 and 4042, the storage device can determine the disk flushing concurrency threshold for different volumes based on the randomness of the write requests corresponding to multiple write requests for different volumes, and thus adopt different disk flushing strategies to flush different volumes according to different disk flushing concurrency thresholds.

[0116] Furthermore, the multiple operation requests in steps 401 and 403 may also include multiple read requests. The model features of the multiple operation requests corresponding to the first volume may also include a read request ratio, which is the ratio of the number of read requests corresponding to the first volume in the cache to the number of operation requests corresponding to the first volume in the cache. The number of operation requests corresponding to the first volume in the cache includes the sum of the number of write requests and the number of read requests corresponding to the first volume in the cache. That is:

[0117] Read request ratio = (Number of read requests / (Number of read requests + Number of write requests)) × 100%

[0118] When the multiple operation requests in steps 401 and 403 may also include at least one read request, the read request may also carry an address to be operated on. This address is the address to be read, i.e., the address of the data to be read from the first volume of the non-volatile storage medium corresponding to the read request. The storage address can be used to determine whether the read request is an operation request corresponding to the first volume based on the address to be read in the read request in the cache. In addition, the read request may also carry the size of the data to be read.

[0119] See Figure 7 When the model features also include the read request ratio, step 4041 above may specifically include:

[0120] 501. When the read request ratio is less than the second preset value, the storage device flushes the first volume according to the first flush concurrency threshold.

[0121] The second preset value is relatively small and can be set according to actual needs; this application embodiment does not impose specific limitations. For example, the second preset value can be 10%.

[0122] In step 501, when the read request ratio is less than the second preset value, it indicates that the number of read requests corresponding to the first volume is very small, while the number of write requests corresponding to the first volume is large. Furthermore, in the case of step 4041, when the randomness of the write requests is small, the probability that the dirty data corresponding to the multiple write requests corresponding to the first volume is small, and the probability that it is written to the first volume sequentially is large. The probability that the dirty data corresponding to the multiple write requests corresponding to the first volume is related content (such as video data) is large, and the probability that the service of the multiple write requests corresponding to the first volume is a bandwidth-intensive service is also large. At this time, the amount of data to be written to the first volume may be large and the duration may be long, and the write task pressure on the storage device is very high. Therefore, a larger flushing concurrency threshold can be used to flush the first volume to the non-volatile storage medium as soon as possible.

[0123] The disk flushing strategy described in step 501 can be called the disk flushing strategy as soon as possible.

[0124] 502. When the read request ratio is greater than or equal to the second preset value, the storage device flushes the first volume according to the first flush concurrency threshold when dirty data aggregates to fill a stripe.

[0125] Among them, when the non-volatile storage medium adopts such Figure 5a or Figure 5b In RAID 5 configurations, write penalties are common. During write operations, misalignment of stripes increases the overhead caused by write penalties. A description of write penalties can be found at the beginning of the example. For example, see... Figure 5aWhen write request 1 for the first volume requests to write data to data disk 1 in stripe 1, and write request 2 requests to write data to data disk 1 in stripe 2, the storage device reads data disks 1-4 and the parity disk in stripe 1 and recalculates the parity data. Then, it writes the data corresponding to write request 1 to data disk 1 in stripe 1 and writes the recalculated parity data to the parity disk. Similarly, the storage device reads data disks 1-4 and the parity disk in stripe 2 and recalculates the parity data. Then, it writes the data corresponding to write request 2 to data disk 1 in stripe 2 and writes the recalculated parity data to the parity disk. In one scenario, if the cache already contains data disks 2-4 of stripe 1, the storage device reads data disk 1 and the parity disk from stripe 1, recalculates the parity data, then writes the data corresponding to write request 1 to data disk 1 in stripe 1, and writes the recalculated parity data to the parity disk. Conversely, if the cache already contains data disks 2-4 of stripe 2, the storage device reads data disk 1 and the parity disk from stripe 2, recalculates the parity data, then writes the data corresponding to write request 2 to data disk 1 in stripe 2, and writes the recalculated parity data to the parity disk. In this scenario, the additional read and write overhead from the write penalty is significant, increasing the pressure on the non-volatile storage medium and reducing its lifespan.

[0126] See Figure 5b If both write requests 1 and 2 for Volume 1 request data to be written to data disk 1 in stripe 1, the storage device reads data disks 1-4 and the parity disk in stripe 1, recalculates the parity data, and then writes the data corresponding to write requests 1 and 2 to data disk 1 in stripe 1, and writes the recalculated parity data to the parity disk. In another scenario, if data disks 2-4 of stripe 1 are already stored in the cache, the storage device reads data disk 1 and the parity disk in stripe 1, recalculates the parity data, and then writes the data corresponding to write requests 1 and 2 to data disk 1 in stripe 1, and writes the recalculated parity data to the parity disk. In this scenario, the additional read and write overhead caused by write penalties is relatively small.

[0127] It is evident that when the dirty data corresponding to the two write requests for the first volume is within the same stripe, the additional read / write overhead caused by write penalties can be reduced, decreasing the number of read / write operations on the storage device and thus improving its processing performance. It is easy to understand that when the dirty data corresponding to multiple write requests for the first volume is within as few stripes as possible, the additional read / write overhead caused by write penalties can be reduced, thereby improving the storage device's performance.

[0128] Therefore, in step 502, when the read request ratio is greater than or equal to the second preset value, it indicates that the number of read requests corresponding to the first volume is relatively large, the number of write requests corresponding to the first volume is relatively large but less than the number of write requests in step 501, and the write task pressure corresponding to the first volume is relatively large but less than the write task pressure in step 501. Furthermore, since the write request randomness is relatively small in the case of step 4041, the probability that the dirty data corresponding to multiple write requests corresponding to the first volume is written to the first volume in stripe order is relatively high. Therefore, the storage device can wait until the dirty data corresponding to multiple write requests corresponding to the first volume is aggregated to fill a stripe, and then flush the first volume to disk according to the first flush concurrency threshold with a larger value. In this way, the number of stripes when flushing the first volume can be reduced by flushing the full stripe, reducing the additional overhead caused by write penalties, improving the processing performance of the storage device, and quickly flushing the dirty data corresponding to the first volume to the non-volatile storage medium of the first volume.

[0129] The disk brushing strategy described in step 502 can be called the full strip disk brushing strategy.

[0130] See Figure 7 When the model features also include the read request ratio, step 4042 above may specifically include:

[0131] 503. When the read request ratio is greater than or equal to the second preset value, if the chunk to be flushed to disk corresponding to the first write request and the chunk to be read corresponding to the first read request are the same chunk, the storage device shall prioritize the read operation corresponding to the first read request.

[0132] 504. After the read operation is completed, the storage device flushes the first volume according to the second flush concurrency threshold.

[0133] Since the write request randomness is relatively high in step 4042, the probability that dirty data corresponding to multiple write requests for the first volume is randomly written to the first volume is relatively high, and the probability that it is written sequentially to the first volume is relatively low. The probability that the dirty data corresponding to multiple write requests for the first volume is related content (such as video data) is relatively low, and the probability that the service of multiple write requests for the first volume is a bandwidth-intensive service is also relatively low. At this time, the amount of data to be written to the first volume and the duration may not be very long, and the write task pressure of the storage device is not particularly high. Therefore, in steps 503-504, the storage device can use a smaller disk flushing concurrency threshold for disk flushing.

[0134] Furthermore, in step 503, when the proportion of read requests is greater than or equal to the second preset value, it indicates that the number of read requests corresponding to the first volume is relatively large. Read requests are usually operations directly related to the host's foreground operations. For example, when the host foreground needs to read user information or play videos, it needs to perform read operations through read requests. Therefore, the latency of performing read operations directly affects the user experience of the host.

[0135] Therefore, when a write request corresponds to the same chunk to be flushed (i.e., the chunk containing data to be written) and a read request corresponds to the same chunk to be read (i.e., the chunk containing data to be read), a conflict occurs between the read and write requests. The storage device can prioritize executing the read operation corresponding to the write request, thus responding to the read request promptly, reducing read operation latency, ensuring the real-time performance of host services, and improving the user experience. After the read operation is completed, the storage device can flush the first volume to disk based on the smaller second flush concurrency threshold.

[0136] The disk flushing strategy described in steps 503 and 504 can be called the read request priority strategy.

[0137] 505. When the read request ratio is less than the second preset value, the storage device flushes the first volume according to the second flush concurrency threshold.

[0138] In step 505, when the read request ratio is less than the second preset value, it indicates that the number of read requests corresponding to the first volume is very small, and the impact on the latency of the host front end is small. Also, since the write request randomness is relatively high in the case of step 4042, the probability that the dirty data corresponding to the multiple write requests corresponding to the first volume is randomly written to the first volume is relatively high, and the probability that it is written sequentially to the first volume is relatively low. The probability that the dirty data corresponding to the multiple write requests corresponding to the first volume is related content (such as video data) is relatively low, and the probability that the service of the multiple write requests corresponding to the first volume is a bandwidth-intensive service is also relatively low. At this time, the amount of data to be written to the first volume may not be particularly large, the duration may not be particularly long, and the write task pressure of the storage device is not particularly high. Therefore, a smaller second flush concurrency threshold can be used for flushing.

[0139] The disk flushing strategy described in step 505 can be called the slow disk flushing strategy.

[0140] In addition, the disk flushing strategies described in steps 501-505 above can also be found in Table 1 below:

[0141] Table 1

[0142] Write request randomness Read request ratio Disk flushing strategy Less than the first preset value Greater than or equal to the second preset value Full strip brushing strategy Less than the first preset value Less than the second preset value Quick disk refresh strategy Greater than or equal to the first preset value Greater than or equal to the second preset value Read request priority strategy Greater than or equal to the first preset value Less than the second preset value Slow flushing strategy

[0143] As can be seen from steps 501-505 above, the storage device can also determine the business type and storage content characteristics of each volume based on the read request ratio corresponding to each volume, and thus adopt a personalized approach to flush different volumes to make reasonable use of the flushing performance of each volume, thereby improving the flushing performance of the entire storage device.

[0144] In this embodiment of the application, step 404 may specifically include:

[0145] 4040. When the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the dirty data corresponding to the first volume in the cache is greater than or equal to the third preset value, the storage device flushes the first volume to disk according to the model characteristics.

[0146] The preset threshold for dirty data corresponding to the first volume in the cache refers to the quota of dirty data corresponding to the first volume in the cache, that is, the maximum capacity of dirty data allowed to be stored in the cache for the first volume. For example, the preset threshold can be a capacity value, such as 1GB (bytes); the preset threshold can also be a ratio relative to the storage space size of the cache, such as 15%. The size of the third preset value can be set according to actual needs, and this embodiment does not specifically limit it. For example, the third preset value can be 20%.

[0147] See Figure 8 When the amount of dirty data corresponding to the first volume in the cache is equal to the preset threshold of the amount of dirty data corresponding to the first volume in the cache (i.e., Figure 8 When the actual ratio in the cache is greater than or equal to the third preset value, it indicates that the amount of dirty data corresponding to the first volume in the cache is large, occupying a lot of cache space. Therefore, it needs to be flushed to the first volume of non-volatile storage media to free up cache space to store newly received operation requests and pending data from the host. For example, when the amount of dirty data corresponding to the first volume in the cache is 300MB, the preset threshold for dirty data corresponding to the first volume in the cache is 1GB, and the third preset value is 20%, the ratio of the amount of dirty data corresponding to the first volume to the preset threshold for dirty data corresponding to the first volume in the cache is greater than the third preset value. The storage device can flush the first volume to disk according to the model characteristics.

[0148] It should be noted that the preset thresholds for dirty data corresponding to different volumes in the storage device in the cache may be different or the same, and this application embodiment does not make specific limitations.

[0149] In step 4040 above, when the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data of the first volume in the cache is greater than or equal to a third preset value, the storage device flushes the first volume to disk according to the model characteristics. Specifically, this may include:

[0150] 601. The storage device determines the amount of data to be flushed to disk corresponding to the first volume based on the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the dirty data corresponding to the first volume in the cache and a third preset value.

[0151] 602. The storage device flushes the first volume based on the amount of data to be flushed and the model characteristics.

[0152] 603. If the amount of data to be flushed to the first volume by the storage device is greater than or equal to the amount of data to be flushed, and the first flushing time of the first volume is less than the preset time, then the second volume is flushed to the second volume within the second time according to the model characteristics of the multiple operation requests corresponding to the second volume. The second time is the difference between the preset time and the first time.

[0153] 604. If the amount of data to be flushed to the first volume by the storage device is less than the amount of data to be flushed, and the first flushing time of the first volume is greater than or equal to the preset time, then the flushing of the first volume shall be stopped.

[0154] In this embodiment, the storage device can use a polling method to flush each volume in the non-volatile storage medium to disk. When the first volume is polled, if the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data in the first volume in the cache is greater than or equal to a third preset value, the storage device can select the dirty data corresponding to the first volume from the cache for flushing to disk according to the least recently used algorithm or sorting algorithm, and the flushing time can be a preset time, for example, the preset time can be 5ms (milliseconds).

[0155] In the specific implementation process, the storage device can, in steps 601-602, determine the amount of data to be flushed to disk corresponding to the first volume based on the ratio of the amount of dirty data corresponding to the first volume in the cache to a preset threshold corresponding to the dirty data in the first volume in the cache, and a third preset value. Then, the first volume is flushed to disk based on the amount of data to be flushed and the model characteristics corresponding to the first volume. In one possible implementation, the amount of data to be flushed to disk corresponding to the first volume can be the difference obtained by subtracting the product of the cache space size and the third preset value from the amount of dirty data corresponding to the first volume.

[0156] For each volume in the storage device, the storage device can accurately calculate the amount of data to be flushed to disk for that volume, and then perform personalized flushing of the volume based on the amount of data to be flushed to disk for that volume.

[0157] Specifically, when the first volume is polled for a disk flush operation, if at the end of the first time period, the amount of data flushed by the storage device based on the model features corresponding to the first volume is less than the amount of data to be flushed for the first volume, the storage device can stop flushing the first volume in step 604. Then, the storage device can flush the third volume, which may be the same as or different from the first volume, according to the polling mechanism.

[0158] When the first volume is polled for a flush operation, if the amount of data to be flushed from the first volume is greater than or equal to the amount of data to be flushed (i.e., the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the amount of dirty data corresponding to the first volume in the cache has decreased to less than a third preset value, and the first flush duration for the first volume is less than the preset duration), then the storage device can flush the second volume within the second duration remaining after subtracting the first duration from the preset duration, based on the model characteristics of the multiple operation requests corresponding to the second volume. This allows for flushing as much dirty data from the cache as possible to the non-volatile storage medium within the preset duration. Specifically, the ratio of the amount of dirty data corresponding to the second volume in the cache to the preset threshold of the amount of dirty data corresponding to the first volume in the cache must be greater than or equal to the third preset value. This ensures that the storage device fully utilizes the time period corresponding to the preset duration to flush dirty data from at least one volume in the storage device each time a flush is performed, thereby improving the overall flush performance of the storage device.

[0159] It is worth noting that in steps 4040 and 602, the specific implementation of the storage device flushing the first volume to disk based on the model characteristics can be achieved through steps 4041-4042 and steps 501-505. The specific implementation of the storage device flushing the second volume to disk based on the model characteristics of the multiple operation requests corresponding to the second volume in step 603 is similar to the specific implementation of the storage device flushing the first volume to disk based on the model characteristics, and will not be described again here.

[0160] Furthermore, the method provided in the embodiments of this application may also include:

[0161] 406. When the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the dirty data corresponding to the first volume in the cache is greater than or equal to the fourth preset value, the storage device flushes the first volume according to the third flush.

[0162] Specifically, the fourth preset value is greater than the third preset value, the third disk flushing concurrency threshold is greater than the second disk flushing concurrency threshold, and the third disk flushing concurrency threshold is greater than or equal to the first disk flushing concurrency threshold. The fourth preset value can be relatively large and greater than the third preset value, and can be set according to actual needs. For example, when the third preset value is 20%, the fourth preset value can be 80%.

[0163] When the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold of the dirty data corresponding to the first volume in the cache is greater than or equal to the fourth preset value, it indicates that the dirty data corresponding to the first volume occupies a large amount of cache space. Therefore, a larger third flush concurrency threshold is needed to flush the dirty data corresponding to the first volume to non-volatile storage media as quickly as possible.

[0164] Furthermore, the above embodiments of this application are mainly described using the first volume as an example. The processing method for the number of other volumes in the non-volatile storage medium is similar to that for the first volume, and will not be repeated here.

[0165] Furthermore, it should be noted that the specific values ​​of the first preset value, the second preset value, the third preset value, and the fourth preset value corresponding to different volumes in the non-volatile storage medium can be the same or different, and the embodiments of this application do not specifically limit this.

[0166] In addition, the storage system applicable to the embodiments of this application can be an enterprise-level storage system with high performance, high reliability and easy expansion.

[0167] The foregoing primarily describes the solutions provided in this application from the perspective of storage devices. It is understood that, in order to achieve the aforementioned functions, the storage device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] This application embodiment can divide the storage device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0169] When dividing each function into modules according to its corresponding function. Figure 9 A schematic diagram of a possible configuration of the storage device 70 involved in the above embodiments is shown, such as... Figure 9As shown, the storage device 70 may include: a cache 71, a non-volatile storage medium 72, a receiving unit 73, an acquisition unit 74, a disk flushing unit 75, and a determination unit 76. The non-volatile storage medium includes multiple volumes.

[0170] The receiving unit 73 can be used to receive multiple operation requests sent by the host, including multiple read requests and / or write requests. The cache 71 can be used to store the multiple operation requests in the cache. The obtaining unit 74 can be used to obtain model features, which represent the characteristics of the multiple operation requests in the cache corresponding to the first volume of the non-volatile storage medium 72. The flushing unit 75 can be used to flush the first volume to disk according to the model features, so as to write the dirty data corresponding to the first volume in the cache into the first volume. The determining unit can be used to determine whether the operation request corresponds to the first volume based on the address to be operated on.

[0171] In addition, the disk brushing unit 75 can also be used to support the storage device 70 in performing steps 4041-4042, or steps 501-505, 406, 4040, and steps 701-704 in the above method embodiments, and / or other processes used in the technology described herein.

[0172] The storage device provided in this application embodiment is used to execute the above-described data processing method, and therefore can achieve the same effect as the above-described data processing method.

[0173] When using integrated units, Figure 10 A schematic diagram illustrating another possible configuration of the storage device involved in the above embodiments is shown. For example... Figure 10 As shown, the storage device 80 may include a processing module 81, a communication module 82, and a storage module 83.

[0174] The processing module 81 is used to control and manage the operations of the storage device 80. For example, the processing module 81 is used to support the storage device 40 in performing operations. Figure 9 The operations of the acquisition unit 74, the disk brushing unit 75, and the determination unit 76 shown herein, and / or other processes used in the technology described herein. The communication module 82 is used to support the storage device 80 in performing... Figure 9 The function of the receiving unit 73 shown is illustrated. The storage module 43 can be used to perform... Figure 9 The operation of cache 71 and non-volatile memory 72 is shown, as well as the program code and data stored in storage device 80.

[0175] The processing module 81 may be a processor or a controller. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP), and other microprocessors. The communication module 82 may be a transceiver, transceiver circuitry, or a communication interface, etc.

[0176] When the processing module 81 is a processor, the communication module 82 is a communication interface, and the storage module 83 is a memory, the structural diagram of the storage device can be shown as follows: Figure 3c As shown.

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0179] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method applied to a storage device, said storage device comprising a cache and multiple volumes, characterized in that, The method includes: Receive write requests sent by the host; The write request is stored in the cache; Obtain model features, which are used to represent the characteristics of the write request corresponding to the first volume; Based on the model characteristics, dirty data corresponding to the first volume in the cache is written into the first volume; each of the plurality of volumes is flushed using a personalized flushing strategy that matches the volume's service type; the service type of the volume includes video services.

2. The method according to claim 1, characterized in that, The model features include write request randomness, which represents the degree of randomness of the pending operation position of the write request corresponding to the first volume. Writing dirty data corresponding to the first volume in the cache into the first volume according to the model features includes: When the randomness of the write request is less than the first preset value, the first volume is flushed to disk according to the first disk flushing concurrency threshold. When the randomness of the write request is greater than or equal to the first preset value, the first volume is flushed according to the second flushing concurrency threshold. Wherein, the first disk flushing concurrency threshold is greater than the second disk flushing concurrency threshold.

3. The method according to claim 2, characterized in that, The randomness of the write request is the ratio of the number of chunks mapped to the dirty data corresponding to the write request to the number of pages.

4. The method according to claim 1, characterized in that, The process of burning the first volume to disk based on the model features includes: When the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data of the first volume in the cache is greater than or equal to a third preset value, the first volume is flushed to disk according to the model characteristics.

5. The method according to claim 1, characterized in that, The write request carries the address to be operated on. Before obtaining the model features, the method further includes: The write request is determined to be the write request corresponding to the first volume based on the address to be operated on.

6. A storage device, characterized in that, include: Cache, receive unit, acquire unit, flush unit, and multiple volumes; The receiving unit is used to receive write requests sent by the host; The cache is used to store the write request in the cache; The acquisition unit is used to acquire model features, which are used to represent the features of the write request corresponding to the first volume. The disk flushing unit is used to write the dirty data corresponding to the first volume in the cache into the first volume according to the model characteristics; Each of the multiple volumes is flushed using a personalized flushing strategy that matches the volume's service type; the service type of the volume includes video services.

7. The storage device according to claim 6, characterized in that, The model features include write request randomness, which represents the degree of randomness of the pending operation position of the write request corresponding to the first volume. The disk flushing unit is specifically used for: When the randomness of the write request is less than the first preset value, the first volume is flushed to disk according to the first disk flushing concurrency threshold. When the randomness of the write request is greater than or equal to the first preset value, the first volume is flushed according to the second flushing concurrency threshold. Wherein, the first disk flushing concurrency threshold is greater than the second disk flushing concurrency threshold.

8. The storage device according to claim 7, characterized in that, The randomness of the write request is the ratio of the number of chunks mapped to the dirty data corresponding to the write request to the number of pages.

9. The storage device according to claim 6, characterized in that, The disk brushing unit is specifically used for: When the ratio of the amount of dirty data corresponding to the first volume in the cache to the preset threshold corresponding to the dirty data of the first volume in the cache is greater than or equal to a third preset value, the first volume is flushed to disk according to the model characteristics.

10. The storage device according to claim 6, characterized in that, The storage device also includes: The determining unit is used to determine, based on the address to be operated, that the write request is the write request corresponding to the first volume.

11. A storage device, characterized in that, include: One or more processors and one or more memories; The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, wherein when the one or more processors execute the computer instructions, the storage device performs the data processing method as described in any one of claims 1-5.

Images