Data prefetching method, control device, storage device and system

Abstract

The invention provides a data prefetching method, a control device, storage equipment and a system, and relates to the technical field of storage. The data prefetching method comprises the following steps: acquiring a first IO request; determining a target prediction model from a plurality of prediction models according to the first feature of the first IO request; inputting the first IO request into a target prediction model to obtain a target prediction address output by the target prediction model; and writing the target prefetched data in the storage medium into a cache. Wherein the first feature comprises a first address of data to be accessed by the first IO request, the prediction model is respectively used for predicting addresses of prefetched data corresponding to IO requests of different addresses, and the data to be accessed comprises target prefetched data stored in a target prediction address. In this way, the prediction accuracy of the prefetched data in a multi-process concurrent random IO request scene is improved.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data prefetching method, control device, storage device and system. Background Technology

[0002] Cache prefetching is a technique that uses overlapping computation and memory access to issue prefetch requests before the cache may expire, so that data blocks are fetched into the cache in advance before the data is actually used.

[0003] Conventional cache prefetching techniques typically focus on identifying and predicting input / output (IO) requests for a single process. However, in the case of multiple random IO requests from multiple processes, conventional cache prefetching techniques suffer from low prediction accuracy. Summary of the Invention

[0004] This application provides a data prefetching method, control device, storage device, and system, which solves the problem of low prediction accuracy of conventional cache prefetching technology in the case of multiple random I / O requests in multiple processes.

[0005] The technical solution adopted in this application is as follows.

[0006] In a first aspect, this application provides a data prefetching method applied to a storage device, the storage device including a processor, a cache, and a storage medium, the storage medium being used to persistently store data to be accessed, and the method being executed by the processor. The data access method includes: acquiring a first I / O request; determining a target prediction model from multiple prediction models based on a first feature of the first I / O request; inputting the first I / O request into the target prediction model to obtain a target prediction address output by the target prediction model; and writing target prefetched data from the storage medium into the cache. The first feature includes a first address of the data to be accessed by the first I / O request, the prediction models are used to predict the addresses of prefetched data corresponding to I / O requests at different addresses, and the data to be accessed includes target prefetched data stored at the target prediction address.

[0007] The address can refer to an address range, that is, the starting and ending positions of the storage location in the storage medium.

[0008] Based on the aforementioned data prefetching method, multiple prediction models are used to predict the prefetched data addresses corresponding to I / O requests at different addresses. Each prediction model makes a prediction for its corresponding address. Since the addresses of data accessed by multiple I / O requests from a single process are usually quite close, having one prediction model predict I / O requests for a corresponding range of addresses is beneficial for predicting the next I / O request of a process when the randomness of concurrent I / O requests from multiple processes is high. Furthermore, because the address range corresponding to each model is relatively small, having each prediction model predict for its corresponding range allows the prediction model to accurately predict the addresses of prefetched data even when using interval regression. Thus, the prediction accuracy of prefetched data in scenarios with concurrent random I / O requests from multiple processes is improved.

[0009] In conjunction with the method provided in the first aspect, in one optional implementation, determining the target prediction model from multiple prediction models based on a first feature of the first IO request may include determining the target prediction model based on a first address and the prediction addresses of the multiple prediction models. Here, the prediction address of the target prediction model is the prediction address obtained by the target prediction model based on a second IO request, where the second IO request precedes the first IO request.

[0010] Optionally, the target prediction model can be a prediction model whose prediction address includes the first address among multiple prediction models. For example, the offset of the first address is greater than or equal to the offset of the target prediction model, and the sum of the offset of the first address and its length is less than the sum of the offset of the target prediction model and its length.

[0011] Optionally, the target prediction model can be the prediction model with the highest address overlap between the predicted address and the first address among multiple prediction models. For example, the offset of the first address is greater than or equal to the offset of the target prediction model and less than the sum of the offset and length of the target prediction model; the sum of the offset and length of the first address is greater than the sum of the offset and length of the target prediction model. Alternatively, the offset of the first address is less than the offset of the target prediction model; the sum of the offset and length of the first address is greater than the offset of the target prediction model and less than the sum of the offset and length of the target prediction model.

[0012] Optionally, the target prediction model can be the prediction model whose predicted address is closest to the first address among multiple prediction models. For example, the first address has zero address overlap with the predicted address of any of the multiple prediction models, and the absolute value of the difference between the offset of the target prediction model and the offset of the first address is the smallest.

[0013] Based on the above implementation, different prediction models are divided according to the address range, so that each prediction model is used to predict the address of the prefetched data corresponding to the IO request within a certain address range. This is beneficial to accurately match the concurrent random IO requests of multiple processes to the corresponding prediction model for prediction.

[0014] In conjunction with the method provided in the first aspect, in one optional implementation, the data prefetching method further includes: obtaining the features of multiple historical IO requests; clustering the features of the multiple historical IO requests to obtain multiple categories, wherein the features of the historical IO requests include the address of the data accessed by the historical IO requests; and establishing a prediction model based on each of the multiple categories to obtain multiple prediction models.

[0015] Based on the above implementation, a prediction model is constructed based on a classification obtained after clustering. The clustering is based on the characteristics of historical IO requests, such as the address of the data accessed by the historical IO requests, which ensures the independence and accuracy of the prediction model in predicting the address of the prefetched data for the corresponding IO request.

[0016] In conjunction with the method provided in the first aspect, in one optional implementation, after establishing multiple prediction models based on classification, streaming clustering is performed on multiple historical IO requests based on the multiple prediction models. Optionally, for the first historical IO request among the multiple historical IO requests, the first historical IO request is input into the multiple prediction models respectively to obtain multiple predicted addresses output by the multiple prediction models; the first prediction model corresponding to the prediction address with the highest address overlap among the multiple predicted addresses is determined, where the address overlap is the overlap between the predicted address obtained by the prediction model based on the first historical IO request and the address of the data to be accessed by the second historical IO request, and the second historical IO request is the next request after the first historical IO request; the first historical IO request is classified into the category corresponding to the first prediction model.

[0017] Based on the above implementation method, the predicted addresses of the prediction model are used to classify historical IO requests, which improves the clustering accuracy of historical IO requests, thereby improving the quality of the training data of the prediction model and thus improving the prediction accuracy of the prediction model.

[0018] In conjunction with the method provided in the first aspect, in one optional implementation, the characteristics of multiple historical IO requests include the process identifier of the process to which the historical IO requests belong. Establishing a prediction model based on each of the multiple categories may include: if the addresses of the data accessed by the historical IO requests in the first category conform to a linear relationship, establishing a ridge regression model corresponding to the first category based on the historical IO requests in the first category; if the addresses of the data accessed by the historical IO requests in the second category do not conform to a linear relationship, establishing an interval regression model corresponding to the second category based on the historical IO requests in the second category.

[0019] Based on the above implementation, when historical IO requests include the process identifier of the process to which they belong, a category can be equivalent to a sequence of IO requests from an actual process. In this case, for IO requests of different categories, depending on whether the category conforms to a linear relationship, either a ridge regression model or an interval regression model can be used to predict the address of the data to be prefetched. This ensures the accuracy of the prediction results while guaranteeing that the address accessed by the IO request conforms to the linear relationship of the actual process, and also ensures the accuracy of the prediction results even if the address accessed by the IO request does not conform to a linear relationship, thereby improving the applicability of the data prefetching method.

[0020] In conjunction with the method provided in the first aspect, in one optional implementation, the characteristics of multiple historical I / O requests do not include the process identifier of the process to which the historical I / O requests belong. Multiple prediction models are obtained by establishing a prediction model for each of the multiple categories. This can include: establishing an interval prediction model for each of the multiple categories; and establishing a linear model based on the predicted addresses of the interval prediction models corresponding to the multiple categories. The linear model is used to predict the predicted address of the interval prediction model corresponding to the next category based on the multiple categories.

[0021] Based on the above implementation, when historical I / O requests do not include the process identifier of the process to which they belong, a category can be equivalent to a combination of I / O requests from multiple actual processes with similar addresses. In this case, for I / O requests of different categories, an interval regression model is used to predict the address of the prefetched data based on the address range corresponding to each category. Simultaneously, a linear model is used to predict the address of the interval prediction model corresponding to the next category. This allows the interval prediction model to be used for prediction when the address of the I / O request corresponds to the predicted address of any interval prediction model; otherwise, the linear model is used to generate the predicted address of the interval prediction model corresponding to the next category, and then the interval prediction model is used for prediction.

[0022] In conjunction with the method provided in the first aspect, in one optional implementation, writing the target prefetch data from the storage medium to the cache may include: generating a first prefetch I / O request based on the target predicted address; reading the target prefetch data from the storage medium based on the first prefetch I / O request; and writing the target prefetch data to the cache. The first prefetch I / O request is used to instruct the reading of the target prefetch data stored at the target predicted address in the storage medium.

[0023] In conjunction with the method provided in the first aspect, in an optional implementation, the data prefetching method process may further include: acquiring multiple prefetch IO requests; merging the multiple prefetch IO requests according to their characteristics to obtain at least one aggregated prefetch IO request; the characteristics of the multiple prefetch IO requests include at least one of filename and address, and the constraint condition for merging n prefetch IO requests into one aggregated prefetch IO request is: the latency gain of the aggregated prefetch IO request relative to the n prefetch IO requests is greater than the sum of the latency and space overhead of the additional prefetching of the aggregated prefetch IO request, where additional prefetching refers to the portion of data that the aggregated prefetch IO request will access that does not overlap with the data that the multiple prefetch IO requests will access; writing at least one prefetched data block from the storage medium into the cache according to at least one aggregated prefetch IO request; the data accessed by one aggregated prefetch IO request is one prefetched data block.

[0024] Based on the above implementation method, the latency benefits brought by the aggregation of prefetch IO requests are balanced with the latency and space overhead brought by additional prefetching, thereby reducing the overall latency of data prefetching while reducing the number of lower-level IO requests.

[0025] In conjunction with the method provided in the first aspect, in one optional implementation, if the total size of at least one prefetched data block is less than or equal to the total size of the available cache, and the number of valid data blocks in at least one prefetched data block is greater than a first preset threshold, writing prefetched data from the storage medium into the cache is stopped; a valid data block refers to a data block in at least one prefetched data block where the number of missed data is greater than a second preset threshold.

[0026] Based on the above implementation, data prefetching is stopped when there are many valid data blocks in the prefetched data block, so as to avoid the prefetched data block with many valid data blocks being evicted from the cache. This ensures that the prefetched data block can be hit every time and is not evicted from the cache in advance, thereby improving resource utilization.

[0027] In a second aspect, this application provides a control device including a processor and a memory, the control device being communicatively connected to a storage medium and a cache, the processor being configured to execute instructions stored in the memory to cause the control device to perform the method provided in the first aspect or any optional implementation thereof.

[0028] Thirdly, this application provides a storage device including a processor, a cache, and a storage medium, wherein the storage medium is used to persistently store data to be accessed, and the processor is used to execute the method provided in the first aspect or any optional implementation thereof.

[0029] As one possible implementation, the processor of the storage device can be the control device described in the second aspect above.

[0030] As one possible implementation, the storage medium is magnetic tape.

[0031] Fourthly, this application provides a storage system including a data access device and at least one storage device as described in the third aspect, wherein the data access device is used to send IO requests to the storage device.

[0032] Fifthly, this application provides a readable storage medium. This readable storage medium stores a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method provided in the first aspect or any optional implementation thereof.

[0033] Sixthly, this application provides a computer program product. The computer program product includes a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method provided in the first aspect or any optional implementation thereof. Attached Figure Description

[0034] Figure 1 A schematic diagram of the structure of a storage system provided in this application;

[0035] Figure 2 A hierarchical diagram of a storage system provided in this application;

[0036] Figure 3 A flowchart illustrating a data prefetching method provided in this application Figure 1 ;

[0037] Figure 4 A flowchart illustrating a data prefetching method provided in this application Figure 2 ;

[0038] Figure 5 A flowchart illustrating the construction steps of a prediction model provided in this application;

[0039] Figure 6 A flowchart illustrating the clustering and prediction steps provided in this application;

[0040] Figure 7 A flowchart illustrating a clustering step provided in this application;

[0041] Figure 8 A flowchart illustrating a prefetch I / O request merging step provided in this application;

[0042] Figure 9 A flowchart illustrating a balanced prefetching step provided for this application;

[0043] Figure 10 A schematic diagram of the structure of a data prefetching device provided in this application;

[0044] Figure 11 This is a schematic diagram of the structure of a controller provided in this application. Detailed Implementation

[0045] This application provides a data prefetching method, control device, storage device, and system. The processor of the storage device acquires a first I / O request and determines a target prediction model from multiple prediction models based on a first feature of the first I / O request. The processor inputs the first I / O request into the target prediction model, obtains the target prediction address output by the target prediction model, and writes the target prefetched data from the storage medium into a cache. The first feature includes the first address of the data to be accessed by the first I / O request. The prediction models are used to predict the addresses of the prefetched data corresponding to I / O requests at different addresses. The data to be accessed includes the target prefetched data stored at the target prediction address. Thus, since the addresses of the data accessed by multiple I / O requests in a single process are usually quite close, predicting I / O requests for a corresponding range of addresses with a single prediction model is beneficial for predicting the next I / O request of a process when the randomness of concurrent I / O requests from multiple processes is high. Simultaneously, since the address range corresponding to each model is small, predicting addresses for a corresponding range by each prediction model allows the prediction model to accurately predict the address of the prefetched data even when based on interval regression. This improves the prediction accuracy of prefetched data in scenarios of concurrent random I / O requests from multiple processes.

[0046] The technical solutions involved in this application may be applied not only to current magnetic tape technology or storage devices, but also to future magnetic tape technology or storage devices, or to storage systems including magnetic tape media storage or storage devices. The terminology used in the embodiments section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. A brief introduction to some concepts that may be involved in this application is given below.

[0047] Storage medium: A storage material used to record sound, images, digital signals, or other signals. This storage material may include, but is not limited to, magnetic tape, such as a tape-shaped material with a magnetic layer used to record sound, images, digital signals, or other signals. Magnetic tape contains a magnetic medium, such as magnetic powder, for storing data. For example, changes in the magnetic field in this magnetic medium are typically achieved by coating a plastic film substrate (support or backing) with a layer of granular magnetic material or by evaporating and depositing a layer of magnetic oxide or alloy film. The substrate of magnetic tape may include, but is not limited to, paper, celluloid, or polyester film.

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the application will now be described in further detail with reference to the accompanying drawings.

[0049] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0050] Furthermore, in this application, directional terms such as "upper" and "lower" are defined relative to the orientation of the components shown in the accompanying drawings. It should be understood that these directional terms are relative concepts, used for relative description and clarification, and can change accordingly depending on the orientation of the components in the accompanying drawings.

[0051] In distributed storage systems, where storage devices store media assets and high-performance computing (HPC) data, multi-process random workloads arise. This means the order in which the storage system receives I / O requests is not the order in which individual processes send I / O requests, but rather a random mixture of I / O request sequences from different processes. For example, after the storage system receives an I / O request from one process, the next I / O request it receives belongs to another process. Prefetching data based on machine learning and reinforcement learning to learn from all random I / O request sequences from multiple processes and predict future tasks focuses on identifying and predicting individual I / O requests. However, its accuracy is low in scenarios with random concurrency of multi-process tasks.

[0052] To address the aforementioned issues, the application scenarios of the embodiments of this application will be described below with reference to the accompanying drawings.

[0053] Figure 1 This application provides a schematic diagram of a storage system. The storage system includes a data access device 100 and a storage device 120. Figure 1 In the application scenario shown, users access data through applications. The computer running these applications can be referred to as a "computing device".

[0054] Data access device 100 can be a physical machine, a virtual machine, or a container. The physical machine can include, but is not limited to, one or both a client and a smart NIC. For example, data access device 100 includes a client, such as a host, desktop computer, server, laptop, or mobile device. Another example is that data access device 100 includes a smart NIC. This smart NIC, also known as a smart network adapter, in addition to performing the network transmission functions of a standard NIC, provides a built-in programmable and configurable hardware acceleration engine to improve application performance and significantly reduce the CPU consumption of the central processing unit (CPU) in the host connected to the smart NIC during communication, thus providing more CPU resources for the application.

[0055] In one possible example, data access device 100 accesses storage device 120 via a network to access data; for example, the network may include switch 110. The storage system may include multiple data access devices 100 and multiple storage devices 120. Figure 1 Only one data access device 100 and one storage device 120 are shown in the diagram. The data access device 100 can communicate with one or more storage devices 120, and the storage device 120 can be connected to one or more data access devices 100.

[0056] In another possible example, data access device 100 may also communicate with storage device 120 via a wired connection, such as a universal serial bus (USB) or a peripheral component interconnect express (PCIe) bus.

[0057] Figure 1 The storage device 120 shown can be a centralized storage architecture. A characteristic of a centralized storage architecture is that it has a unified entry point; all data from external devices must pass through this entry point, which is the engine 121 of the centralized storage architecture. Engine 121 has management functions.

[0058] like Figure 1 As shown, engine 121 may have one or more controllers. Figure 1The following example illustrates the concept of engine 121 containing one controller. In one possible example, if engine 121 has multiple controllers, any two controllers can have a mirror channel, enabling any two controllers to serve as backups for each other, thereby preventing hardware failures from causing the entire storage device 120 to become unavailable. It should be understood that if engine 121 includes multiple controllers, then engine 121 can also be referred to as the array controller of storage device 120.

[0059] Engine 121 also includes a front-end interface 1211 and a back-end interface 1214. The front-end interface 1211 is used to communicate with the data access device 100 to provide data access services to the data access device 100. The back-end interface 1214 is used to communicate with hard drives to expand the capacity of the storage device 120. Through the back-end interface 1214, engine 121 can connect to more hard drives, thereby forming a very large storage resource pool.

[0060] In terms of hardware, such as Figure 1 As shown, the controller includes at least a processor 1212, memory 1213, and cache 1215. The processor 1212 can be a central processing unit (CPU), graphics processing unit (GPU), or neural network processing unit (NPU), etc., used to process data access requests from outside the storage device 120 (server or other storage devices), and also to process requests generated internally by the storage device 120. For example, when the processor 1212 receives write data requests from the data access device 100 through the front-end interface 1211, it temporarily stores the data in these write data requests in memory 1213. When the total amount of data in memory 1213 reaches a certain threshold, the processor 1212 sends the data stored in memory 1213 to at least one of the following hard drives for persistent storage: mechanical hard drive 1221, solid-state drive (SSD) 1222, tape drive 1223, or other hard drives 1224, through a back-end port. The processor 1212 can be deployed with multiple prediction models, which are used to predict the address of the prefetched data corresponding to the IO request at different addresses. The processor 1212 inputs the received IO request into the target prediction model among the multiple prediction models that matches the address to be accessed by the IO request, obtains the target predicted address, and then reads the prefetched data of the target predicted address from the mechanical hard disk 1221, solid-state hard disk 1222, tape drive device 1223 or other hard disk 1224 into the cache 1215 for storage.

[0061] It should be noted that, Figure 1Only one engine 121 is shown in the figure. However, in actual applications, the storage device 120 may contain two or more engines 121, and redundancy or load balancing may be performed between multiple engines 121.

[0062] The hard drive enclosure 122 includes a control unit 1225 and several hard drives. The control unit 1225 can have various forms. In one case, the hard drive enclosure 122 is a smart enclosure, such as... Figure 1 As shown, the control unit 1225 includes a CPU and memory. The CPU performs operations such as address translation and reading / writing data. The memory is used to temporarily store data to be written to the hard disk or to read data from the hard disk to be sent to the controller. Alternatively, the control unit 1225 can be a programmable electronic component, such as a data processing unit (DPU). A DPU has the versatility and programmability of a CPU, but is more specialized, capable of efficiently operating on network packets, storage requests, or analysis requests. A DPU differs from a CPU by its high degree of parallelism (the ability to handle a large number of requests). Optionally, the DPU can be replaced by a graphics processing unit (GPU), an embedded neural network processing unit (NPU), or other processing chips. Typically, there can be one, two, or more control units 1225. The functions of the control unit 1225 can be offloaded to the network interface card (NIC) 1226. In other words, in this embodiment, the hard disk enclosure 122 does not contain a control unit 1225; instead, the NIC 1226 performs data reading / writing, address translation, and other computational functions. At this point, network interface card 1226 is a smart network interface card. It can contain a CPU and memory. The CPU is used to perform address translation and data reading / writing operations. Memory is used to temporarily store data to be written to the hard drive or to read data from the hard drive to be sent to the controller. It can also be a programmable electronic component, such as a DPU. There is no ownership relationship between network interface card 1226 and the hard drives in hard drive enclosure 122; network interface card 1226 can access any hard drive in hard drive enclosure 122 (e.g., ...). Figure 1 The mechanical hard drive 1221, solid-state drive 1222, tape drive 1223 and other hard drives 1224 shown are examples of hard drives that make it easier to expand hard drives when storage space is insufficient.

[0063] In this embodiment, the tape drive device 1223 refers to a memory that includes a magnetic tape medium. In hardware implementation, this magnetic tape medium memory may include, but is not limited to, a magnetic head, a magnetic tape, and a tape drive. The tape drive can be used to drive the magnetic tape for winding, and the magnetic head can access the magnetic tape during winding, such as writing data to or reading data from the magnetic tape.

[0064] In terms of hardware implementation, the hard disk enclosure 122 can be installed in the storage device 120, or the hard disk enclosure 122 can be encapsulated and set up independently. When the hard disk enclosure 122 exists independently, the hard disk enclosure 122 can also be called a storage device, and this application does not limit it in this way.

[0065] It is worth noting that the above examples are merely possible implementations of the storage system provided in this embodiment and should not be construed as limiting this application. For example, Figure 1 In the storage device 120 shown, data is stored as files on each hard drive. As another example, such as... Figure 2 As shown, the data access device 100 of the storage system is regarded as the access layer, the controller as the cache layer or control layer, and the disk enclosure 122 as the persistent layer. The storage system can be any hardware architecture with the above three layers of functions.

[0066] The following example illustrates the data prefetching method provided in this application, using processor 1212 as an example.

[0067] Figure 3 A flowchart illustrating a data prefetching method provided in this application Figure 1 This data prefetching method can be applied to the above. Figure 1 The controller shown is, for example, executed by processor 1212, using this data prefetching method. Please refer to... Figure 3 The data prefetching method provided in this application may include the following S301-S304.

[0068] S301, Processor 1212 obtains the first IO request.

[0069] In this scenario, the first I / O request is sent by another device to the processor 1212 of the controller in the storage device 120, such as the data access device 100 in the aforementioned embodiment. Corresponding to S301 above, the data access device 100 sends the first I / O request to the processor 1212 of the storage device 120. For example, the first I / O request is an I / O request generated by the data access device 100 based on the data access requirements of the first task. The first I / O request includes an offset and a length, and may also include a process identifier, filename, etc. The offset and length can be referred to as the address of the data to be accessed by the I / O request. The process identifier can be the process identifier of the process to which the I / O request belongs, such as a universally unique identifier (UUID).

[0070] As one possible implementation, the first I / O request can be one of multiple random I / O requests that occur concurrently across multiple processes.

[0071] S302, the processor 1212 determines the target prediction model from multiple prediction models based on the first feature of the first IO request.

[0072] The processor 1212 determines a target prediction model that matches the first feature from multiple prediction models based on the first feature of the first I / O request. A match between the target prediction model and the first feature means a match between the target prediction model and the first I / O request. The multiple prediction models are used to predict the addresses of prefetched data corresponding to I / O requests at different addresses. For example, Figure 3 The prediction addresses of the prediction models shown are [ab], [cd], and [ef].

[0073] The first feature includes the first address of the data that the first I / O request will access. Optionally, the first feature also includes the process identifier, filename, etc., that the first I / O request will access.

[0074] As one possible implementation, multiple prediction models can be deployed in the controller of storage device 120, in the memory 1213 of storage device 120, or in other computing or storage devices that are communicatively connected to storage device 120.

[0075] As one possible implementation, the processor 1212 determines the target prediction model based on the first address of the first IO request and the prediction addresses of multiple prediction models.

[0076] Optionally, the prediction address of each prediction model in the multiple prediction models is the address of the prefetched data obtained by each prediction model based on the input IO request. For example, the second IO request is the previous IO request that was determined by the processor 1212 to match the target prediction model before the first IO request, and the prediction address of the target prediction model is the address of the prefetched data obtained by the target prediction model based on the second IO request.

[0077] For example, the first address of the first IO request includes an offset of 210 and a length of 20, so the address range is [210, 230]. Multiple prediction models include prediction model 1 and prediction model 2, where prediction model 1 has a predicted address of [200, 260] and prediction model 2 has a predicted address of [3, 18].

[0078] In the first example, the offset of the first address is matched with the prediction model to which the nearest prediction address belongs. For example, if the offset of the first address is 210 and belongs to [200, 260], then the processor 1212 determines that prediction model 1 is the target prediction model. In other possible examples, if the offset of the first address is 270, although this offset does not belong to [200, 260], it is closer to [200, 260] than [3, 18], and the processor 1212 can also determine that prediction model 1 is the target prediction model.

[0079] In the second example, the prediction model to which the first address belongs is matched with the prediction model of the prediction address with the highest address overlap. If the address range of the first address [210, 230] belongs to [200, 260], then the processor 1212 determines prediction model 1 as the target prediction model. In other possible examples, if the address range of the first address is [240, 270], and the address overlap between the first address and [200, 260] is greater than the address overlap between the first address and [3, 18], then the processor 1212 determines prediction model 1 as the target prediction model.

[0080] In the third example, the first address matches the prediction model to which the nearest prediction address belongs. In other possible examples, if the offset of the first address is 270 and the length is 20, the range of the first address [270, 290], although not belonging to [200, 260], is closer to [200, 260] than [3, 18], and the processor 1212 can also determine that the prediction model 1 is the target prediction model.

[0081] The three examples above are merely examples provided in this application and do not limit the matching method of features of prediction models and IO requests.

[0082] As one possible implementation, multiple prediction models are constructed from multiple historical I / O requests. Please refer to the steps for constructing the prediction model. Figure 5 The S501-S503 shown will not be described again here.

[0083] As one possible implementation, the multiple prediction models can be of various types, such as linear models and interval prediction models. Please refer to [the relevant documentation] for the prediction methods of different types of prediction models. Figure 5 The S503 shown will not be described in detail here.

[0084] S303, processor 1212 inputs the first IO request into the target prediction model and obtains the target prediction address output by the target prediction model.

[0085] Processor 1212 inputs a first I / O request into the target prediction model, which then outputs a target prediction address based on the first I / O request. The target prediction address is the address of the data that the next I / O request, predicted by the target prediction model, will access based on the first I / O request. Storage media such as hard disk drives 1221, solid-state drives 1222, tape drives 1223, and other hard disks 1224 store the data to be accessed. This data includes the target prefetch data stored at the target prediction address; that is, the target prediction address is the address (logical or physical address) of the target prefetch data in the storage medium.

[0086] As one possible implementation, the target predicted address is the address of the data that the next IO request after the first IO request will access, or the address of the data that the next n IO requests after the first IO request will access, where n is a positive integer greater than or equal to 2.

[0087] S304, Processor 1212 writes the target prefetch data in the storage medium to cache 1215.

[0088] The processor 1212 generates a first prefetch I / O request based on the first address, and writes the target prefetch data in the storage medium to the cache 1215 according to the first prefetch I / O request. The first prefetch I / O request is used to instruct the reading of the target prefetch data stored at the target predicted address in the storage medium.

[0089] As one possible implementation, such as Figure 4 As shown, Figure 4 A flowchart illustrating a data prefetching method provided in this application Figure 2 The data prefetching method provided in this application may also include a prefetch IO request merging step and a balanced prefetching step.

[0090] In a scenario involving multiple concurrent random I / O requests across multiple processes, the processor 1212 receives multiple I / O requests. These requests are then input into corresponding prediction models to obtain multiple predicted addresses. The processor 1212 generates multiple prefetch I / O requests based on these predicted addresses and needs to send these requests to the storage medium for prefetching. Considering that a large number of lower-disk I / O requests can cause queuing for sending and processing prefetch I / O requests, leading to high latency, this application can merge multiple prefetch I / O requests. During the merging process, the latency benefits of reducing the number of prefetch I / O requests are balanced with the latency and space overhead caused by additional prefetching, thereby reducing latency. Please refer to the above prefetch I / O request merging steps. Figure 8 S801-S803 shown will not be described again here.

[0091] Due to the size limitation of cache 1215, some prefetched blocks are evicted before slow processes have hit them, leading to significant inconsistencies in process speed and increasing overall latency. In this embodiment, processor 1212 can employ a balanced prefetching method when executing S304. For example, processor 1212 first estimates the current progress of each process, and then stops prefetching when the process difference is too large, ensuring that the prefetched data is hit each time (hit rate greater than or equal to the threshold) and is not prematurely evicted from cache 1215. Please refer to the above balanced prefetching steps. Figure 9 The S901-S910 shown will not be described in detail here.

[0092] Based on the aforementioned data prefetching method, multiple prediction models are used to predict the prefetched data addresses corresponding to I / O requests at different addresses. Each prediction model makes a prediction for its corresponding address. Since the addresses of data accessed by multiple I / O requests from a single process are usually quite close, having one prediction model predict I / O requests for a corresponding range of addresses is beneficial for predicting the next I / O request of a process when the randomness of concurrent I / O requests from multiple processes is high. Furthermore, because the address range corresponding to each model is relatively small, having each prediction model predict for its corresponding range allows the prediction model to accurately predict the addresses of prefetched data even when using interval regression. Thus, the prediction accuracy of prefetched data in scenarios with concurrent random I / O requests from multiple processes is improved.

[0093] The above text combined Figure 3 and Figure 4 The data prefetching method provided in this application has been described in general. The prediction model in the above embodiments will be described in detail below with reference to the accompanying drawings.

[0094] Please refer to Figure 5 , Figure 5 This is a flowchart illustrating the steps for constructing a prediction model provided in this application. Taking the processor 1212 in the storage device 120 as an example, the steps for constructing the prediction model may include the following steps S501-S503.

[0095] S501, processor 1212 acquires the characteristics of multiple historical IO requests.

[0096] In this context, historical I / O requests are sent from other devices to the processor 1212 of the controller in storage device 120, such as the data access device 100 in the aforementioned embodiment. Corresponding to S501 above, the data access device 100 sends multiple historical I / O requests to the processor 1212 of storage device 120. For example, a historical I / O request is an I / O request generated by the data access device 100 based on the data access requirements of a task. The historical I / O request includes offset and length, and may also include process identifier, filename, etc.

[0097] Multiple historical I / O requests include multiple random I / O requests from multiple processes. These multiple historical I / O requests are a fixed sequence, meaning that the next I / O request for any one of the multiple historical I / O requests is predetermined.

[0098] As one possible implementation, the characteristics of a historical I / O request include the address of the data that the historical I / O request will access. Optionally, this characteristic may also include the process identifier, filename, etc., that the historical I / O request will access.

[0099] S502 and processor 1212 cluster based on the characteristics of multiple historical IO requests to obtain multiple categories.

[0100] Processor 1212 clusters the addresses of data to be accessed by multiple historical I / O requests, resulting in multiple categories. These categories can be viewed as clusters obtained from clustering, or as pseudo-processes composed of multiple I / O requests.

[0101] As one possible implementation, the above clustering can be based on one or more of the following methods: partitioning (e.g., k-means clustering algorithm), hierarchical methods (e.g., balanced iterative reducing and clustering using hierarchies, BIRCH algorithm), density algorithms (e.g., density-based spatial clustering of applications with noise, DBSCAN algorithm), graph theory clustering, grid algorithms (e.g., statistical information grid, STING algorithm), and model algorithms. These will not be elaborated further here.

[0102] As one possible implementation, in addition to the address of the data that the historical IO request will access, the clustering metrics can also include the process identifier, file name, etc. that the historical IO request will access.

[0103] For example, the clustering metrics for historical IO requests identify the processes accessed, with each of the multiple categories corresponding to an actual process; that is, a category contains multiple historical IO requests sent by a single process.

[0104] S503 and processor 1212 establish a prediction model for each of the multiple categories to obtain multiple prediction models.

[0105] The embodiments of this application do not limit the specific type of the prediction model. The prediction model can be a linear model, an interval regression model, etc.

[0106] Among them, the linear model can be the ridge regression model. The ridge regression model is a biased estimation regression method specifically used for collinear data analysis. In essence, it is a modified least squares estimation method. By abandoning the unbiasedness of the least squares method, it obtains regression coefficients that are more realistic and reliable at the cost of losing some information and reducing accuracy. It fits ill-conditioned data better than the least squares method.

[0107] Interval regression is a statistical model used to handle interval-valued data, where both the independent and dependent variables are intervals rather than exact values. It differs from traditional point predictive regression models, which typically provide a single predicted value; interval regression models provide a prediction interval, i.e., the possible range of variable values.

[0108] As a first possible implementation, the characteristics of multiple historical I / O requests include the process identifier of the process to which the historical I / O request belongs. If the addresses of the data accessed by the historical I / O requests in the first category of the multiple categories conform to a linear relationship, the processor 1212 establishes a ridge regression model corresponding to the first category based on the historical I / O requests in the first category. If the addresses of the data accessed by the historical I / O requests in the second category of the multiple categories do not conform to a linear relationship, the processor 1212 establishes an interval regression model corresponding to the second category based on the historical I / O requests in the second category.

[0109] For example, such as Figure 6 As shown, multiple IO requests include [50, 8, 60, 10, 80, 110, 15, 150, 6, 11], where each number represents the offset of a different IO request. The processor 1212 clusters these multiple IO requests into two categories based on their offsets: [50, 60, 80, 110, 150] and [8, 10, 15, 6, 11]. [50, 60, 80, 110, 150] satisfies a linear relationship, so the processor 1212 constructs a ridge regression model based on [50, 60, 80, 110, 150] and uses this model to predict the address of the data to be accessed by the next two IO requests as [200, 260]. [8, 10, 15, 6, 11] does not satisfy a linear relationship, so the processor 1212 constructs an interval regression model based on [8, 10, 15, 6, 11] and uses this model to predict the address to be accessed by the next IO request as [3-18].

[0110] Thus, for different categories of IO requests, depending on whether the category conforms to a linear relationship, the ridge regression model or interval regression model can be selected to predict the address of the data to be prefetched. This ensures the accuracy of the prediction results while guaranteeing that the address to be accessed by the IO request conforms to the linear relationship of the actual process. It also ensures the accuracy of the prediction results when the address to be accessed by the IO request does not conform to the linear relationship, thereby improving the applicability of the data prefetching method.

[0111] As a second possible implementation, the characteristics of multiple historical I / O requests do not include the process identifier of the process to which the historical I / O requests belong. Processor 1212 establishes an interval prediction model for each of the multiple categories, and also establishes a linear model based on the prediction addresses of the interval prediction models corresponding to the multiple categories. The linear model is used to predict the prediction address of the interval prediction model corresponding to the next category based on the multiple categories.

[0112] For example, all I / O requests from the same process have the same length (let's say 1), maintaining the same interval streaming pattern, specifically manifested as: for example, process number i This represents the first I / O request of process i, and the pattern of each offset is as follows: The value is a constant, much larger than the total number of processes, and k is a positive integer. Since process identifiers cannot be obtained, processor 1212 clusters multiple I / O requests into categories corresponding to different predicted addresses, such as 1-50, 1-50+interval, and 1-50+2*interval, grouping I / O requests with similar addresses from different processes into one category. Due to the out-of-order nature of I / O requests from different processes, the linear model corresponding to each category becomes invalid. Processor 1212 uses an interval prediction model to estimate the predicted address of the I / O requests for each category. Furthermore, for relationships between different categories, a linear model is used to predict the arrival of the I / O request for the next category. For example, maintaining a linear model that outputs the minimum and maximum values ​​to predict the address of the I / O request for the next category can also yield an interval prediction result of 1-50+k*interval.

[0113] As one possible implementation, the historical IO requests in this embodiment can be training sets used to build and train multiple prediction models, or they can be requests made earlier than [prefetching timeframe] in actual prefetching scenarios. Figure 3 The first IO request in S301 refers to one or more IO requests sent by any data access device 100 to the storage device 120 according to the task during the actual process execution. That is to say, historical IO requests can be the training set of multiple prediction models during the training process, or IO requests received by multiple prediction models in the actual execution flow of the data prefetching method, such as the first IO request mentioned above, for streaming clustering and prediction.

[0114] Thus, for IO requests of different categories, an interval regression model is used to predict the address of the prefetched data based on the address range corresponding to each category. At the same time, a linear model is used to predict the address of the interval prediction model corresponding to the next category. This allows the interval prediction model to be used for prediction when the address of the IO request corresponds to the predicted address of any interval prediction model. When the address of the IO request does not correspond to the predicted address of any interval prediction model, the linear model is used to generate the predicted address of the interval prediction model corresponding to the next category, and then the interval prediction model is used for prediction.

[0115] The construction steps of multiple prediction models have been explained above with reference to the accompanying diagrams. After the multiple prediction models are constructed, the processor 1212 can further cluster subsequent IO requests based on the multiple prediction models, classifying the subsequent IO requests into the categories of the corresponding prediction models, and performing streaming training on the prediction models. These subsequent IO requests can be any additional historical IO requests following the multiple historical IO requests described in S501 above.

[0116] Please refer to Figure 7 , Figure 7 This is a flowchart illustrating a clustering step provided in this application. The clustering step may include the following steps S701-S703.

[0117] S701 and processor 1212 input the first historical IO request into multiple prediction models respectively, and obtain multiple prediction addresses output by multiple prediction models.

[0118] The processor 1212 inputs the first historical IO request into multiple prediction models respectively, and obtains the prediction address output by each prediction model based on the first historical IO request.

[0119] After S701, the predicted addresses of multiple prediction models are not updated based on the first historical IO request.

[0120] S702, processor 1212 determines the first prediction model corresponding to the prediction address with the highest address overlap among multiple prediction addresses.

[0121] The address overlap is defined as the degree of overlap between the predicted address obtained by the prediction model based on the first historical I / O request and the address of the data to be accessed by the second historical I / O request. The second historical I / O request is the next request after the first historical I / O request among multiple historical I / O requests.

[0122] As one possible implementation, the prediction model is based on the overlap between the predicted address obtained from the first historical IO request and the address of the data to be accessed by the second historical IO request. The specific calculation method can be found in the following formula (1).

[0123] g = argmax gHitbytes(req,g) (1)

[0124] Where g is the first prediction model, argmax g This indicates that (req,g) is taken as the value that makes Hitbytes(req,g) reach its maximum value. Hitbytes(req,g) represents the degree of overlap between the predicted address obtained by the prediction model based on the first historical IO request and the address of the data to be accessed by the second historical IO request. req represents the first historical IO request.

[0125] Since multiple historical I / O requests are a sequence ordered by request time, the processor 1212 can determine the next request after the first historical I / O request, i.e., the second historical I / O request, based on the timestamps of the multiple historical I / O requests.

[0126] S703, processor 1212 classifies the first historical IO request into the category corresponding to the first prediction model.

[0127] After classifying the first historical IO request into the category corresponding to the first prediction model, the processor 1212 updates the prediction address of the first prediction model to the address output by the first prediction model based on the first historical IO request.

[0128] In this way, using the predicted addresses of the prediction model to classify historical IO requests improves the clustering accuracy of historical IO requests, thereby improving the quality of the training data of the prediction model and thus improving the prediction accuracy of the prediction model.

[0129] The above text combined Figures 3-7 The paper details how to construct multiple prediction models and how to use them to obtain the target predicted address. After obtaining the target predicted address, the controller writes the target prefetch data from the storage medium to the cache 1215 via the processor 1212. In scenarios with multiple concurrent random I / O requests from multiple processes, the processor 1212 will receive multiple prefetch I / O requests from multiple prediction models simultaneously or within a short period. The number of disk I / O requests that can be processed simultaneously within a unit of time in the storage medium is limited. Considering the latency required for disk processing, an excessive number of disk requests within a unit of time will lead to a significant increase in disk request queuing time and increased latency due to process blocking. Therefore, this application can also merge (or aggregate) multiple prefetch I / O requests to improve the overall efficiency of data prefetching.

[0130] Please refer to Figure 8 , Figure 8 This application provides a flowchart illustrating a prefetch I / O request merging step. This prefetch I / O request merging step may include the following steps S801-S803.

[0131] S801, processor 1212 acquires multiple prefetch I / O requests.

[0132] Among them, multiple prefetch I / O requests can be obtained by the processor 1212 inputting multiple I / O requests into multiple prediction models.

[0133] S802, processor 1212 merges multiple prefetch IO requests according to their characteristics to obtain at least one aggregated prefetch IO request.

[0134] The characteristics of multiple prefetch I / O requests include at least one of the following: filename and address.

[0135] As one possible implementation, the constraint for merging n prefetch I / O requests into a single aggregated prefetch I / O request is that the latency gain of the aggregated prefetch I / O request compared to the issuance of n individual prefetch I / O requests is greater than the sum of the latency and space overhead of the additional prefetching in the aggregated prefetch I / O request.

[0136] Optionally, the above constraints can be expressed as formula (2) below. If n prefetch IO requests and multiple unmerged IO requests satisfy formula (2), then proceed to step S803. If n prefetch IO requests and multiple unmerged IO requests do not satisfy formula (2), then write the prefetched data in the storage medium to cache 1215 according to the n prefetch IO requests.

[0137] N waste (t read +λC space )-(nN IO (Number of items distributed)*t seek >0 (2)

[0138] Where, N waste For additional prefetching, t read For latency overhead (read latency), C space For space overhead, N IO (Number of requests sent) represents the number of aggregated prefetch IO requests, t seek This refers to the latency of sending IO requests.

[0139] The aforementioned additional prefetch refers to the portion of data that the aggregated prefetch IO request will access that does not overlap with the data accessed by multiple prefetch IO requests.

[0140] S803, processor 1212 writes at least one prefetch data block from storage medium to cache 1215 based on at least one aggregated prefetch I / O request.

[0141] Processor 1212 caches prefetched data blocks according to the aggregated prefetch I / O request and Figure 3 The principle of S304 is the same, so it will not be repeated here.

[0142] In this way, the latency benefits of aggregating prefetch IO requests are balanced with the latency and space overhead of additional prefetching, thereby reducing the overall latency of data prefetching while reducing the number of lower-level IO requests.

[0143] The above text combined Figure 8 The method for merging multiple prefetch I / O requests is described in detail. After the controller caches at least one prefetch data block through processor 1212, due to the limitation of cache size 1215, some prefetch data blocks are evicted before slow processes have hit them. This leads to severe inconsistencies in process speed and increases the overall latency. This application can also determine whether to continue issuing prefetch I / O requests or aggregate prefetch I / O requests based on the hit status of each prefetch data block to avoid prefetch data blocks being prematurely evicted from memory.

[0144] The principle of the above-mentioned balanced prefetching step is as follows: if the total size of at least one prefetched data block is less than or equal to the total available cache size of cache 1215, and the number of valid data blocks in at least one prefetched data block is greater than a first preset threshold, the processor 1212 stops writing prefetched data from the storage medium to cache 1215. The processor 1212 stopping writing prefetched data from the storage medium to cache 1215 can be based on, for example... Figure 3 As shown in S304, processor 1212 refuses to cache the target prefetch data corresponding to the target prediction address determined in S303 according to the equalization prefetch step. This can also be for... Figure 3 The next IO request after the first IO request shown in S304, i.e., the processor 1212 refuses to cache the target prefetch data corresponding to the target prediction address determined after being input into the target prediction model.

[0145] Please refer to Figure 9 , Figure 9 This application provides a flowchart illustrating a leveling prefetch step. The leveling prefetch step may include the following steps S901-S910.

[0146] S901, Processor 1212 Initialization S total =0.

[0147] Among them, S total The total size of the prefetched data, which includes at least one prefetched data block.

[0148] S902, Processor 1212 determines S total After increasing the size of the prefetch data block corresponding to the prefetch I / O request, S total Is it greater than the total size of the available cache?

[0149] A prefetch IO request can be a single prefetch IO request or an aggregated prefetch IO request resulting from the merging of multiple prefetch IO requests.

[0150] S903, if S total >M, processor 1212 refuses to issue a prefetch I / O request.

[0151] Where M is the total size of the available cache for cache 1215.

[0152] S904, if S total ≤M, processor 1212 executes S total =S total +S i .

[0153] Among them, S i The size of the prefetch data block corresponding to the prefetch IO request.

[0154] S905, processor 1212 determines whether the prefetched data block has been hit.

[0155] S906. If the prefetch block is hit, TN k =TN k +1.

[0156] Among them, TN k This represents the actual number of times prefetched data block k was hit by IO requests.

[0157] S907, Processor 1212 determines TN k Is it less than N? k .

[0158] Where, N k This represents the predicted hit count of prefetched data block k being hit by an IO request.

[0159] S908, if TN k <N k The processor 1212 determines that the prefetched data block k is a valid data block.

[0160] TN k <N k This indicates that there is still un-hit data in the prefetched data block k. A valid data block is a data block in which at least one prefetched data block has more un-hit data than a second preset threshold. The second preset threshold can be N. k Thus, the number of valid data blocks in at least one database increases by 1.

[0161] S909, the processor 1212 determines whether the number of valid data blocks in at least one prefetched data block is greater than a first preset threshold.

[0162] The first preset threshold can be determined based on the percentage of the number of at least one prefetched data block. For example, if at least one prefetched data block includes 10 prefetched data blocks, the first preset threshold can be 80% of the 10 prefetched data blocks, that is, the first preset threshold is 8.

[0163] S910. If the number of valid data blocks in at least one prefetched data block is greater than the first preset threshold, stop writing the prefetched data in the storage medium to the cache 1215.

[0164] In a possible embodiment, if the number of valid data blocks in at least one prefetched data block is less than or equal to a first preset threshold, the prefetched data in the storage medium is written to cache 1215.

[0165] As one possible implementation, the above S total S i TN k N k The statistics for prefetched data blocks are stored in a linked list maintained by processor 1212. In TN... k =N k When the processor 1212 removes the linked list entry corresponding to the prefetched data block k from the linked list and removes the prefetched data block k from the cache 1215.

[0166] In this way, when there are many valid data blocks in the prefetched data block, the data prefetching is stopped, which avoids the prefetched data block with a large number of valid data blocks being evicted from the cache. This ensures that the prefetched data block can be hit every time and is not evicted from the cache in advance, thus improving resource utilization.

[0167] To complement the data prefetching method provided in this application, this application also provides a data prefetching apparatus 1000, which is used to execute the aforementioned data prefetching method. Figure 10 As shown, the data prefetching device 1000 includes an acquisition module 1010, a model determination module 1020, a prediction module 1030, and a caching module 1040.

[0168] For example, the data prefetching device 1000 can achieve Figure 3 The functions of the processor 1212.

[0169] The acquisition module 1010 is used to acquire the first input / output IO request;

[0170] The model determination module 1020 is used to determine a target prediction model from multiple prediction models based on a first feature of the first IO request; the first feature includes a first address of the data to be accessed by the first IO request, and the multiple prediction models are used to predict the address of the prefetched data corresponding to IO requests at different addresses;

[0171] The prediction module 1030 is used to input the first IO request into the target prediction model and obtain the target prediction address output by the target prediction model; the data to be accessed includes the target prefetch data stored in the target prediction address;

[0172] The cache module 1040 is used to write target prefetch data from the storage medium into the cache.

[0173] As one possible implementation, the model determination module 1020 is specifically used to: determine the target prediction model based on the first address and the prediction addresses of multiple prediction models; the prediction address of the target prediction model is the prediction address obtained by the target prediction model based on the second IO request, and the second IO request is earlier than the first IO request.

[0174] As one possible implementation, the acquisition module 1010 is also used to: acquire the features of multiple historical IO requests. The data prefetching device 1000 also includes a clustering module and a model building module. The clustering module is used to: cluster the multiple historical IO requests according to their features to obtain multiple categories; the features of the historical IO requests include the addresses of the data that the historical IO requests will access. The model building module is used to: build a prediction model based on each of the multiple categories to obtain multiple prediction models.

[0175] As one possible implementation, the clustering module is specifically used for: for the first historical IO request among multiple historical IO requests, inputting the first historical IO request into multiple prediction models respectively to obtain multiple predicted addresses output by multiple prediction models; determining the first prediction model corresponding to the prediction address with the highest address overlap among multiple prediction addresses; the address overlap is the degree of overlap between the prediction address obtained by the prediction model based on the first historical IO request and the address of the data to be accessed by the second historical IO request, the second historical IO request being the next request after the first historical IO request; and classifying the first historical IO request into the category corresponding to the first prediction model.

[0176] As one possible implementation, the characteristics of multiple historical I / O requests include the process identifier of the process to which the historical I / O requests belong. Specifically, the model building module is used to: if the addresses of the data accessed by historical I / O requests in the first category of multiple categories conform to a linear relationship, establish a ridge regression model corresponding to the first category based on the historical I / O requests in the first category; if the addresses of the data accessed by historical I / O requests in the second category of multiple categories do not conform to a linear relationship, establish an interval regression model corresponding to the second category based on the historical I / O requests in the second category.

[0177] As one possible implementation, the characteristics of multiple historical I / O requests do not include the process identifier of the process to which the historical I / O requests belong. The model building module is specifically used for: establishing an interval prediction model for each of the multiple categories; establishing a linear model based on the prediction addresses of the interval prediction models corresponding to the multiple categories; and using the linear model to predict the prediction address of the interval prediction model corresponding to the next category based on the multiple categories.

[0178] As one possible implementation, the cache module 1040 is specifically used to: write target prefetch data in the storage medium into the cache according to the first prefetch IO request; the first prefetch IO request is used to instruct the reading of target prefetch data stored at the target prediction address in the storage medium.

[0179] As one possible implementation, the cache module 1040 is also used to: obtain multiple prefetch IO requests; merge the multiple prefetch IO requests according to their characteristics to obtain at least one aggregated prefetch IO request; the characteristics of the multiple prefetch IO requests include at least one of filename and address, and the constraint condition for merging n prefetch IO requests into one aggregated prefetch IO request is that the latency gain of the aggregated prefetch IO request relative to the n prefetch IO requests is greater than the sum of the latency and space overhead of the additional prefetch of the aggregated prefetch IO request, where additional prefetch refers to the part of the data that the aggregated prefetch IO request will access that does not overlap with the data that the multiple prefetch IO requests will access; write at least one prefetched data block from the storage medium into the cache according to at least one aggregated prefetch IO request; the data that one aggregated prefetch IO request will access is one prefetched data block.

[0180] As one possible implementation, the cache module 1040 is further configured to: stop writing prefetched data from the storage medium into the cache if the total size of at least one prefetched data block is less than or equal to the total size of the available cache, and the number of valid data blocks in at least one prefetched data block is greater than a first preset threshold; a valid data block refers to a data block in at least one prefetched data block where the number of missing data is greater than a second preset threshold.

[0181] It should be understood that the above Figure 10 The data prefetching device 1000 provided is illustrated by the division of the above-mentioned functional modules when implementing its functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0182] This application also provides a control device, which includes a processor and is communicatively connected to a memory and a cache. The processor is used to execute instructions stored in the memory to cause the control device to perform the above-described data prefetching method.

[0183] For example, the control device is a controller 1100, which can be called a storage controller. Figure 11 As shown, the controller 1100 includes a chip 1110 and a communication interface 1120. The communication interface 1120 is used to receive I / O requests. The chip 1110 is used to process I / O requests and issue instructions through the communication interface 1120.

[0184] The controller 1100 may include one or more chips 1110, and each chip 1110 may be an integrated circuit. An operating system and other software programs are installed on the chip 1110, enabling it to access magnetic tape media storage and various PCIe devices. The chip 1110 includes one or more cores.

[0185] Optionally, the controller 1100 may also include, but is not limited to, other storage media: dynamic random access memory (DRAM), static random access memory (SRAM), etc., for caching data from the magnetic tape storage medium for processing by the chip 1110. Alternatively, other storage media may be read-only memory (ROM). For example, read-only memory may be programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc. This embodiment does not limit the number or type of other storage media. Furthermore, other storage media can be configured to have power-saving functionality. Power-saving functionality means that when the system experiences a power outage and is then powered on again, the data stored in the memory will not be lost. Storage media with power-saving functionality are called non-volatile memory.

[0186] For example, controller 1100 can be used to implement such Figure 3 The function of processor 1212 in the data prefetching method shown.

[0187] This application also provides a magnetic tape system. The magnetic tape system includes: a communication interface, a storage controller, and the magnetic tape media storage provided in any of the foregoing embodiments. The magnetic tape media storage is used for persistent storage of access data, and the communication interface is used for receiving I / O requests; the storage controller is used to manage target magnetic tape media storage in the magnetic tape system according to I / O requests (such as read requests or write requests). The magnetic tape system is, for example, a magnetic tape library, or a computer / server that includes magnetic tape media storage as a persistent storage medium.

[0188] The storage controller includes one or more processors, such as the control device described above. The processor can be a very large-scale integrated circuit (VLSI). The processor contains an operating system and other software programs, enabling it to access the magnetic tape media storage and various PCIe devices. The processor includes one or more processor cores. These processor cores can be, for example, CPUs or other ASICs. The processor can also be other general-purpose processors, digital signal processing (DSP) chips, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In practical applications, the magnetic tape system may also include multiple controllers.

[0189] Optionally, the tape system may also include, but is not limited to, other storage media such as DRAM and SRAM, for caching data in the tape media memory for processor processing. Other storage media can also be ROM. For read-only memory, for example, it could be PROM or EPROM. This embodiment does not limit the number or type of other storage media. Furthermore, other storage media can be configured to have power-saving functionality. Power-saving functionality means that when the system experiences a power outage and is then powered on again, the data stored in the memory will not be lost. Storage media with power-saving functionality are called non-volatile memory.

[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Various equivalent modifications or substitutions can be conceived within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data prefetching method, characterized in that, Applied to a storage device, the storage device including a processor, a cache, and a storage medium, the storage medium being used for persistent storage of data to be accessed, the method being executed by the processor, including: Get the first input / output I / O request; A target prediction model is determined from multiple prediction models based on a first feature of the first IO request; the first feature includes a first address of the data to be accessed by the first IO request, and the multiple prediction models are respectively used to predict the address of the prefetched data corresponding to IO requests at different addresses; The first IO request is input into the target prediction model to obtain the target prediction address output by the target prediction model; the data to be accessed includes target prefetch data stored in the target prediction address. The target prefetch data in the storage medium is written into the cache.

2. The method according to claim 1, characterized in that, The step of determining the target prediction model from multiple prediction models based on the first feature of the first IO request includes: The target prediction model is determined based on the first address and the prediction addresses of the plurality of prediction models; the prediction address of the target prediction model is the prediction address obtained by the target prediction model based on the second IO request, the second IO request being earlier than the first IO request.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain the characteristics of multiple historical I / O requests; Clustering is performed based on the characteristics of the multiple historical I / O requests to obtain multiple categories; the characteristics of the historical I / O requests include the address of the data that the historical I / O requests will access; The multiple prediction models are obtained by building a prediction model for each of the multiple categories.

4. The method according to claim 3, characterized in that, The method further includes: For the first historical IO request among the multiple historical IO requests, the first historical IO request is input into the multiple prediction models respectively to obtain multiple predicted addresses output by the multiple prediction models; A first prediction model is determined for the prediction address with the highest address overlap among the multiple prediction addresses; the address overlap is the degree of overlap between the prediction address obtained by the prediction model based on the first historical IO request and the address of the data to be accessed by the second historical IO request, and the second historical IO request is the next request after the first historical IO request. The first historical IO request is assigned to the category corresponding to the first prediction model.

5. The method according to claim 3 or 4, characterized in that, The features of the multiple historical I / O requests include the process identifier of the process to which the historical I / O requests belong, and the prediction model built based on each of the multiple classifications includes: If the addresses of the data accessed by historical IO requests in the first category of the multiple categories conform to a linear relationship, a ridge regression model corresponding to the first category is established based on the historical IO requests in the first category. If the addresses of data accessed by historical IO requests in the second category of the multiple categories do not conform to a linear relationship, an interval regression model corresponding to the second category is established based on the historical IO requests in the second category.

6. The method according to claim 3 or 4, characterized in that, The features of the multiple historical I / O requests do not include the process identifier of the process to which the historical I / O requests belong. The multiple prediction models are obtained by building a prediction model for each of the multiple classifications, including: Establish an interval prediction model for each of the multiple categories; A linear model is established based on the predicted addresses of the interval prediction models corresponding to the multiple categories; the linear model is used to predict the predicted address of the interval prediction model corresponding to the next category based on the multiple categories.

7. The method according to any one of claims 1-6, characterized in that, The step of writing the target prefetch data in the storage medium into the cache includes: A first prefetch I / O request is generated based on the target predicted address; Read the target prefetch data from the storage medium according to the first prefetch IO request; Write the target prefetched data into the cache.

8. The method according to claim 7, characterized in that, The method further includes: Get multiple prefetch I / O requests; The multiple prefetch I / O requests are merged based on their characteristics to obtain at least one aggregated prefetch I / O request. The characteristics of the multiple prefetch I / O requests include at least one of filename and address. The constraint condition for merging n prefetch I / O requests into one aggregated prefetch I / O request is that the latency gain of the aggregated prefetch I / O request relative to the n prefetch I / O requests is greater than the sum of the latency and space overhead of the additional prefetching of the aggregated prefetch I / O request. The additional prefetching refers to the part of the data that the aggregated prefetch I / O request will access that does not overlap with the data that the multiple prefetch I / O requests will access. At least one prefetched data block in the storage medium is written to the cache according to the at least one aggregated prefetch IO request; an aggregated prefetch IO request accesses a prefetched data block.

9. The method according to claim 8, characterized in that, The method further includes: If the total size of the at least one prefetched data block is less than or equal to the total size of the available cache of the cache, and the number of valid data blocks in the at least one prefetched data block is greater than a first preset threshold, the writing of prefetched data from the storage medium into the cache is stopped; the valid data block refers to the data block in the at least one prefetched data block where the number of missing data is greater than a second preset threshold.

10. A control device, characterized in that, The control device includes a processor and is communicatively connected to a memory and a cache. The processor is configured to execute instructions stored in the memory to cause the control device to perform the method as described in any one of claims 1-9.

11. A storage device, characterized in that, include: Processor, cache, and storage media; The storage medium is used to persistently store data to be accessed; The processor is used to perform the method as described in any one of claims 1-9.

12. The device according to claim 11, characterized in that, The storage medium is magnetic tape.

13. A storage system, characterized in that, include: A data access device, and one or more storage devices as described in claim 11 or claim 12; The data access device is used to send I / O requests to the storage device.

14. A computer program product containing instructions, characterized in that, When the instruction is executed by the processor, the processor performs the method as described in any one of claims 1-9.

15. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-9.

Images