Data pre-reading method, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a data pre-reading method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of communication, the method is applied to an operating system, the operating system comprises a general block layer, an IO scheduling layer and a block device driving layer, and the method comprises the steps that in a first idle period of the general block layer, the IO scheduling layer is switched to the block device driving layer; determining pre-reading window information based on the current reading request through the general block layer, wherein the pre-reading window information is used for indicating a pre-reading request of the to-be-pre-read object; applying for a first physical memory for the to-be-pre-read object based on the pre-read window information through the block device driving layer, wherein the first physical memory is used for making cache preparation for the to-be-pre-read object; wherein the first idle time period is a waiting time period for waiting to receive read request completion information after the general block layer issues the IO request corresponding to the current read request to the IO scheduling layer. According to the method and the device, under the condition of not increasing the overhead burden, the data reading rate of the operating system is improved, and the universality is relatively high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and particularly relates to a data pre-reading method, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] In the related art, the pre-reading algorithm and optimization thereof are generally applied in a distributed storage system (i.e., a file system abstraction layer) or a user state application program, and a mainstream idea is to reduce the number of IO (Input / Output) times by merging or expanding a read request issued by a client, or to pre-read disk information into a cache according to a pre-reading range determined by a current read request range, so as to expect a cache hit in a next read request.

[0003] However, the traditional mainstream pre-reading optimization scheme has the following disadvantages:

[0004] 1) Poor universality

[0005] In a microkernel architecture, the software level division of a file system to a physical storage device includes a file system abstraction layer, a general block layer, an IO scheduling layer and a block device driver layer, which all exist in the form of a user state service. Compared with a monolithic kernel, the general block layer, the IO scheduling layer and the block device driver layer are located in the microkernel space.

[0006] This results in that when a specific implementation is involved, due to the great difference between the implementations of various layers related to the block device in different architectures of operating systems, the method applicable in the monolithic kernel architecture has poor applicability in the microkernel architecture, and vice versa.

[0007] 2) Lack of optimization of time-consuming parts in the overall software process

[0008] With some mainstream data copy or migration technologies, such as DMA (Direct Memory Access) and SIMD (Single Instruction Multiple Data) instructions, and the continuous optimization of specific storage devices, the performance of data access and processing is getting higher and higher, and in addition, the pre-reading and writing functions supported by the storage devices are getting stronger and stronger.

[0009] This results in a large number of communications between different software layers, frequent communications between user mode services and the kernel (or kernel mode and user mode switching), and performance loss during these periods, which accounts for a large proportion of the performance loss during the entire file reading period. In particular, considering the characteristics of the microkernel architecture (i.e., the microkernel can reduce the complexity of the kernel, improve the security of the kernel, and make the kernel more stable and reliable. However, the microkernel often needs to use inter-process communication and other methods to implement certain functions, which often reduces the execution efficiency of the system), this proportion is more obvious.

[0010] How to reduce these communication losses needs to be optimized as a priority. Unfortunately, the mainstream pre-reading optimization often does not involve it.

[0011] 3) Increased overhead burden

[0012] First, the traditional mainstream pre-reading behavior is performed during the sensitive period of the entire read request. The sensitive period refers to the period when the current service process is performing actual effective work business processes, such as analyzing the content of the obtained IO request, and some overhead has been introduced, thereby increasing the overhead burden. SUMMARY

[0013] The main purpose of the present application is to provide a data pre-reading method, an electronic device, a storage medium and a computer program product, aiming to solve at least one technical problem mentioned in the above background art.

[0014] To achieve the above-mentioned purpose, the present application provides a data pre-reading method, which is applied to a microkernel, the microkernel includes a general block layer, an IO scheduling layer and a block device driver layer, and the method comprises:

[0015] In the first idle period of the general block layer, the general block layer determines the pre-reading window information based on the current read request, and the pre-reading window information is used to indicate the pre-reading request for the to-be-pre-read object;

[0016] The block device driver layer applies the first physical memory for the to-be-pre-read object based on the pre-reading window information, and the first physical memory is used for cache preparation for the to-be-pre-read object;

[0017] Among them, the first idle period is a waiting period for the general block layer to wait for receiving read request completion information after the general block layer issues the IO request corresponding to the current read request to the IO scheduling layer.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a data pre-reading method, which is applied to a microkernel, the microkernel includes a general block layer, an IO scheduling layer and a block device driver layer, and the method comprises:

[0019] In the second idle period of the block device driver layer, a pre-read window information is determined by the block device driver layer based on the to-be-read object of the current IO request, and the pre-read window information is used to indicate a pre-read request for the to-be-pre-read object;

[0020] A second physical memory is applied for the to-be-pre-read object by the block device driver layer based on the pre-read window information, and the second physical memory is used for cache preparation for the to-be-pre-read object.

[0021] The second idle period is a waiting period in which the block device driver layer sends a physical memory call request of the to-be-read object to a file system abstraction layer and waits for DMA migration of the to-be-read object to be completed, and the physical memory call request is used to trigger application of a third physical memory corresponding to the to-be-read object to the file system abstraction layer, and after the third physical memory application is completed, the to-be-read object is DMA migrated to the third physical memory for caching.

[0022] In addition, to achieve the above object, the present application also provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the data pre-reading method.

[0023] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and a data pre-reading method program is stored on the computer readable storage medium, and the data pre-reading method program is executed by a processor to implement the steps of the data pre-reading method.

[0024] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the data pre-reading method.

[0025] The present application provides a data pre-reading method, and the present application embodiment first applies the data pre-reading method to an operating system, the operating system comprising a general block layer, an IO scheduling layer and a block device driver layer, and the technical solution of the present application embodiment is that in a first idle period of the general block layer, a pre-read window information is determined by the general block layer based on a current read request, the pre-read window information being used to indicate a pre-read request for a to-be-pre-read object, and then a first physical memory is applied for the to-be-pre-read object by the block device driver layer based on the pre-read window information, and the first physical memory is used for cache preparation for the to-be-pre-read object. The first idle period is a waiting period in which the general block layer waits for read request completion information returned by the IO scheduling layer after the IO request corresponding to the current read request is sent to the IO scheduling layer.

[0026] Alternatively, the technical solution of the embodiment of the present application is to determine, by the block device driver layer, the pre-reading window information of the object to be read based on the object to be read of the current IO request in the second idle period of the block device driver layer, the pre-reading window information being used to indicate a pre-reading request for the object to be pre-read, and to apply, by the block device driver layer, the second physical memory for the object to be pre-read based on the pre-reading window information, the second physical memory being used for cache preparation for the object to be pre-read. The second idle period is a waiting period for the block device driver layer to send a physical memory call request of the object to be read to the file system abstraction layer, and wait for DMA migration of the object to be read to be completed. The physical memory call request is used to trigger the file system abstraction layer to apply a third physical memory corresponding to the object to be read, and after the third physical memory is applied, the object to be read is migrated to the third physical memory for caching.

[0027] The embodiment of the present application optimizes pre-reading based on the above background technology:

[0028] 1) In response to the current read request, the non-sensitive time period (i.e. the first idle period or the second idle period) in the common service process of the general block layer or the device driver layer is effectively utilized, that is, the pre-reading window information is calculated according to the current read request range, and the time-consuming part of the next read IO request is performed in advance based on the calculated pre-reading window information, which greatly reduces the work of responding to the next read IO request.

[0029] The pre-reading optimization method of the embodiment of the present application can be agglomerated in a specific software level, and does not depend on other components. That is, the embodiment of the present application can be implemented in the general block layer or the device driver layer alone, and does not conflict with other module optimization. Even if the pre-reading optimization already exists in the upper layer (such as the file system abstraction layer), it will not conflict with it, and will also have the effect of adding a cherry on top. Macro kernel and micro kernel architecture can be applied, and the universality is strong, especially for micro kernel architecture, which is more obvious.

[0030] The pre-reading optimization process of the embodiment of the present application belongs to the common service process part of the general block layer or the block device driver layer, and does not need to be partitioned and customized according to the characteristics of different types of file systems or block devices. Compared with the traditional mainstream pre-reading scheme, the universality is stronger.

[0031] 2) The pre-reading window size is determined by analyzing the received IO request, and in the non-sensitive time period of the current business process, the work of reducing the time-consuming part of the next read IO operation is performed in advance according to the pre-reading window range, so as to realize optimization of the time-consuming part from the whole software process.

[0032] 3) in the current response process, the non-sensitive time period in the common block layer or the common service process of the driving layer is effectively utilized, so that the most of the overhead in the next data reading process is well "hidden" without introducing additional overhead, and the overhead burden is reduced. In addition, unlike the traditional pre-reading scheme, although the physical memory page is applied in advance in the embodiment of the application, no disk data moving operation is performed, and only when the next read request hits the pre-reading window, the data moving operation is performed, thereby reducing the overhead introduced by the invalid operation when the pre-reading window is not hit. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0035] Figure 1 The flowchart provided for the data pre-reading method embodiment one of the application;

[0036] Figure 2 The flowchart provided for the data pre-reading method embodiment three of the application;

[0037] Figure 3 The hierarchical structure diagram from the file system in the operating system to the storage device;

[0038] Figure 4 The core service component diagram of the block device under the microkernel architecture;

[0039] Figure 5 The core service component diagram of the block device under the macrokernel architecture;

[0040] Figure 6 The flowchart of the traditional sequential reading of the disk file;

[0041] Figure 7 The flowchart provided for the data pre-reading method embodiment one of the application;

[0042] Figure 8 The flowchart provided for the data pre-reading method embodiment two of the application;

[0043] Figure 9 A first processing schematic diagram for pre-reading the physical memory of the application;

[0044] Figure 10 A second processing schematic diagram for pre-reading the physical memory of the application;

[0045] Figure 11 A third processing schematic diagram for pre-reading the physical memory of the application;

[0046] Figure 12 A fourth processing schematic diagram for pre-reading the physical memory of the application;

[0047] Figure 13 A fifth processing schematic diagram for pre-reading the physical memory of the application;

[0048] Figure 14 A device structure schematic diagram of a hardware running environment involved in the data pre-reading method in the embodiment of the application.

[0049] The object implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0050] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise represented. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0051] It should be understood that the specific embodiments described herein merely serve to explain the technical solutions of the present application and do not limit the present application.

[0052] At present, the traditional mainstream pre-reading optimization scheme has the following disadvantages:

[0053] 1) Poor universality

[0054] Under the microkernel architecture, in the software level division of the file system to the physical storage device, the file system abstraction layer, the general block layer, the IO scheduling layer and the block device driver layer all exist in the form of user state service. Compared with the macrokernel, the general block layer, the IO scheduling layer and the block device driver layer are located in the microkernel space.

[0055] This results in that when it comes to specific implementation, due to the great difference between the implementations of each layer of the block device of different architectures of operating systems, the methods applicable in the macrokernel architecture have poor applicability in the microkernel architecture, and vice versa.

[0056] 2) Lack of optimization of time-consuming parts from the overall software flow

[0057] With some mainstream data copy or migration technologies, such as DMA, SIMD instructions, and continuous optimization of specific storage devices, the performance of data access and processing is getting higher and higher, and in addition, the pre-reading and writing functions supported by the storage device itself are getting stronger and stronger.

[0058] This results in a large amount of communication between different software layers due to a single read disk file operation, frequent communication between user mode services and the kernel (or kernel mode and user mode switching), and the performance loss during these periods, which will account for a large proportion of the performance loss during the entire file reading period. In particular, considering the characteristics of the microkernel architecture (i.e., the microkernel can reduce the complexity of the kernel, improve the security of the kernel, and make the kernel more stable and reliable. However, the microkernel often needs to use inter-process communication and other methods to implement certain functions, which often reduces the execution efficiency of the system), this proportion will be more obvious.

[0059] How to reduce these communication losses needs to be optimized as a priority, unfortunately, the mainstream pre-reading optimization often does not involve it.

[0060] 3) Growing overhead burden

[0061] The traditional mainstream pre-reading behavior is performed during the sensitive period of the entire read request. The sensitive period refers to the period when the current service process is performing actual effective business processes, such as analyzing the content of the obtained IO request, and some overhead has been introduced, thereby increasing the overhead burden.

[0062] The main solution of the embodiment of the present application is: in the first idle period of the general block layer, the general block layer determines the pre-reading window information based on the current read request, the pre-reading window information is used to indicate the pre-reading request for the to-be-pre-read object; the block device driver layer applies for the first physical memory for the to-be-pre-read object based on the pre-reading window information, and the first physical memory is used for cache preparation for the to-be-pre-read object, wherein the first idle period is a waiting period for the general block layer to wait for receiving the read request completion information after the IO request corresponding to the current read request is sent to the IO scheduling layer.

[0063] Alternatively, in a second idle period of the block device driver layer, the block device driver layer determines pre-reading window information based on the to-be-read object of the current IO request, the pre-reading window information being used to indicate a pre-reading request for the to-be-pre-read object; and the block device driver layer applies second physical memory for the to-be-pre-read object based on the pre-reading window information, the second physical memory being used to prepare a cache for the to-be-pre-read object; wherein the second idle period is a waiting period for the block device driver layer to send a physical memory call request of the to-be-read object to the file system abstraction layer and wait for DMA migration of the to-be-read object to be completed, wherein the physical memory call request is used to trigger an application of third physical memory corresponding to the to-be-read object to the file system abstraction layer, and after the application of the third physical memory is completed, the to-be-read object is DMA migrated to the third physical memory for caching.

[0064] The present application optimizes pre-reading based on the above background technology:

[0065] 1) In response to the current read request, the non-sensitive time period (i.e. the first idle period or the second idle period) in the common service process of the general block layer or the driver layer is effectively utilized, that is, the pre-reading window information is calculated according to the current read request range, and based on the calculated pre-reading window information, a large part of the time-consuming work of the next read IO request is performed in advance, greatly reducing the work of responding to the next read IO request.

[0066] The pre-reading optimization method of the present application can be agglomerated in a specific software level and does not depend on other components. That is, the present application can be implemented in the general block layer or the device driver layer alone, and does not conflict with other modules, even if there is pre-reading optimization in the upper layer (such as the file system abstraction layer), it will not conflict with it, and will also have an added effect. Macrokernel and microkernel architecture can be used, and the universality is strong, especially for microkernel architecture, which is more obvious.

[0067] The pre-reading optimization process of the present application belongs to the common service process part of the general block layer or the block device driver layer, and does not need to be partitioned and customized according to the characteristics of different types of file systems or block devices. Compared with the traditional mainstream pre-reading scheme, the present application has stronger universality.

[0068] 2) The received IO request is analyzed by an algorithm to determine the pre-reading window size, and in the non-sensitive time period of the current business process, the pre-reading window range is used to perform the work of reducing the time-consuming proportion of the next read IO operation in advance, thereby optimizing the time-consuming part from the overall software process.

[0069] 3) in the current response process, the non-sensitive time period in the common block layer or the common service process in the drive layer is effectively utilized, so that the next business process is well "hidden" in the premise of not introducing additional overhead, and most of the overhead in the data reading process is reduced. In addition, unlike the traditional pre-reading scheme, although the physical memory page is applied in advance, no disk data moving operation is performed, and only when the next read request hits the pre-reading window, the data moving operation is performed, thereby reducing the overhead introduced by invalid operations when the pre-reading window is not hit.

[0070] It should be noted that the execution subject of the present application can be a terminal device, and the terminal device has an operating system, which can be a microkernel architecture operating system or a macrokernel architecture operating system, and the present application does not make specific limitations. The following will take the microkernel architecture operating system as an example to illustrate the following embodiments.

[0071] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings and specific embodiments.

[0072] Please refer to Figure 1 , Figure 1 The flowchart provided by the data pre-reading method embodiment of the present application.

[0073] The data pre-reading method of the first embodiment is proposed, which is applied to an operating system, the operating system includes a common block layer, an IO scheduling layer and a block device drive layer, and the data pre-reading method includes steps S100-S200:

[0074] Step S100, in the first idle period of the common block layer, the pre-reading window information is determined based on the current read request through the common block layer, and the pre-reading window information is used to indicate the pre-reading request for the to-be-pre-read object.

[0075] In an embodiment, the data pre-reading method of the present application is applied to the microkernel of the operating system. In another embodiment, the data pre-reading method of the present application is applied to the macrokernel of the operating system.

[0076] Preferably, the data pre-reading method of the present application is applied to the microkernel of the operating system. As known by those skilled in the art, the microkernel architecture operating system is a design philosophy opposite to the traditional macrokernel architecture. Under the microkernel architecture, the hierarchical division of the file system to the physical storage device includes: file system abstraction layer, common block layer, IO scheduling layer, block device drive layer. These levels generally exist in the form of user state service.

[0077] Among them, the File System Abstraction Layer provides a unified interface for users to interact with the file system, such as creating, deleting, reading, and writing files. The File System Abstraction Layer abstracts the details of different file system types, so that the upper-layer application program does not need to care about the specific implementation of the underlying file system. The File System Abstraction Layer is also responsible for converting logical file operations into operations on index nodes and data blocks.

[0078] The Generic Block Layer is located below the File System Abstraction Layer, and its main responsibility is to receive read and write requests from the File System Abstraction Layer and process them, such as request merging, splitting, and converting into specific IO requests, and then sending these IO requests to the IO scheduling layer.

[0079] The IO scheduling layer (Input / Output Scheduler Layer) is also known as the disk scheduler, usually integrated in the Generic Block Layer, and is part of the Generic Block Layer, responsible for managing and optimizing the execution order of I / O requests to improve overall IO performance. The IO scheduling layer determines when and how to send IO requests from the Generic Block Layer to the Block Device Driver Layer based on specific algorithms to minimize seek time and delay and maximize disk throughput.

[0080] The Block Device Driver Layer directly interacts with physical storage devices, responsible for identifying different physical storage devices, managing their initialization, registration, and converting block I / O requests from the upper layer into commands that the block device can understand, and processing the results or error information returned by the block device. Among them, the block device is an abstract concept from the perspective of the operating system, referring to a storage device that can perform read and write operations on fixed-size data blocks. The operating system interacts with these block devices through the block device interface without worrying about the specific physical implementation of the block device. Block devices can be actual physical storage devices such as hard drives and solid-state drives, or virtualized block devices or network block devices.

[0081] It is not difficult to understand that a read request (Read Request) is an operation request initiated by the operating system or application to read data from a storage device (such as a disk). When an application needs to read data from a disk to memory, it will generate one or more read requests through an operating system call, each read request usually specifying the starting address (block number) and size (number of blocks) of the data block to be read.

[0082] It should be noted that in the present embodiment, the current read request refers to a read request currently being processed by the operating system. The pre-reading window refers to a continuous data region predicted and set by a specific algorithm based on the data range of the current read request, and the pre-reading window contains data blocks that are likely to be accessed and read in the near future in addition to the data blocks accessed and read by the current read request. The object to be pre-read refers to the data blocks in the data region determined by the pre-reading window. The pre-reading window information is information describing the specific parameters of the pre-reading window, including but not limited to the start address, end address, window size, etc. of the pre-reading window. The pre-reading window information is the basis for the operating system to allocate memory space for the object to be pre-read, and indicates the pre-reading request for the object to be pre-read.

[0083] In addition, it should be noted that in the present embodiment, the size and position of the pre-reading window and other information can be calculated by a specific algorithm based on the actual data range of the current read request, the historical access mode, the file type, the read mode and other factors. Based on the pre-reading window information, the operating system can complete the time-consuming physical memory preparation work in the next read request processing flow in advance during the idle period of processing the current read request, and pre-apply the physical memory resources required for the next read request, so as to reduce the delay in future actual reading.

[0084] It is worth mentioning that the first idle period in the present embodiment refers to the waiting period of the general block layer after the IO request corresponding to the current read request is sent to the IO scheduling layer. In a possible implementation, the read request completion information can be a signal or message fed back by the block device driver layer in the operating system to the general block layer, indicating that the IO request corresponding to the current read request has been successfully completed.

[0085] In step S200, the block device driver layer applies the first physical memory for the object to be pre-read based on the pre-reading window information, and the first physical memory is used for cache preparation for the object to be pre-read.

[0086] The first idle period is a waiting period of the general block layer after the IO request corresponding to the current read request is sent to the IO scheduling layer.

[0087] In the present embodiment, the first physical memory is the physical memory space allocated by the operating system through the block device driver layer based on the pre-reading window information determined in the foregoing.

[0088] It is understandable that the embodiment pre-applies the first physical memory, and the scheme for preparing cache for the to-be-pre-read object can directly move the data to be read to the pre-applied first physical memory without waiting for the allocation of the physical memory when the to-be-pre-read object is really requested to read by the application program or the system, thereby avoiding the data reading lag caused by the delay of the allocation of the physical memory, further shortening the total delay time of data reading, and improving the response speed and operation efficiency of the system.

[0089] In the first idle period of the general block layer, the general block layer determines the pre-read window information based on the current read request, and the block device driver layer applies the first physical memory for the to-be-pre-read object based on the pre-read window information, so as to effectively utilize the non-sensitive time period (i.e. the first idle period) in the general block layer public service process, calculate the pre-read window information according to the data range of the current read request, and perform the part of work with a large time-consuming proportion in the next read request in advance based on the calculated pre-read window information, thereby greatly reducing the workload in responding to the next read request without introducing additional overhead, and optimizing the part with a high time consumption from the overall software process.

[0090] In addition, unlike the traditional mainstream pre-read scheme, although the embodiment pre-applies the physical memory page (i.e. the first physical memory), no disk data moving operation is performed, and the data moving operation is performed only when the next read request hits the pre-read window, thereby reducing the additional overhead introduced by the invalid operation when the pre-read window is not hit.

[0091] It is worth mentioning that in a feasible implementation manner, after step S100, the method can further include steps A10-A20.

[0092] Step A10: converting, by the general block layer, the pre-read window information into an IO request corresponding to the pre-read window information.

[0093] It should be noted that in the present embodiment, after the general block layer completes the determination of the pre-read window information, it will convert the information into an actual IO request format. This means that the pre-read window information (such as the start address, length, etc.) is encapsulated into specific instructions that can be understood and executed by the IO scheduling layer and the block device driver layer, i.e. the IO request.

[0094] Step A20: scheduling and processing, by the IO scheduling layer, the IO request corresponding to the pre-read window information, and then delivering the IO request to the block device driver layer.

[0095] It is understandable that the converted pre-read window information corresponding to the IO request is then transmitted to the IO scheduling layer.

[0096] In the embodiment, the IO scheduling layer schedules the IO request corresponding to the pre-reading window information according to a preset strategy, considers current system load, priority of the request, disk seek time and other factors, optimizes execution order of all IO operations, maximizes overall IO throughput, and minimizes delay of data reading.

[0097] The embodiment converts the pre-reading window information into specific IO requests through step A10, makes the physical memory pre-application operation of the to-be-pre-read object more flexible, can better adapt to specific needs in different application scenarios, improves the pertinence and accuracy of the physical memory pre-application operation, and through step A20, introduces intelligent scheduling of the IO scheduling layer, so that the operating system can dynamically adjust the execution order of the IO request according to current IO load, avoids disordered or inefficient IO operations, and improves overall IO throughput and response speed.

[0098] Step S200 can further include step S210.

[0099] In step S210, the block device driver layer applies the first physical memory for the to-be-pre-read object based on the IO request corresponding to the pre-reading window information.

[0100] In the embodiment, after the scheduling processing of the IO scheduling layer, the IO request corresponding to the pre-reading window information is issued to the block device driver layer, at this time, the block device driver layer executes the actual physical memory pre-application operation according to the scheduling result, and applies the corresponding physical memory, that is, the first physical memory, for the to-be-pre-read object.

[0101] The embodiment refines step S200 on the basis of step A10 and step A20, and clearly indicates that the application of the first physical memory is based on the IO request after the scheduling processing of the IO scheduling layer, so as to ensure that the physical memory allocation is tightly coupled with the actual physical memory pre-application operation, that is, the operation of applying the first physical memory for the to-be-pre-read object, avoids unnecessary memory occupation, improves the efficiency of memory use, and improves the instant availability of the pre-applied physical memory, that is, the first physical memory.

[0102] Based on the first embodiment, a data pre-reading method of the second embodiment of the application is provided.

[0103] In the second embodiment of the application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be described in detail.

[0104] In the embodiment, after the block device driver layer applies the first physical memory for the to-be-pre-read object based on the pre-reading window information, the method can further include step S300.

[0105] Step S300, in the case that, based on the new read request, it is determined that there is a first data object of hit part and a second data object of non-hit part in the to-be-pre-read object, the DMA moves the first data object to a first target memory in the first physical memory for caching, and releases a second target memory in the first physical memory.

[0106] The first target memory is a memory applied based on the first data object, and the second target memory is a memory applied based on the second data object.

[0107] It is not difficult to understand that, after a read request is processed by the operating system, the next read request to be processed by the operating system is the new read request. The read request generally includes storage location information (such as a start address, an end address, a data amount, etc.) of a data object to be read, so as to accurately read the required data from the actually stored physical space.

[0108] As known by those skilled in the art, DMA is a data transmission mode in a computer system, which allows external devices (such as hard drives, network cards, etc.) to directly exchange data with system memory without the continuous intervention of a CPU (Central Processing Unit). The DMA technology can perform data transmission while the CPU performs other tasks, thereby improving the data transmission speed and reducing the burden of the CPU.

[0109] It should be noted that, in the embodiment, the hit part in the to-be-pre-read object is taken as the first data object, and the non-hit part in the to-be-pre-read object is taken as the second data object.

[0110] In the embodiment, the operating system first calculates a data range corresponding to the new read request according to the new read request, so as to accurately identify which data in the to-be-pre-read object corresponding to the pre-read window is required to be read by the new read request (i.e., the first data object) and which data is not involved in the new read request (i.e., the second data object). This accurate judgment avoids resource waste caused by blind pre-reading and ensures the pertinence and effectiveness of the read operation.

[0111] In the embodiment, when the operating system determines that the first data object and the second data object exist in the pre-reading object based on the new read request, the first data object is directly moved to the first target memory in the first physical memory for caching by the DMA technology, and the CPU is bypassed by the DMA operation to directly transfer data between the memory and the I / O device, thereby significantly reducing the data transmission delay and accelerating the data reading speed. Meanwhile, for the second data object that does not hit, the second target memory in the first physical memory is released to recycle the memory space that is no longer needed, thereby improving the memory utilization and avoiding the memory fragmentation, maintaining the continuity of the memory space, reserving sufficient and effective resources for subsequent data caching, and further improving the system response speed and overall performance.

[0112] Exemplarily, when the pre-reading object is the data block between the disk addresses 001 and 100, the first physical memory pre-applied and allocated is the memory address 801 to 900, and the data to be read by the new read request is the data block between the disk addresses 051 and 150, the data block between the disk addresses 051 and 100 is the first data object, the first target memory is the region between the memory addresses 851 and 900 in the first physical memory, and the data block between the disk addresses 001 and 050 is the second data object, and the second target memory is the region between the memory addresses 801 and 850 in the first physical memory. At this time, the first data object between the disk addresses 051 and 100 is directly moved to the first target memory between the memory addresses 851 and 900 in the first physical memory by the DMA, and the second target memory between the memory addresses 801 and 850 in the first physical memory is released.

[0113] Further, in a feasible implementation, after the block device driver layer applies the first physical memory for the pre-reading object based on the pre-reading window information, the method can further include steps S400-S500:

[0114] Step S400, when it is determined that the pre-reading object hits based on the new read request, the pre-reading object is moved to the first physical memory for caching by the DMA.

[0115] Step S500, or, when it is determined that the pre-reading object does not hit based on the new read request, the first physical memory is released.

[0116] In the embodiment, if the operating system determines that the pre-reading object hits based on the new read request, the pre-reading object is directly moved to the first physical memory for caching by the DMA technology, and if the operating system determines that the pre-reading object does not hit based on the new read request, the first physical memory is released.

[0117] For example, when the to-be-pre-read object is data blocks between disk addresses 001 and 100, and the first physical memory pre-applied for allocation is memory addresses 801 to 900, if the data to be read by the new read request is data blocks between disk addresses 001 and 100, the to-be-pre-read object between disk addresses 001 and 100 is directly moved to the first physical memory between memory addresses 801 and 900 by DMA, and if the data to be read by the new read request is data blocks between disk addresses 101 and 200, the first physical memory between memory addresses 801 and 900 is released.

[0118] On the basis of step S300, in the case that the to-be-pre-read object is all hit, the to-be-pre-read object is moved to the first physical memory by DMA once through step S400, so that the pre-applied physical memory space (i.e., the first physical memory) is fully utilized, the data transmission time is reduced, and the overall efficiency of data reading is improved. In the case that the to-be-pre-read object is all not hit, the first physical memory pre-applied for allocation for the to-be-pre-read object is immediately released through step S500, so that when the pre-reading prediction does not achieve the expected effect (i.e., the to-be-pre-read object is all not hit), the memory resources are released in time, the invalid occupation of the memory resources is prevented, the memory resources can be reallocated to other more urgent tasks, and the flexibility and efficiency of memory resource use are improved.

[0119] It is worth mentioning that, in a feasible implementation, the pre-reading window information is a fixed preset data amount for the pre-reading data amount of the to-be-pre-read object.

[0120] It should be noted that the pre-reading data amount is an estimated data amount required for reading by the next read request when the operating system performs pre-reading preparation in an idle period of processing the current read operation.

[0121] In the embodiment, the pre-reading data amount is a data amount size pre-set and fixed by the operating system, i.e., a fixed preset data amount. The size of the pre-reading data amount is matched with the size of the first physical memory pre-applied for the to-be-pre-read object in the subsequent steps.

[0122] For example, if the pre-reading window information is a fixed preset data amount 1000 megabytes for the pre-reading data amount of the to-be-pre-read object, the size of the first physical memory pre-applied for the to-be-pre-read object based on the pre-reading window information is 1000 megabytes.

[0123] The embodiment sets the pre-reading data amount of the pre-reading object in the pre-reading window information to a fixed preset data amount, simplifies the complexity of the pre-reading strategy, makes the operating system not need a complex dynamic adjustment algorithm, effectively reduces the workload of system design and maintenance, and further improves the stability of the system and reduces the development cost. At the same time, the preset fixed pre-reading data amount makes the allocation of physical memory and other system resources predictable, facilitates the operating system to better plan and manage resources, thereby completing the step of memory application in the pre-reading operation within a deterministic time, avoiding the calculation overhead and the reaction delay that may be caused by the dynamic real-time calculation of the pre-reading data amount of each read request, and effectively improving the speed of data reading in subsequent operations.

[0124] In another possible implementation, the pre-reading data amount of the pre-reading object in the pre-reading window information is slidingly changed based on a pre-reading data hit rate of a last read request, and a maximum value of the pre-reading data amount after the sliding change is less than or equal to a preset threshold, where the pre-reading data hit rate refers to a ratio of the first data object in the hit part of the pre-reading object to the data amount of the pre-reading object.

[0125] It should be understood that the current read request refers to a read request being processed by the operating system, and the last read request refers to a read request processed by the operating system before the current read request.

[0126] It should be noted that in the embodiment, the pre-reading data hit rate of the last read request refers to a ratio of the first data object in the hit part of the pre-reading object of the last read request to the data amount of the pre-reading object of the last read request.

[0127] For example, the pre-reading object of the last read request is a data block between disk addresses 001 and 100, and the data to be read by the current read request is a data block between disk addresses 051 and 150, the hit part of the pre-reading object of the last read request is a data block between disk addresses 051 and 100, that is, the first data object in the hit part of the pre-reading object of the last read request is a data block between disk addresses 051 and 100. At this time, the ratio of the first data object in the hit part of the pre-reading object of the last read request to the data amount of the pre-reading object of the last read request can be calculated as 50%.

[0128] It is worth mentioning that in the embodiment, the pre-reading data amount of the pre-reading object in the pre-reading window information is slidingly changed based on the pre-reading data hit rate of the last read request, and a maximum value of the pre-reading data amount after the sliding change is less than or equal to a preset threshold. The preset threshold can be set by the user according to actual needs.

[0129] For the convenience of understanding the inventive concept of the embodiment, in an example, the pre-read window information can be a product of a pre-read data hit rate of a last read request and a preset threshold, for a pre-read data amount of an object to be pre-read.

[0130] In another example, the pre-read window information can also be a product of a pre-read data hit rate of a last read request and a data amount actually read by a last read request multiplied by two, for a pre-read data amount of an object to be pre-read. It is not difficult to understand that when the product of the pre-read data hit rate of the last read request and the data amount actually read by the last read request multiplied by two is greater than the preset threshold, the pre-read window information is equal to the preset threshold for the pre-read data amount of the object to be pre-read.

[0131] The embodiment is a dynamic pre-read strategy. The pre-read window information is not fixed for a pre-read data amount of an object to be pre-read, but is adjusted according to a pre-read data hit rate of a last read request. If the pre-read data hit rate of the last read request is high, it indicates that the pre-read strategy is relatively accurate, and the system will increase the pre-read data amount at the next read request; otherwise, the pre-read data amount is reduced. Meanwhile, the embodiment also sets a maximum value of the pre-read data amount not to exceed a certain preset threshold, to prevent a large amount of resources from being occupied due to a large amount of physical memory applied in an extreme case. Compared with a way in which the pre-read window information is fixed for a preset data amount of a pre-read data amount of an object to be pre-read, the embodiment is more intelligent and efficient, and can adaptively adjust the physical memory applied according to an actual use condition, to find a more optimal balance point between resource utilization and performance improvement.

[0132] The application provides a data pre-reading method in a third embodiment. In the third embodiment, the same or similar contents as those in the above embodiments can be referred to the above description, and will not be described hereinafter.

[0133] Please refer to Figure 2 , Figure 2 The application provides a flowchart of the data pre-reading method in the third embodiment.

[0134] In the embodiment, the method is applied to an operating system, the operating system includes a general block layer, an IO scheduling layer and a block device driving layer, and the method includes steps S10-S20.

[0135] In step S10, in a second idle period of the block device driving layer, the block device driving layer determines pre-read window information based on an object to be read of a current IO request, the pre-read window information is used to indicate a pre-read request for an object to be pre-read.

[0136] It is worth mentioning that, in the embodiment, the second idle period is for the block device driver layer to send a physical memory calling request of the to-be-read object to the file system abstraction layer, and wait for a waiting period for completion of DMA migration of the to-be-read object, wherein the physical memory calling request is used to trigger application of calling a third physical memory corresponding to the to-be-read object to the file system abstraction layer, and after completion of the third physical memory application, DMA migration of the to-be-read object to the third physical memory for caching.

[0137] It is not difficult to understand that the current IO request in the embodiment refers to an IO request being processed by the block device driver layer, which is used to read data from the external storage device, and generally includes position information (such as a logical block address) of data to be accessed and data amount information. The to-be-read object refers to a specific data block or data set pointed to by the current IO request, that is, data to be read by the current IO request from the external storage device.

[0138] In the embodiment, the pre-reading window refers to a continuous data region predicted and set by a specific algorithm based on a data range of the to-be-read object of the current IO request, and the pre-reading window includes data blocks to be accessed and read by the current IO request (that is, the to-be-read object of the current IO request) in addition to data blocks to be accessed and read in the near future as determined by the operating system, that is, the to-be-pre-read object.

[0139] In addition, it should be noted that, in the embodiment, the size and position of the pre-reading window and other information can be obtained by a specific algorithm based on a plurality of factors such as an actual data range of the to-be-read object of the current IO request, a historical access mode, a file type, a reading mode, and the like. Based on the pre-reading window information, the operating system can complete time-consuming physical memory preparation work in advance in the idle period (that is, the second idle period) of the block device driver layer waiting for completion of processing of the current IO request in the next read request processing flow, and pre-apply physical memory resources required for the next read request, so as to reduce delay in future actual reading.

[0140] In step S20, the block device driver layer applies a second physical memory for the to-be-pre-read object based on the pre-reading window information, and the second physical memory is used for caching preparation of the to-be-pre-read object.

[0141] In the embodiment, the second physical memory is a physical memory space allocated by the operating system through the block device driver layer based on the pre-reading window information determined in the foregoing.

[0142] It can be understood that the second physical memory is pre-applied, and the scheme of preparing cache for the to-be-pre-read object can directly move the data to be read to the pre-applied second physical memory without waiting for the allocation of the physical memory when the to-be-pre-read object is really requested to read by the application program or the system, thereby avoiding the data reading lag caused by the delay of the allocation of the physical memory, further shortening the total delay time of data reading, and improving the response speed and operation efficiency of the system.

[0143] In the second idle period of the block device driver layer, the block device driver layer determines the pre-reading window information based on the to-be-read object of the current IO request, and the block device driver layer applies the second physical memory for the to-be-pre-read object based on the pre-reading window information, the non-sensitive time period (that is, the second idle period) in the common service process of the block device driver layer is effectively utilized, the pre-reading window information is calculated according to the actual data range of the to-be-read object of the current IO request, and the time-consuming part of the next read request is performed in advance based on the calculated pre-reading window information, thereby greatly reducing the workload of responding to the next read request, without introducing additional overhead, and optimizing the time-consuming part from the overall software process.

[0144] In addition, unlike the traditional mainstream pre-reading scheme, although the embodiment pre-applies the physical memory page (that is, the second physical memory), no disk data moving operation is performed, and the data moving operation is performed only when the next read request hits the pre-reading window, thereby reducing the additional overhead introduced by the invalid operation when the pre-reading window is not hit.

[0145] Based on the third embodiment, the data pre-reading method of the fourth embodiment of the application is proposed.

[0146] In the fourth embodiment of the application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be described in detail.

[0147] In the embodiment, after the block device driver layer applies the second physical memory for the to-be-pre-read object based on the pre-reading window information, the method can further include step S30:

[0148] Step S30, in the case that the third data object of the hit part and the fourth data object of the non-hit part exist in the to-be-pre-read object based on the new read request, the third data object is moved to the third target memory in the second physical memory for caching by DMA, and the fourth target memory in the second physical memory is released.

[0149] The third target memory is the memory applied based on the third data object, and the fourth target memory is the memory applied based on the fourth data object.

[0150] It is understandable that after a read request is processed by the operating system, the next read request to be processed by the operating system is a new read request. The read request generally includes storage location information (such as a start address, an end address, a data amount, etc.) of a data object to be read, so as to accurately read the required data from the actually stored physical space.

[0151] As known by those skilled in the art, DMA (Direct Memory Access) is a data transmission mode in a computer system, which allows external devices (such as hard drives, network cards, etc.) to directly exchange data with system memory without the continuous intervention of a CPU (Central Processing Unit). The DMA technology can perform data transmission while the CPU performs other tasks, thereby improving data transmission speed and reducing the burden on the CPU.

[0152] It should be noted that in this embodiment, the hit part in the pre-read object is taken as the third data object, and the non-hit part in the pre-read object is taken as the fourth data object.

[0153] In this embodiment, the operating system first calculates the data range corresponding to the new read request according to the new read request, so as to accurately identify which data in the pre-read object corresponding to the pre-read window is required to be read by the new read request (i.e., the third data object) and which data is not involved in the new read request (i.e., the fourth data object). This accurate judgment avoids resource waste caused by blind pre-reading and ensures the pertinence and effectiveness of the read operation.

[0154] In this embodiment, when the operating system determines that the third data object and the fourth data object exist in the pre-read object based on the new read request, the third data object is directly moved to the third target memory in the second physical memory for caching by the DMA technology for the hit third data object. By using the feature of DMA operation that bypasses the CPU to directly transmit data between the memory and the I / O device, the data transmission delay is significantly reduced, and the data read speed is accelerated. At the same time, for the non-hit fourth data object, the fourth target memory in the second physical memory is released, and the memory space that is no longer needed is recycled in time, thereby improving the memory utilization rate, avoiding memory fragmentation, maintaining the continuity of the memory space, reserving sufficient and effective resources for subsequent data caching, and further improving the system response speed and overall performance.

[0155] For example, when the to-be-pre-read object is a data block between disk addresses 001 and 100, the pre-allocated second physical memory is memory addresses 801 to 900, if the data to be read by the new read request is a data block between disk addresses 051 and 150, the data block between disk addresses 051 and 100 is a third data object, the third target memory is a region between memory addresses 851 and 900 in the third physical memory, the data block between disk addresses 001 and 050 is a fourth data object, and the fourth target memory is a region between memory addresses 801 and 850 in the second physical memory. At this time, the third data object between disk addresses 051 and 100 is directly moved to the third target memory between memory addresses 851 and 900 in the second physical memory by DMA, and the fourth target memory between memory addresses 801 and 850 in the second physical memory is released.

[0156] Further, in a possible implementation, after the block device driver layer applies the second physical memory for the to-be-pre-read object based on the pre-read window information, the method can further include steps S40-S50:

[0157] Step S40, in a case where the to-be-pre-read object is determined to be all hit based on the new read request, the to-be-pre-read object is moved to the second physical memory for caching by DMA.

[0158] Step S50, or, in a case where the to-be-pre-read object is determined to be all missed based on the new read request, the second physical memory is released in total.

[0159] In the embodiment, if the operating system determines that the to-be-pre-read object is all hit based on the new read request, the to-be-pre-read object is directly moved to the second physical memory for caching by DMA, and if the operating system determines that the to-be-pre-read object is all missed based on the new read request, the second physical memory is released in total.

[0160] For example, when the to-be-pre-read object is a data block between disk addresses 001 and 100, the pre-allocated second physical memory is memory addresses 801 to 900, if the data to be read by the new read request is a data block between disk addresses 051 and 150, the data block between disk addresses 051 and 100 is a third data object, the third target memory is a region between memory addresses 851 and 900 in the third physical memory, the data block between disk addresses 001 and 050 is a fourth data object, and the fourth target memory is a region between memory addresses 801 and 850 in the second physical memory. At this time, the third data object between disk addresses 051 and 100 is directly moved to the third target memory between memory addresses 851 and 900 in the second physical memory by DMA, and the fourth target memory between memory addresses 801 and 850 in the second physical memory is released.

[0161] On the basis of step S30, the embodiment, through step S40, in the case that the to-be-pre-read objects are all hit, uses DMA to move the to-be-pre-read objects to the second physical memory at one time, so as to fully utilize the pre-applied physical memory space (that is, the second physical memory), reduce the data transmission time, and improve the overall efficiency of data reading. Through step S50, in the case that the to-be-pre-read objects are all not hit, the second physical memory previously allocated for the to-be-pre-read objects is immediately released, so that when the pre-reading prediction does not achieve the expected effect (that is, the to-be-pre-read objects are all not hit), the memory resources are timely released, the invalid occupation of the memory resources is prevented, the memory resources can be reallocated to other more urgently needed tasks, and the flexibility and efficiency of the use of the memory resources are improved.

[0162] It is worth mentioning that, in a feasible embodiment, the pre-reading window information is a fixed preset data amount for the pre-reading data amount of the to-be-pre-read object.

[0163] It should be noted that the pre-reading data amount is an estimated data amount to be read by the next reading request when the operating system performs pre-reading preparation in an idle period of processing the current reading operation.

[0164] In the embodiment, the pre-reading data amount is a data amount size pre-set and fixed by the operating system, that is, a fixed preset data amount. The size of the pre-reading data amount matches the size of the first physical memory applied for the to-be-pre-read object in the subsequent steps.

[0165] For example, if the pre-reading window information is a fixed preset data amount 1000 megabytes for the pre-reading data amount of the to-be-pre-read object, the size of the second physical memory applied for the to-be-pre-read object based on the pre-reading window information is 1000 megabytes.

[0166] The embodiment sets the pre-reading data amount for the to-be-pre-read object in the pre-reading window information as a fixed preset data amount, simplifies the complexity of the pre-reading strategy, makes the operating system not need a complex dynamic adjustment algorithm, effectively reduces the workload of system design and maintenance, and further improves the stability of the system and reduces the development cost. At the same time, the fixed pre-reading data amount makes the allocation of the physical memory and other system resources predictable, facilitates the operating system to better plan and manage resources, so that the step of memory application in the pre-reading operation is completed within a deterministic time, the calculation overhead possibly caused by dynamic real-time calculation of the pre-reading data amount for each reading request and the possible reaction delay are avoided, and the speed of data reading in the subsequent operation can be effectively improved.

[0167] In another possible implementation, the pre-read window information is based on a pre-read data hit rate of a last read request to slide the pre-read data amount of the to-be-pre-read object, and a maximum value of the pre-read data amount after the sliding is less than or equal to a preset threshold, where the pre-read data hit rate refers to a ratio of the third data object in the to-be-pre-read object hit part to the data amount of the to-be-pre-read object.

[0168] It should be understood that the current read request refers to a read request being processed by the operating system, and the last read request refers to a read request processed by the operating system before the current read request.

[0169] It should be understood that the pre-read data hit rate of the last read request refers to a ratio of the third data object in the to-be-pre-read object hit part of the last read request to the data amount of the to-be-pre-read object of the last read request.

[0170] For example, the to-be-pre-read object of the last read request is a data block between disk addresses 001 and 100, and the data to be read by the current read request is a data block between disk addresses 051 and 150, and the to-be-pre-read object hit part of the last read request is a data block between disk addresses 051 and 100, that is, the third data object in the to-be-pre-read object hit part of the last read request is a data block between disk addresses 051 and 100. At this time, the ratio of the third data object in the to-be-pre-read object hit part of the last read request to the data amount of the to-be-pre-read object of the last read request can be calculated as 50%.

[0171] It should be understood that in the embodiment, the pre-read data amount of the to-be-pre-read object in the pre-read window information is based on the pre-read data hit rate of the last read request to slide, and a maximum value of the pre-read data amount after the sliding is less than or equal to a preset threshold. The preset threshold can be set by the user according to actual needs.

[0172] In order to facilitate understanding of the inventive concept of the embodiment, in an example, the pre-read data amount of the to-be-pre-read object in the pre-read window information can be a product of the pre-read data hit rate of the last read request and a preset threshold.

[0173] In another example, the pre-read data amount of the to-be-pre-read object in the pre-read window information can also be a product of the pre-read data hit rate of the last read request and the data amount actually read by the last read request multiplied by two. It should be understood that when the product of the pre-read data hit rate of the last read request and the data amount actually read by the last read request multiplied by two is greater than the preset threshold, the pre-read data amount of the to-be-pre-read object in the pre-read window information is equal to the preset threshold.

[0174] The embodiment is a dynamic pre-reading strategy. The pre-reading window information is not fixed for the pre-reading data amount of the to-be-pre-read object, but is adjusted according to the pre-reading data hit rate of the last read request. If the pre-reading data hit rate of the last read request is high, it means that the pre-reading strategy is relatively accurate, and the system will increase the pre-reading data amount at the next read request; otherwise, the pre-reading data amount is reduced. At the same time, the embodiment also sets a maximum value of the pre-reading data amount, which does not exceed a certain preset threshold, to prevent occupying more resources due to excessive physical memory application in extreme cases. Compared with the way in which the pre-reading window information is fixed for the pre-reading data amount of the to-be-pre-read object, the pre-reading data amount is a preset data amount, the embodiment is more intelligent and efficient, and can adaptively adjust the pre-applied physical memory according to the actual use, and find a better balance point between resource utilization and performance improvement.

[0175] In order to facilitate understanding of the technical concept or technical principle of the above-mentioned embodiments of the data pre-reading method of the present application, specific embodiment one and specific embodiment two are listed.

[0176] As shown in Figure 3 In the design of modern operating systems, the hierarchical structure of the file system to the storage device is generally a file system abstraction layer, a general block layer, an IO scheduling layer, a block device driver layer, and a physical storage device. This hierarchical structure can provide conversion from file read-write operations to access to physical storage devices, that is, for the user's read-write disk file behavior, the operating system will be divided according to the software level, and the process of accessing the block device by the application program read-write file operation will be gradually performed.

[0177] Among them, the general block layer is mainly used to capture the read-write request issued by the file system abstraction layer, and perform request merging, splitting and other operations through algorithms, so that the read-write operation is converted into an IO request and issued to the IO scheduling layer. The IO scheduling layer buffers and schedules IO requests through specific algorithms, and then issues them to the block device driver layer. The block device driver layer is responsible for identifying different physical storage devices, initializing, registering devices, managing and processing IO requests, applying for resource allocation (such as memory allocation through operating system calls), accessing and processing physical storage devices (such as DMA moving disk data), and other work.

[0178] As shown in Figure 4 Under the microkernel architecture, the hierarchical division of the file system to the physical storage device is that the file system abstraction layer, the general block layer, the IO scheduling layer, and the block device driver layer all exist in the form of user state services, and are integrated into the kernel (Kernel), but are located in the user space (UserSpace). Under the microkernel architecture, there is a lot of inter-process communication between the upper and lower level service programs, and there is a lot of communication between the kernel and each level service program.

[0179] As shown in Figure 5As shown, in the monolithic kernel architecture, the file system to physical storage device hierarchy includes the file system abstraction layer, general block layer, IO scheduling layer, and block device driver layer, all located in the kernel. Similarly, there is a large amount of inter-process communication between the upper and lower level service programs.

[0180] like Figure 6 As shown, in the traditional sequential disk file reading process, when an application needs to read a file from the disk, the application generates a read request by calling the file system abstraction layer through the operating system. After receiving the read request, the general block layer server converts it into an I / O request and sends it to the scheduling module (i.e., the I / O scheduling layer). It then enters an idle period (the first idle period) waiting for the I / O request to complete until it receives a corresponding response. The scheduling module then schedules the I / O requests sent by the general block layer and sends the processed I / O requests to the block device driver layer. The block device driver layer server receives the I / O requests processed by the scheduling module, begins preparations such as allocating physical memory, configures DMA information, and triggers migration. Simultaneously, it enters an idle period (the second idle period) waiting for the DMA migration to complete until it receives a migration completion message from the DMA controller. Finally, the block device driver layer receives the transfer completion information from the DMA controller. After confirming the completion of the IO request, it updates the status of the IO request and notifies the general block layer that the IO request has been completed by calling the interface function provided by the general block layer. After receiving the information that the IO request has been completed, if there are still incomplete IO requests in the IO requests converted from this read request, the general block layer notifies the scheduling module to schedule the next IO request. If the current read request has been completed, it replies to the file system abstraction layer that the current read request has been completed. The file system abstraction layer processes the information and then reports back to the application that the current read operation has been completed.

[0181] Among them, the idle time period when the general block layer waits for the IO request to be completed until it receives the corresponding reply, and the idle time period when the block device driver layer waits for the DMA transfer to be completed until it receives the transfer completion information from the DMA controller are non-sensitive time periods. The remaining time periods are sensitive time periods (sensitive time periods refer to the business process periods during which necessary and effective work is being carried out for the current service process). The preparatory work such as requesting physical memory, which is the responsibility of the block device driver layer, is the most time-consuming part of the process.

[0182] like Figure 7 As shown, for the block device driver layer, specific embodiment one is as follows: Figure 6Based on the illustrated process, the non-sensitive time periods in the block device driver layer's business process are effectively utilized. After configuring and triggering DMA to move disk information, during the non-sensitive time period while waiting for the DMA controller to reply that the move is complete, a new step C10 is added to calculate the pre-read window based on the range of the current IO request. Steps C20 and C30 complete pre-read optimization operations. That is, during the second idle period of the block device driver layer, the block device driver layer determines the pre-read window information based on the object to be read in the current IO request, and then allocates second physical memory for the object to be pre-read based on the pre-read window information. Thus, without introducing additional overhead, the time-consuming part of the next IO request's business process is completed, speeding up the response to the next IO request.

[0183] like Figure 8 As shown, for the general block layer, specific embodiment two is as follows: Figure 6 Based on the illustrated process, the non-sensitive time periods in the general block layer business process are effectively utilized. After the current read IO request is sent to the scheduling layer, during the non-sensitive time period while waiting for the reply to complete the current IO request, a new step D10 is added to calculate the pre-read window, step D20 calculates the pre-read window based on the range of the current read request, step D30 converts the IO request and sends it to the scheduling module, step D40 schedules the IO request, step D50 the block device receives and processes the pre-read IO request, step D60 allocates physical memory through system calls, and step D70 completes the pre-read optimization operation. That is, during the first idle period of the general block layer, the general block layer determines the pre-read window information based on the current read request, and the block device driver layer allocates the first physical memory for the object to be pre-read based on the pre-read window information, thereby completing the time-consuming part of the next read IO request business process in advance (such as allocating physical memory through system calls).

[0184] Taking physical memory as an example, in both Specific Embodiment 1 and Specific Embodiment 2 above, when processing a read I / O request, the system first checks within the pre-read window whether the object to be read in the read I / O request overlaps with the object to be pre-read in the pre-read window. For the overlapping portion, there is no need to re-allocate physical memory; the corresponding portion of the physical memory allocated by the pre-read is used directly. For the non-overlapping portion, physical memory needs to be allocated for the non-overlapping portion of the object to be read, and the corresponding portion of the physical memory allocated by the pre-read needs to be released for the non-overlapping portion of the object to be pre-read. That is, based on a new read request, if it is determined that there is a first data object with a hit portion and a second data object with a miss portion in the object to be pre-read, the DMA moves the first data object to the first target memory in the first physical memory for caching and releases the second target memory in the first physical memory.

[0185] like Figure 9As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole.

[0186] As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole. Figure 10 to Figure 11 As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole.

[0187] As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole. Figure 12 As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole.

[0188] As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole. Figure 13 As shown, for the physical memory of the pre-reading application, if it is found that the to-be-read object does not overlap with the to-be-pre-read object, the physical memory of the pre-reading application is released in whole, that is, in the case that it is determined that the to-be-pre-read object is all missed based on the new read request, the first physical memory is released in whole.

[0189] In the embodiment, the pre-reading window can be extended in steps according to the hit condition. For example, if the first read request is for a length of L and the pre-reading window is also of the same length L, when the second read request hits, the length of the pre-reading window is extended to 2L, and when the third read request misses, the length of the pre-reading window is reduced to L, and so on until the length of the pre-reading window is extended to a certain upper limit.

[0190] It can be understood that, in addition to the specific embodiment one and the specific embodiment two, the idle time of the general block layer or the block device driver layer can also be used together to calculate the pre-reading window and to perform the part of the next read IO request business process in advance. In addition, the same technical idea can also be used to move the above-mentioned optimization process to other levels, for example, the same technical idea can be used in the file system abstraction layer to extract the non-sensitive time period of the file system abstraction layer to perform the part of the next read request business process.

[0191] In addition to the specific embodiment one and the specific embodiment two in the sequential read scenario, the technical concept of the present application can also be applied to the random read, mixed random read-write, mixed sequential read-write and other scenarios.

[0192] It should be noted that the specific embodiments are only used to assist understanding of the present application, and do not constitute limitation to the data pre-reading method of the present application, and more forms of simple transformation based on the technical concept are within the protection scope of the present application.

[0193] In addition, please refer to Figure 14 , Figure 14 The device structure diagram of the hardware running environment involved in the data pre-reading method in the embodiments of the present application.

[0194] The present application also provides an electronic device, which comprises at least one processor, and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data pre-reading method in the above embodiments.

[0195] Reference will be made to Figure 14 , which shows a structure diagram of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle terminals (such as vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 14 The electronic device shown is only an example, and should not bring any limitation to the function and use range of the embodiments of the present application.

[0196] As Figure 14As shown, the electronic device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded into a random access memory (RAM) 1004 from a storage apparatus 1003. Various programs and data required for operation of the electronic device are also stored in the RAM 1004. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input apparatus 1007 including, for example, a touch panel, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output apparatus 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the electronic device to perform wireless or wired communication with other devices to exchange data. Although the electronic device having various systems is shown in the figure, it should be understood that all of the illustrated systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0197] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0198] The electronic device provided by the present disclosure adopts the data pre-reading method in the above embodiments, and can solve at least one technical problem mentioned in the background art. Compared with the prior art, the electronic device provided by the present disclosure has the same beneficial effects as the data pre-reading method provided by the above embodiments, and other technical features in the electronic device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0199] It should be understood that various parts of the present application can be realized with hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0200] The above description is merely that of specific embodiments of the present application, but the scope of the present application is not limited thereto. Any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present application, and all such changes and substitutions should be encompassed within the scope of the present application. Therefore, the scope of the present application should be decided based on the scope of the claims.

[0201] In addition, the present application also provides a storage medium having stored thereon computer readable program instructions (i.e., computer programs) for performing the data pre-reading method in the above-described embodiments.

[0202] The storage medium provided by the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, system, or device. The program code contained on the storage medium can be transmitted in any suitable medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency), or the like, or any suitable combination thereof.

[0203] The above-described storage medium can be included in an electronic device; or can exist separately without being assembled into an electronic device.

[0204] The storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: in a first idle period of the general block layer, determine, by the general block layer based on a current read request, pre-read window information, the pre-read window information being used to indicate a pre-read request for a to-be-pre-read object; and apply, by the block device driver layer based on the pre-read window information, first physical memory for the to-be-pre-read object, the first physical memory being used for cache preparation for the to-be-pre-read object. The first idle period is a waiting period for the general block layer to wait for receiving read request completion information after the general block layer delivers an IO request corresponding to the current read request to the IO scheduling layer.

[0205] Or, cause the electronic device to: in a second idle period of the block device driver layer, determine, by the block device driver layer based on a to-be-read object of a current IO request, pre-read window information, the pre-read window information being used to indicate a pre-read request for a to-be-pre-read object; and apply, by the block device driver layer based on the pre-read window information, second physical memory for the to-be-pre-read object, the second physical memory being used for cache preparation for the to-be-pre-read object. The second idle period is a waiting period for the block device driver layer to wait for DMA migration completion of the to-be-read object after the block device driver layer sends a physical memory call request of the to-be-read object to the file system abstraction layer. The physical memory call request is used to trigger the file system abstraction layer to apply for calling third physical memory corresponding to the to-be-read object, and DMA migrate the to-be-read object to the third physical memory for caching after the third physical memory application is completed.

[0206] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0207] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0208] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the names of the modules do not limit the modules themselves.

[0209] The storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned data pre-reading method, and can solve at least one technical problem mentioned in the background art. Compared with the prior art, the storage medium provided by the present application has the same beneficial effects as the data pre-reading method provided by the above-mentioned embodiments, which will not be repeated here.

[0210] In addition, the embodiments of the present application also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the data pre-reading method in the above-mentioned embodiments is realized.

[0211] The computer program product provided by the present application can solve at least one technical problem mentioned in the background art. Compared with the prior art, the computer program product provided by the embodiments of the present application has the same beneficial effects as the data pre-reading method provided by the above-mentioned embodiments, which will not be repeated here.

[0212] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A data prefetching method, characterized in that, The method is applied to an operating system, which includes a general block layer, an I / O scheduling layer, and a block device driver layer. The method includes: During the first idle period of the general block layer, the general block layer determines pre-read window information based on the current read request. The pre-read window information is used to indicate the pre-read request for the object to be pre-read. The block device driver layer requests first physical memory for the object to be pre-read based on the pre-read window information, and the first physical memory is used to prepare a cache for the object to be pre-read. The first idle period is the waiting period after the general block layer sends the IO request corresponding to the current read request to the IO scheduling layer and waits to receive the read request completion information.

2. The data prefetching method as described in claim 1, characterized in that, After the block device driver layer requests first physical memory for the object to be pre-read based on the pre-read window information, the method includes: If, based on a new read request, it is determined that there is a first data object with a hit portion and a second data object with a miss portion in the object to be pre-read, the DMA moves the first data object to the first target memory in the first physical memory for caching, and releases the second target memory in the first physical memory. Wherein, the first target memory is memory allocated based on the first data object, and the second target memory is memory allocated based on the second data object.

3. The data prefetching method as described in claim 1, characterized in that, After the block device driver layer requests first physical memory for the object to be pre-read based on the pre-read window information, the method includes: If, based on a new read request, it is determined that all objects to be pre-read have been hit, the DMA moves the objects to be pre-read to the first physical memory for caching; or... If, based on a new read request, it is determined that all of the objects to be pre-read are missed, the first physical memory is released entirely.

4. The data prefetching method as described in any one of claims 1 to 3, characterized in that, The pre-read window information specifies a fixed preset data amount for the object to be pre-read; or, The pre-read window information adjusts the amount of pre-read data for the object to be pre-read based on the pre-read data hit rate of the previous read request, and the maximum value of the pre-read data after the adjustment is less than or equal to a preset threshold. The pre-read data hit rate refers to the ratio of the first data object of the pre-read object to the total amount of data of the object to be pre-read.

5. The data prefetching method as described in any one of claims 1 to 3, after determining the prefetch window information based on the current read request through the general block layer, includes: The general block layer converts the prefetch window information into an IO request corresponding to the prefetch window information. The IO scheduling layer processes the IO requests corresponding to the pre-read window information and then sends them to the block device driver layer. The step of requesting first physical memory for the object to be pre-read based on the pre-read window information through the block device driver layer includes: The block device driver layer requests the first physical memory for the object to be pre-read based on the IO request corresponding to the pre-read window information.

6. A data prefetching method, characterized in that, The method is applied to an operating system, which includes a general block layer, an I / O scheduling layer, and a block device driver layer. The method includes: During the second idle period of the block device driver layer, the block device driver layer determines the pre-read window information based on the object to be read in the current IO request. The pre-read window information is used to indicate the pre-read request for the object to be read. The block device driver layer requests a second physical memory for the object to be pre-read based on the pre-read window information, and the second physical memory is used to prepare a cache for the object to be pre-read. The second idle period is the waiting period during which the block device driver layer sends a physical memory call request for the object to be read to the file system abstraction layer and waits for the DMA migration of the object to be read to be completed. The physical memory call request is used to trigger the application to the file system abstraction layer to call the third physical memory corresponding to the object to be read, and after the application to the third physical memory is completed, the DMA migrates the object to be read to the third physical memory for caching.

7. The data prefetching method as described in claim 6, characterized in that, After the block device driver layer requests second physical memory for the object to be pre-read based on the pre-read window information, the method includes: If, based on a new read request, it is determined that there is a third data object with a hit portion and a fourth data object with a miss portion in the object to be pre-read, the DMA moves the third data object to the third target memory in the second physical memory for caching, and releases the fourth target memory in the second physical memory. Wherein, the third target memory is memory allocated based on the third data object, and the fourth target memory is memory allocated based on the fourth data object.

8. The data prefetching method as described in claim 6, characterized in that, After the block device driver layer requests second physical memory for the object to be pre-read based on the pre-read window information, the method includes: If, based on a new read request, it is determined that all objects to be pre-read have been hit, the DMA moves the objects to be pre-read to the second physical memory for caching; or... If, based on a new read request, it is determined that all objects to be pre-read are missed, the second physical memory is released entirely.

9. The data prefetching method as described in any one of claims 6 to 8, characterized in that, The pre-read window information specifies a fixed preset data amount for the object to be pre-read; or, The pre-read window information adjusts the amount of pre-read data for the object to be pre-read based on the pre-read data hit rate of the previous read request, and the maximum value of the pre-read data after the adjustment is less than or equal to a preset threshold. The pre-read data hit rate refers to the ratio of the third data object of the pre-read object to the amount of data of the object to be pre-read.

10. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data prefetching method as described in any one of claims 1 to 5, or 6 to 9.

11. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the data prefetching method as described in any one of claims 1 to 5, or 6 to 9.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the data prefetching method as described in any one of claims 1 to 5, or 6 to 9.

Images