Data access method, apparatus, device, and readable storage medium

Abstract

The application discloses a data access method and device, equipment and a readable storage medium. After a server receives a data access request, it is judged whether target data requested by the data access request exists in a cache or memory. If the target data does not exist in the memory and the cache, it is determined that a reading request is initiated to a hard disk. Then, the server judges a medium type of the hard disk storing the target data. When the medium type indicates that the hard disk is a solid state disk, a pre-reading behavior of the reading request is closed, and the reading request with the closed pre-reading behavior is initiated to the hard disk to acquire the target data. When the medium type indicates that the hard disk is a solid state disk, the pre-reading behavior is closed, so that the reading request is prevented from being blocked probabilistically due to garbage collection, the concurrent capability of the server is improved, the data access speed is improved, and the possibility of system jitter is reduced.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a data access method, apparatus, device, and readable storage medium. Background Technology

[0002] With the development of electronic information technology, hard drives have been widely used in storage systems due to their powerful storage capabilities. Common hard drives include mechanical hard drives, solid-state drives, and hybrid hard drives.

[0003] Normally, the central processing unit (CPU) is much faster than a hard drive, and the CPU cannot directly operate on the hard drive. Therefore, modern operating system designs incorporate memory and cache. Data is pre-fetched into memory to prevent the CPU from waiting due to the slow hard drive. When the CPU accesses target data, it first searches for it in the cache. If found, it is immediately read and sent to the CPU for processing. If the target data is not in the cache, it searches for it in memory. If the target data is also not in memory, it sends an input / output (I / O) request to the storage device to read the target data into memory, and then loads the target data from memory into the cache. In this way, the CPU can directly read the target data from the cache.

[0004] Most current data prefetching algorithms are applicable to hard disk drives (HDDs), whose random I / O capability is measured by input / output per second (IOPS), typically ranging from 60 to 120 times per second. With the development of storage media, solid-state drives (SSDs) and other devices with stronger random I / O capabilities have emerged, rendering existing data prefetching algorithms unsuitable. Summary of the Invention

[0005] This application provides a data access method, apparatus, device, and readable storage medium. It determines whether to enable pre-read behavior based on the media type of the hard disk and adopts different pre-read methods for different hard disks, which is highly adaptable and achieves the purpose of improving data access speed.

[0006] In a first aspect, embodiments of this application provide a data access method, including:

[0007] Receive data access requests;

[0008] If the target data requested by the data access request is not present in the cache or memory, a read request will be initiated to the hard disk.

[0009] Determine the media type of the hard disk;

[0010] Based on the media type, determine whether to enable the pre-read behavior of the read request and acquire the target data when acquiring the target data.

[0011] Secondly, embodiments of this application provide a data access device, including:

[0012] The transceiver module is used to receive data access requests;

[0013] The determination module is used to determine to initiate a read request to the hard disk when the target data requested by the data access request does not exist in the cache and memory;

[0014] A processing module is used to determine the media type of the hard disk;

[0015] The reading module is used to determine whether to enable the pre-read behavior of the reading request when acquiring the target data based on the media type.

[0016] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to implement the method described in the first aspect or various possible implementations of the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the method described in the first aspect or various possible implementations of the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product comprising a computing program, wherein when the computer program is executed by a processor, it implements the method described in the first aspect or various possible implementations of the first aspect.

[0019] The data access method, apparatus, device, and readable storage medium provided in this application embodiment allow the server to determine whether the target data requested by the data access request exists in the cache or memory after receiving a data access request. If the target data is not found in either memory or cache, a read request is initiated to the hard disk. Subsequently, the server determines the media type of the hard disk storing the target data and decides whether to enable pre-read behavior based on the media type. Different pre-read methods are used for different hard disks, providing strong adaptability and improving data access speed. Furthermore, when the media type indicates that the hard disk is a solid-state drive (SSD), pre-read behavior is disabled to avoid probabilistic blocking of read requests due to garbage collection, thereby improving the server's concurrency capabilities, increasing data access speed, and reducing the possibility of system jitter. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the network architecture of the data method provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the server in the data access method provided in the embodiments of this application;

[0023] Figure 3 This is a flowchart of the data access method provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the stripes in the data access method provided in the embodiments of this application;

[0025] Figure 5 This is another flowchart of the data method provided in the embodiments of this application;

[0026] Figure 6 A schematic diagram of a data access device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0029] Servers typically include hardware such as a Central Processing Unit (CPU), hard drive, and memory. Both hard drives and memory are storage media. Because hard drives are driven by mechanical components, the CPU's speed in reading data from the hard drive is much slower than its speed in reading data from memory. In other words, hard drive speed is significantly slower than memory speed. When the CPU performs data operations, if it directly reads or writes to the hard drive (also known as initiating an I / O request to the hard drive), the CPU will wait due to the slow hard drive speed, resulting in low CPU utilization. To address this, current operating system designs use prefetching to avoid CPU waiting. In prefetching, when a read / write operation to the hard drive is required, a prefetching algorithm reads the target data and a portion of the data before and after it (usually the portion after the target data) into memory in advance, and then loads it into the cache. This allows the CPU to directly retrieve the target data from the cache, avoiding CPU blocking due to waiting for hard drive operations and improving system performance.

[0030] Verification has revealed that existing prefetching algorithms have at least the following drawbacks:

[0031] Disadvantage 1: It is not applicable to solid-state drives (SSDs), which may cause read requests to SSDs to be blocked probabilistically, reducing the server's concurrency capabilities.

[0032] When the target data is not in the cache or memory, the operating system initiates an I / O request to the disk to read the data. This I / O request includes random I / O requests and sequential I / O requests. Random I / O requests occur when the target data for several consecutive I / O requests is scattered across different sectors of different pages on the disk, resulting in slow data reading. Sequential I / O requests occur when the target data for several consecutive I / O requests is adjacent to each other, resulting in fast data reading. Random I / O requests include both read and write requests.

[0033] The development and evolution of current read-ahead algorithms are largely based on hard disk drives (HDDs), which are characterized by poor random I / O capabilities but strong sequential I / O capabilities. However, with the development of storage media, solid-state drives (SSDs), due to their near-zero access latency and insensitivity to random access, have been widely used in high-performance computing environments. SSDs offer random I / O capabilities of tens of thousands to hundreds of thousands of operations per second, hundreds or even thousands of times faster than HDDs. Existing read-ahead algorithms do not consider the characteristics of SSDs. For example, when concurrent write I / O occurs on an SSD, internal hardware garbage collection (GC) can cause read requests to be blocked probabilistically. The larger the number of read requests, the greater the probability of read requests being blocked, leading to poor concurrency capabilities.

[0034] Disadvantage 2: Initially, if the read request is small, there is a "slow start" problem with the prefetch window, meaning that the prefetch window needs to be expanded multiple times to reach the ideal state. Here, "small read request" refers to a small amount of data being targeted by the read request; such read requests are also called small requests.

[0035] During read and write operations on a hard drive, the time spent reading and writing data mainly includes seek time T1, rotational latency T2, and transfer time T3. Seek time T1 refers to the time required for the read / write head to move to a specified track. The time required for the read / write head to move to the specified track varies depending on the offset between the track currently occupied by the read / write head and the specified track. The industry typically uses "average seek time" to represent T1, which is relatively fixed and generally around 6 milliseconds (ms).

[0036] Rotational latency T2 refers to the time it takes to move a sector under the read / write head, and it is closely related to the rotational speed of the hard drive. The rotational latency T2 of mainstream mechanical hard drives is approximately 3ms-6ms.

[0037] The transfer time T3 refers to the time it takes to read data from or write data to the hard drive, and it is directly proportional to the amount of data, i.e., T3 = data amount ÷ transfer speed. The transfer speed is approximately in the range of 100MB / s to 200MB / s.

[0038] Due to the aforementioned characteristics of hard drives, their ability to process read requests is significantly limited. Furthermore, this ability is also constrained by the size of the read requests. Currently, the random I / O capability of mechanical hard drives is typically between 60 and 120 times per second, meaning they can process 60-120 random I / O requests per second.

[0039] Based on prefetching, data needs to be read from the hard drive into memory in advance. This prefetched data is usually larger than the target data requested by the read request and includes the target data. This prefetched data is called prefetched data. The key to prefetching is timeliness and accuracy.

[0040] Timeliness refers to the requirement that prefetched data should be read from the hard drive into memory before the CPU accesses it. This means that more data is read each time, resulting in a larger amount of prefetched data. The larger the amount of prefetched data, the better the timeliness is guaranteed. However, the larger the amount of prefetched data, the larger the aforementioned transmission time T3. Moreover, due to memory size limitations, after each prefetch, the old prefetched data must be evicted from memory. If the prefetched data is evicted too early and cannot be accessed in time, the same data will be read multiple times, causing system performance fluctuations. Therefore, the operating system usually specifies an upper limit to restrict the amount of prefetched data.

[0041] Accuracy refers to the utilization of prefetched data. Higher utilization results in higher accuracy. Prefetch algorithms typically maintain historical information as a basis for prediction, dynamically adjusting the amount of prefetched data for the next IO request to avoid reading too much useless data. The size of the prefetched data is called the size of the prefetch window.

[0042] Taking a server running a Linux operating system as an example, in current read-ahead algorithms, when accessing a file, the server's operating system sends multiple read requests to the hard drive to load the file into memory. By default, the operating system determines the initial read-ahead window based on the size of the first read request (rqsize) and the maximum read request size supported by the hard drive (ramax). For example, when the read request is small (rqsize <= ramax / 32), the initial read-ahead window is 4 × rqsize. Similarly, when the read request is medium (rqsize <= ramax / 4), the initial read-ahead window is 2 × rqsize. And when the read request is large (rqsize > ramax / 4), the initial read-ahead window is ramax.

[0043] After the initial prefetch window is determined, the prefetch algorithm enters a dynamic window adjustment process. Ideally, the prefetch algorithm assumes that when the window expansion condition is met, the size of the prefetch window for the next read request will be 2 or 4 times the size of the current prefetch window. Therefore, based on the current prefetch algorithm, multiple window expansions are required to reach the maximum window size, i.e., ramax.

[0044] For example, if ramax = 512KB and the initial read request size is 4KB (meaning the initial read request is a small request), then the initial prefetch window is 4 × rqsize = 16KB. It takes 5 adjustments to reach ramax. During the adjustment process, the values ​​of the 5 prefetch windows are 16KB, 64KB, 128KB, 256KB, and 512KB respectively.

[0045] For example, when read requests are moderate, assuming the initial read request size is 32KB, the initial prefetch window size is 2 × rqsize = 64KB, requiring four adjustments to reach ramax. During the adjustment process, the values ​​of the four prefetch windows are 64KB, 128KB, 256KB, and 512KB respectively.

[0046] As can be seen from the above, when the read request is relatively small or the IO request is a medium-sized request, the existing prefetch algorithm is not very user-friendly, and the prefetch window needs to be adjusted several times to reach the ideal size.

[0047] In particular, when the aforementioned prefetch algorithm is applied to servers in cloud computing, multiple applications (tenants) share the server's CPU and storage capacity. The server cannot directly control the behavior of the applications, and applications may, due to poor design or even malicious intent, trigger the operating system to continuously send small read requests to the hard drive. The prefetch algorithm dynamically expands, shrinks, or maintains the prefetch window by detecting changes. This prefetch algorithm has broad adaptability, but its performance is not efficient in scenarios with relatively fixed business characteristics, especially when the file size is less than twice the ideal prefetch window size. For example, if a server reads a 1MB file through multiple read requests with a prefetch window of 64KB, then on the fourth prefetch, the prefetch window will be 512KB. When the prefetch window reaches the ideal size, the 1MB file is almost read from the hard drive into memory, making the prefetch algorithm inefficient. However, if the server reads a 1000MB file through multiple read requests, expanding the prefetch window from 64KB to 512KB can indeed improve efficiency. For example, in applications like databases where local randomness is high, if the application triggers the operating system to continuously send unreasonable small requests to the hard drive, it can easily exhaust the random I / O capabilities of the host machine using mechanical hard drives as storage devices. Even for host machines using solid-state drives as storage devices, as business volume increases, the concurrency of random requests becomes large, and existing prefetching algorithms for expanding or shrinking the prefetch window are insufficient to guarantee system stability.

[0048] To address this, this application provides a data access method. When the media type indicates the hard drive is a solid-state drive (SSD), prefetching is disabled to prevent read requests from being probabilistically blocked due to garbage collection, thereby improving server concurrency and data access speed. Simultaneously, when the hard drive is a mechanical hard drive (HDD), the prefetching algorithm is improved to efficiently handle small requests or small requests triggered by locally randomized business processes while maintaining sequential I / O request performance.

[0049] Figure 1 This is a schematic diagram of the network architecture of the data method provided in the embodiments of this application. Please refer to... Figure 1 The network architecture includes server 11 and terminal device 12.

[0050] Server 11 possesses significant computing and storage capabilities, enabling it to provide services to terminal devices. Server 11 can be either hardware or software. When server 11 is hardware, it can be a single server or a distributed server cluster composed of multiple servers. When server 11 is software, it can be multiple software modules or a single software module, etc., and this application embodiment is not limited thereto.

[0051] Server 11 is equipped with a CPU, cache, memory, and hard disk. When the CPU accesses target data, it first searches for the target data in the cache. If found, it is immediately read and sent to the CPU for processing. If the target data is not in the cache, it searches for it in memory. If the target data is also not in memory, a read request is sent to the storage device.

[0052] In this embodiment, when server 11 is used in a cloud computing scenario, server 11 acts as a host machine, providing the ability to share hardware through software technologies such as containers or virtual machines in a Linux system. For example, different tenants (applications) share CPU computing power or storage power, thereby achieving the purpose of resource sharing. Different servers 11 have significantly different random I / O capabilities due to differences in hard drive media types. Typically, cloud computing providers deploy services with similar characteristics onto the same server 11. For example, if a server 11 has a solid-state drive with strong random I / O capabilities, services that are prone to triggering random requests will be deployed on this server. Conversely, if a server 11 has a mechanical hard drive with poor random I / O capabilities, services that are less likely to trigger random requests will be deployed on this server.

[0053] Figure 2 This is a schematic diagram of the server in the data access method provided in this application embodiment. Please refer to... Figure 2 In the Linux operating system, a server has user space and kernel space. Applications in user space can be considered as different tenants or services deployed on the server. When a user on terminal device 12 needs to retrieve target data while browsing the internet, they send a data access request to server 11. After receiving the data access request, if the target data is not in the cache or memory, the user space application triggers the operating system to initiate a read request to the hard disk. When the hard disk is a solid-state drive (SSD), read-ahead behavior is disabled to avoid probabilistic blocking of read requests due to garbage collection, thus improving the server's concurrency capabilities. When the hard disk is a mechanical hard drive (HDD), the read-ahead algorithm is improved to efficiently handle small requests or small requests triggered by locally randomized business processes while maintaining sequential I / O request performance.

[0054] Terminal device 12 can be either hardware or software. When terminal device 12 is hardware, it can be, for example, a mobile phone, tablet computer, personal computer, e-book reader, laptop computer, or desktop computer running Android, Microsoft operating system, Symbian operating system, Linux operating system, or Apple iOS operating system. When terminal device 12 is software, it can be installed on the aforementioned hardware devices. In this case, terminal device 12 can be, for example, multiple software modules or a single software module, and this application embodiment is not limited to this.

[0055] It should be understood that, Figure 1 The number of servers 11 and terminal devices 12 shown is merely illustrative. In actual implementation, any number of servers 11 and terminal devices 12 can be deployed according to actual needs.

[0056] Below, based on Figure 1 The implementation environment shown is Figure 2 The server shown illustrates the data access method provided in this application embodiment. For examples, please refer to... Figure 3 . Figure 3 This is a flowchart of a data access method provided in an embodiment of this application. This embodiment is described from the perspective of a server and includes:

[0057] 301. Receive data access request.

[0058] like Figure 1 As shown, when a user accesses the internet using a terminal device, they send a data access request to the server. For example, if a user is browsing a short video, they send a data access request to the server to request the video data, which is the target data. Similarly, if a user is browsing online images, they send a data access request to the server to request the image data, which is the target data.

[0059] 302. When the target data requested by the data access request does not exist in the cache or memory, a read request is initiated to the hard disk.

[0060] After receiving a data access request, the server first checks if the requested data exists in the cache. If the data exists in the cache, it encodes and processes the data before sending it to the terminal device. If the data does not exist in the cache, it checks if the data exists in memory. If the data exists in memory, it is loaded into the cache, and the CPU reads the data from the cache, encodes and processes it before sending it to the terminal device.

[0061] If the target data is not in memory, the server will initiate a read request to the hard drive to read the target data.

[0062] 303. Determine the media type of the hard drive.

[0063] 304. Determine whether to enable the pre-read behavior of the read request and acquire the target data when acquiring the target data based on the media type.

[0064] For example, in steps 303 and 304, the server distinguishes between hard drives and uses different pre-read methods for different media types. Existing pre-read methods do not differentiate between hard drives based on media type; regardless of the type of hard drive, a portion of the data (pre-read data) is pre-read into memory using a pre-read algorithm.

[0065] In this embodiment, after determining the hard drive's media type, the server decides whether to enable or disable pre-read behavior based on the media type. If pre-read behavior is enabled, a portion of the data (pre-read data) is pre-read into memory using a pre-read algorithm. If pre-read behavior is not enabled, no pre-read is required.

[0066] After obtaining the target data, the server encodes and processes the target data before sending it to the terminal device in response.

[0067] The data access method provided in this application involves the server receiving a data access request and determining whether the requested target data exists in the cache or memory. If the target data is not found in either memory or the cache, a read request is initiated to the hard disk. Subsequently, the server determines the media type of the hard disk storing the target data and, based on the media type, decides whether to enable pre-read behavior. Different pre-read methods are used for different hard disks, demonstrating strong adaptability and achieving the goal of improving data access speed.

[0068] Optionally, in the above embodiments, when the media type of the hard drive indicates that the hard drive is a solid-state drive, the read-ahead behavior of the read request is disabled. Then, a read request with the read-ahead behavior disabled is initiated to the hard drive to obtain the target data.

[0069] For example, when it is necessary to read target data from a hard drive, the server determines the media type of the hard drive storing the target data. When the media type indicates that the hard drive is a solid-state drive (SSD), the server disables the read-ahead behavior for the read request. SSDs include common Serial Advanced Technology Attachment (SATA) SSDs, mid-range PCIe NVMe SSDs, and high-end memory-like hard drives, etc.

[0070] For mechanical hard drives (HDDs), the time required for a single read request includes the aforementioned seek time T1, rotational latency T2, and transfer time T3. However, for solid-state drives (SSDs) with strong random I / O capabilities, since there is no seek time T1 or rotational latency T2, the time required for a read request is the transfer time T3, which is proportional to the amount of data transferred. Therefore, the size of the prefetch window set by the prefetch algorithm directly affects the time required for this read request.

[0071] Typically, garbage collection on solid-state drives (SSDs) can cause read requests to be blocked for several milliseconds to tens of milliseconds. This possibility increases with the size of the read request; therefore, large read-ahead windows are not suitable for SSDs.

[0072] Furthermore, due to memory and cache size limitations, prefetched data will discard older cached data already in memory once it enters memory. In high-concurrency scenarios common in cloud computing, a large prefetch window can cause different processes to interfere with each other, discarding data they are about to use. If the existing method of repeatedly expanding the prefetch window is used, the prefetch window will frequently enter a process of expansion, contraction, and repeated probing, making it difficult to guarantee system performance stability.

[0073] In this embodiment, considering that the transfer speed of the solid-state drive during the reading process is generally 500MB / s or even 1GB / s, when the amount of target data requested by the read request is relatively large, this type of read request is also called a large request. Taking a target data size of 512KB as an example, the transfer time T3 is within 1 millisecond. In high-concurrency business scenarios of cloud computing, the CPU may be executing other tasks while waiting for the target data. Therefore, when the time required for the read request is 1 millisecond, this time is within the acceptable range of the CPU.

[0074] Therefore, in this embodiment of the application, when the hard drive is a solid-state drive, the server chooses not to prefetch. This can avoid the drawback that the time required for reading requests is amplified due to the impact of garbage collection, and also reduce the possibility of system jitter caused by data prefetching.

[0075] In this embodiment, disabling read-ahead behavior for read requests means that the target data is read directly without pre-reading during the read process. This way, when the target data is not in the cache or memory, the operating system initiates a read request to the hard disk to read the target data, instead of reading the larger amount of pre-read data. For example, if a read request is 4KB in size, with read-ahead behavior enabled, the amount of pre-read data might be 64KB, including the 4KB target data. If read-ahead behavior is disabled, the operating system reads the 4KB target data from the hard disk.

[0076] Because read-ahead is disabled, the server's operating system reads the target data (4KB) directly from the hard drive. After reading the target data, it is loaded into memory, and then into the cache. The CPU then reads the target data directly from the cache.

[0077] This approach disables read-ahead behavior when the media type indicates that the hard drive is a solid-state drive, avoiding the probabilistic blocking of read requests due to garbage collection, thereby improving the server's concurrency capabilities, increasing data access speed, and reducing the possibility of system jitter.

[0078] Optionally, in the above embodiments, when the hard drive is a mechanical hard drive, the server determines the size of the pre-read window and enables the pre-read behavior of the read request. Then, a read request with pre-read behavior enabled is sent to the hard drive to read pre-read data according to the size of the pre-read window; the pre-read data includes the target data.

[0079] For example, data prefetching can still play a significant role for mechanical hard drives with weak random I / O capabilities. As mentioned above, for small read requests, if the initial prediction window is set too small, the prefetching algorithm needs to perform several probes to reach the ideal prefetch window size, resulting in low data access efficiency. This inefficiency is further amplified when the file size is less than twice the ideal prefetch window size. In this embodiment, when the hard drive's media type indicates it is a mechanical hard drive, the server's operating system determines the size of the prefetch window, which is larger than the initial prefetch window, which is either twice or four times the size of the initial read request.

[0080] In this embodiment, the size of the pre-read window is a fixed size, such as 128KB. For example, a cloud computing provider may deploy services with similar characteristics on the same server. Different services may have different pre-read window sizes.

[0081] By adopting this approach, for mechanical hard drives, a larger prefetch window can be defined, thus avoiding the problem of slow prefetch window startup caused by a small initial prefetch window.

[0082] Optionally, in the above embodiments, after the operating system initially initiates a read request to the hard disk and determines the size of the prefetch window, subsequent prefetches are also performed according to this prefetch window. That is, the size of the prefetch window is used to indicate the size of the prefetch data for the read request and subsequent read requests, until the entire file is read.

[0083] For example, when the hard drive is a mechanical hard drive, the server's operating system determines the size of the prefetch window, such as 128KB. The operating system sends a read request to the hard drive, reads 128KB of prefetch data from the hard drive, and then loads the prefetch data into memory. When the server sends another read request, it continues to use the initial prefetch window size, that is, it continues to read 128KB of prefetch data from the hard drive and load it into memory. Subsequent read requests use the same prefetching method until the entire file is read.

[0084] It should be noted that when the prefetch window is larger, the amount of data transferred increases, and the aforementioned transfer time T3 increases. However, for mechanical hard drives, this impact is very limited. Taking a mechanical hard drive with a transfer speed of 120MB / s as an example, when the prefetch window is 4KB, 32KB, 128KB, and 256KB, the transfer time T3 is 0.03ms, 0.26ms, 1.04ms, and 2.08ms respectively. Assuming that the seek time T1 and rotational latency T2 total 10ms, the time required for a read request is 10.03ms, 10.26ms, 11.04ms, and 12.08ms respectively. Therefore, for small requests below 256KB, the duration of a read request mainly depends on the seek time and rotational latency; increasing the size of the prefetch window within a reasonable range will not significantly increase the time required for the read request. The total time required for a read request includes the aforementioned seek time T1, rotation time T2, and transmission time T3. The total time required for a read request is also referred to as the read request latency.

[0085] This approach, by determining the initial read request's prefetch window size, ensures that subsequent read requests are prefetched according to this window size, abandoning the existing prefetch algorithm's behavior of probing and expanding / shrinking the prefetch window. Furthermore, because the initial prefetch window is relatively large, subsequent prefetches use the same window size, resolving the slow start issue of the prefetch window. Moreover, the fixed prefetch window does not expand further, reducing the pressure on cache and memory caused by excessive prefetch window expansion.

[0086] Optionally, in the above embodiments, a default value is preset. When the media type of the hard drive indicates that the hard drive is a mechanical hard drive, the size of the prefetch window is determined according to the default value. For example, the preset default value is 128KB. Regardless of what applications or services are deployed on the server, the default size of the prefetch window is 128KB. This approach allows for the rapid determination of the prefetch window size.

[0087] Optionally, in the above embodiments, when the media type of the hard disk indicates that the hard disk is a mechanical hard disk, the server determines the size of the pre-read window according to the issued configuration file, and the size of the pre-read window is different for different services.

[0088] For example, cloud computing providers deploy services with similar characteristics onto the same server. The characteristics of these services on the same server are relatively fixed, and their prefetching behavior is predictable. Therefore, users can learn the appropriate prefetch window size for this type of service through machine learning and other methods, and then distribute this information to the server via a configuration file. When the operating system initially initiates a read request, it determines the prefetch window size based on the configuration file, and then performs prefetching based on this prefetch window each time. For example, a virtual machine used to provide video data might have a prefetch window size of 256KB, and this virtual machine is deployed on... Figure 1 On the server shown. For example, a virtual machine is used to provide image data, and the prefetch window size is 128KB.

[0089] This approach, where the size of the prefetch window is related to the type of business deployed on the server, can improve the server's throughput and resource utilization.

[0090] Typically, a read request only requests a portion of a file, and the operating system needs to initiate multiple read requests to read the complete file. For clarity, the file containing the target data will be referred to as the target file.

[0091] In the above embodiments, the server pre-divides the target file on the hard disk into multiple stripes according to the size of the pre-read window. Data is stored in each stripe, and the target file is the file containing the target data requested by the read request. When the server's operating system initiates a read request with pre-read behavior enabled to read pre-read data according to the size of the pre-read window, the read request carries a starting address, which the server's operating system determines from the read request. Then, the operating system determines the stripe containing the starting address from the multiple stripes; this stripe is called the target stripe. In other words, the operating system determines which stripe the starting address falls into, uses the stripe containing the starting address as the target stripe, and uses the data stored in the target stripe as the pre-read data.

[0092] Figure 4 This is a schematic diagram of the stripes in the data access method provided in the embodiments of this application. Please refer to... Figure 4 The server divides the target file into N stripes, each stripe being the size of the read window. For example, if the read window size is 128KB, then each stripe is 128KB. Assuming the starting address of a read request falls into stripe M, the server uses the data in stripe M as prefetch data, meaning it reads the entire data in stripe M into memory. The advantage of this is that it normalizes small read requests, making a small read request a larger prefetch data request. Reading one stripe of data as prefetch data each time results in high disk efficiency.

[0093] Furthermore, dividing the target file into stripes based on the size of the pre-read window effectively avoids unfriendly read requests in cloud computing leasing services. For example, some applications trigger the operating system to continuously initiate random small requests, i.e., initiate a large number of read requests for target data with relatively small amounts of data. In this embodiment, by using the entire target stripe data as pre-read data and reading this data into memory in advance, the CPU can directly read the data from memory without initiating read requests to the hard drive.

[0094] Furthermore, "striping" can also effectively handle localized random reads, as long as the random width is within a finite range, it can be handled by a matching pre-read window.

[0095] This approach, through "striping," can efficiently handle small requests or requests for reading local random features.

[0096] Optionally, in the above embodiments, after the server determines the target strip containing the starting address from multiple stripes, it further determines a first cache page, a second cache page, and a third cache page from the target stripe. The second cache page is a cache page located at a preset position in the target stripe, and the first cache page and the third cache page are respectively located within preset ranges on the left and right sides of the second cache page. Subsequently, when the second cache page is accessed, the pre-read data for the next read request is determined based on the first cache page and the second cache page.

[0097] For example, each time the operating system initiates a read request, it can predict the target stripe for the next read request based on the target stripe corresponding to the current read request. For instance, if the target stripe for the current read request is stripe M, when the operating system accesses the second cached page in stripe M, it can determine the first cached page and the third cached page based on the second cached page, and then determine whether the prefetched data for the next read request is stripe (M-1) or stripe N based on the first cached page and the third cached page.

[0098] Please refer to Figure 4 The second cache page is, for example, the middle cache page in stripe M, i.e., the cache page marked 2 in the diagram. The first cache page is located to the left of the second cache page, and the third cache page is located to the right of the second cache page, and the third cache page cannot be the last cache page in stripe M. In this way, by predicting the pre-fetched data corresponding to the next read request in advance, the CPU can avoid being in a waiting state, thus improving data access speed.

[0099] Optional, please refer to Figure 4In the above embodiment, the first cache page, the second cache page, and the third cache page are located at one-quarter, one-half, and three-quarters positions of the target stripe, respectively. For example, if stripe M is 256KB in total, then the one-quarter, one-half, and three-quarters positions are distributed as 64KB, 128KB, and 192KB, respectively. This allows for rapid determination of the location of each cache page.

[0100] It should be noted that although the above description uses the example of the first cached page, the second cached page, and the third cached page being located at one-quarter, one-half, and three-quarters of the target strip, respectively, the embodiments of this application are not limited. In other feasible implementations, as long as the first cached page and the third cached page are located on either side of the second cached page, and the first cached page is not the first cached page of the target strip, and the third strip is not the last cached page in the target strip, it is acceptable.

[0101] Optionally, in the above embodiments, extracting the pre-read data into memory using a pre-read algorithm is a mechanism under the Linux operating system. Another mechanism under the Linux operating system is the triggering of asynchronous pre-read.

[0102] In this embodiment, a "direction marker" is introduced to guide the triggering of asynchronous prefetching. A preset checkpoint is used to predict the prefetch data of the next prefetch window, i.e., whether the next read request is forward or backward, to ensure the efficiency of sequential I / O (including reverse access) disk access. For example, please refer to Table 1, which is a schematic table illustrating the relationship between prefetch direction and each cache page.

[0103] Table 1

[0104] Mark 2 Mark 1 Mark 3 Pre-read direction √ √ × backward √ × √ forward √ √ √ No pre-read √ × × No pre-read

[0105] Referring to the second row of Table 1, optionally, in the above embodiments, when accessing the second cached page, if the first cached page has already been accessed and the third cached page has not been accessed, then the pre-read data for the next read request is determined to be the first stripe after the target stripe. For example, if the target stripe for the current read request is stripe M, then the target stripe for the next read request is stripe N.

[0106] Please refer to the third row of Table 1. When accessing the second cached page, if the first cached page has not been accessed and the third cached page has been accessed, then the pre-read data for the next read request is determined to be the first stripe before the target stripe. For example, if the target stripe for the current read request is stripe M, then the target stripe for the next read request is stripe (M-1).

[0107] Using this approach, the midpoint of the target strip is selected as the checkpoint for asynchronous read-ahead triggering, i.e. Figure 4 The system uses marker 2 to indicate the location of the second cached page, and then checks whether the first and second cached pages have been accessed to determine the direction of the stripe corresponding to the next read request. This method is fast and accurate.

[0108] Please refer to rows four and five of Table 1. When accessing the second cached page, if both the first and second cached pages have been accessed, then it is determined that the next read request has no data to pre-read. When accessing the second cached page, if neither the first nor the third cached page has been accessed, then it is determined that the next read request has no data to pre-read.

[0109] For example, the midpoint of the target strip is selected as the checkpoint for asynchronous read-ahead triggering, i.e. Figure 4 The system identifies the location of the second cached page indicated by marker 2, and then checks whether the first and second cached pages have been accessed. If both cached pages have been accessed or neither has been accessed, it is determined to be a local random request, and asynchronous reading is not triggered to avoid invalid pre-reading.

[0110] Figure 5 This is another flowchart of the data method provided in this application embodiment. This embodiment includes:

[0111] 501. It has been determined that there is no target data in either memory or cache.

[0112] 502. Determine the media type of the hard drive storing the target data. If the media type indicates that the hard drive is a solid-state drive, proceed to step 503; if the media type indicates that the hard drive is a mechanical hard drive, proceed to step 504.

[0113] 503. Disable the read-ahead behavior of the read request and initiate a read request to the hard disk with the read-ahead behavior disabled to obtain the target data.

[0114] The aforementioned server is, for example, a host machine in cloud computing. When the storage device of the host machine in cloud computing is a solid-state drive, disabling read-ahead behavior can improve the server's concurrency capabilities and reduce the possibility of read request blocking due to hardware garbage collection to some extent.

[0115] 504. Determine the size of the pre-read window and stripe the target file.

[0116] For example, the server determines the size of the prefetch window based on a default value or a configuration file provided by the user. Then, the target file is striped according to the size of the prefetch window. The target file is the file containing the target data. Typically, a single read request only requests a portion of a file, and the operating system needs to initiate multiple read requests to read the complete file.

[0117] 505. Determine the target stripe based on the starting address of the read request.

[0118] For example, the server determines the target strip containing the starting address from multiple stripes based on the starting address carried in the read request, and uses the data stored in the target strip as pre-read data.

[0119] 506. Initiate a read request to the hard drive.

[0120] 507. During the reading of the target stripe, when the triggering condition for asynchronous prefetching is met, predict the target stripe for the next read request.

[0121] For example, when the server reads a cached page at a preset location, such as the cached page in the middle of the target stripe (i.e., the second cached page), it predicts the target stripe for the next read request based on the first and third cached pages within a preset range to the left and right of the second cached page. The target stripe for the next read request is predicted when there is one accessed cached page and one unaccessed cached page among the first and third cached pages.

[0122] If both the first cached page and the third cached page have been accessed or neither has been accessed, then the next read request will not be pre-read, i.e., the pre-read behavior will be turned off.

[0123] The aforementioned server is, for example, a host machine in cloud computing. When the server's storage device is a mechanical hard drive, the pre-read algorithm in this embodiment, while maintaining efficient sequential I / O, can reduce inefficient read requests submitted by upper-layer services through "striping," such as continuous small requests or locally random small requests. This can handle complex leasing businesses and enable the hard drive to maintain high I / O service capabilities.

[0124] Furthermore, in the above embodiments, when the server acts as the host, the host may have two types of hard drives installed, namely, both solid-state drives (SSDs) and hard disk drives (HDDs). However, the target data requested by the data access request is either located on the HDD or the SSD. Therefore, the data access method provided in this application embodiment is also applicable.

[0125] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0126] Figure 6 This is a schematic diagram of a data access device provided in an embodiment of this application. The data access device 600 includes: a transceiver module 61, a determination module 62, a processing module 63, and a reading module 64.

[0127] Transceiver module 61 is used to receive data access requests;

[0128] The determination module 62 is used to determine to initiate a read request to the hard disk when the target data requested by the data access request does not exist in the cache and memory;

[0129] Processing module 63 is used to determine the media type of the hard disk;

[0130] The reading module 64 is used to determine whether to enable the pre-read behavior of the reading request when acquiring the target data based on the media type and to acquire the target data.

[0131] In one feasible implementation, when the reading module 64 determines whether to enable the pre-read behavior of the reading request when acquiring the target data based on the media type and acquires the target data, it is used to disable the pre-read behavior of the reading request when the media type of the hard disk indicates that the hard disk is a solid-state drive; and to initiate a reading request with the pre-read behavior disabled to the hard disk to acquire the target data.

[0132] In one feasible implementation, when the reading module 64 determines whether to enable the pre-read behavior of the read request when acquiring the target data based on the media type and acquires the target data, it is used to determine the size of the pre-read window, enable the pre-read behavior of the read request, and initiate a read request with the pre-read behavior enabled to the hard drive to read the pre-read data according to the size of the pre-read window when the media type of the hard drive indicates that the hard drive is a mechanical hard drive. The pre-read data includes the target data.

[0133] In one feasible implementation, the size of the prefetch window is used to indicate the size of the prefetch data for the read request and subsequent read requests.

[0134] In one feasible implementation, the reading module 64 is used to determine the starting address from the reading request, determine the target strip containing the starting address from multiple stripes, and use the data in the target strip as the pre-read data. The multiple stripes are obtained by dividing the target file according to the size of the pre-read window, and the target file is a file containing the target data.

[0135] In one feasible implementation, after the reading module 64 determines the target strip containing the starting address from multiple stripes, the processing module 63 is further configured to determine a first cache page, a second cache page, and a third cache page from the target stripe. The second cache page is a cache page at a preset position in the target stripe. The first cache page and the third cache page are respectively located within preset ranges on the left and right sides of the second cache page. When accessing the second cache page, the pre-read data for the next read request is determined based on the first cache page and the second cache page.

[0136] In one feasible implementation, the processing module 63 is configured to, when accessing the second cached page, if the first cached page has been accessed and the third cached page has not been accessed, determine that the pre-read data for the next read request is the first stripe after the target stripe; and when accessing the second cached page, if the first cached page has not been accessed and the third cached page has been accessed, determine that the pre-read data for the next read request is the first stripe before the target stripe.

[0137] In one feasible implementation, the processing module 63 is configured to determine that the next read request has no pre-read data if both the first cache page and the second cache page have been accessed when the second cache page is accessed; and to determine that the next read request has no pre-read data if neither the first cache page nor the third cache page has been accessed when the second cache page is accessed.

[0138] In one feasible implementation, the first cached page, the second cached page, and the third cached page are located at one-quarter, one-half, and three-quarters positions of the target strip, respectively.

[0139] In one feasible implementation, the determining module 62 is used to determine the size of the pre-read window based on a default value when the media type of the hard disk indicates that the hard disk is a mechanical hard disk; or, when the media type of the hard disk indicates that the hard disk is a mechanical hard disk, to determine the size of the pre-read window based on the issued configuration file, wherein the size of the pre-read window is different for different services.

[0140] The data access device provided in this application embodiment can perform the actions of the server in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0141] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 is, for example, the server described above, and the electronic device 700 includes:

[0142] Processor 71 and memory 72;

[0143] The memory 72 stores computer instructions;

[0144] The processor 71 executes the computer instructions stored in the memory 72, causing the processor 71 to perform the traffic attack protection method described above as a control center; or, causing the processor 71 to perform the traffic attack protection method described above as an anti-attack node.

[0145] The specific implementation process of processor 71 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0146] Optionally, the electronic device 700 also includes a communication component 73. The processor 71, memory 72, and communication component 73 can be connected via a bus 74.

[0147] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the data access method implemented by the server as described above.

[0148] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the data access method implemented by the server as described above.

[0149] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0150] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data access method, characterized by, The method comprises: receiving a data access request; determining to initiate a read request to a hard disk when target data requested by the data access request does not exist in a cache and in a memory; determining a medium type of the hard disk; determining whether to start a pre-reading behavior of the read request when the target data is acquired according to the medium type and acquiring the target data; the determining whether to start the pre-reading behavior of the read request when the target data is acquired according to the medium type and acquiring the target data comprises: when the medium type of the hard disk indicates that the hard disk is a mechanical hard disk, determining a size of a pre-reading window, the size of the pre-reading window being a preset fixed value; starting the pre-reading behavior of the read request; initiating the read request with the started pre-reading behavior to the hard disk to read pre-reading data according to the size of the pre-reading window, the pre-reading data containing the target data, the read request being a random IO request, and a data volume of the target data requested by the random IO request being less than a preset data volume.

2. The method of claim 1, wherein, the determining whether to start the pre-reading behavior of the read request when the target data is acquired according to the medium type and acquiring the target data comprises: when the medium type of the hard disk indicates that the hard disk is a solid state disk, closing the pre-reading behavior of the read request; initiating the read request with the closed pre-reading behavior to the hard disk to acquire the target data.

3. The method of claim 1, wherein: the size of the pre-reading window is used to indicate a size of pre-reading data of the read request and a read request following the read request.

4. The method of claim 1, wherein, the initiating the read request with the started pre-reading behavior to the hard disk to read pre-reading data according to the size of the pre-reading window comprises: determining a start address from the read request; determining a target stripe containing the start address from a plurality of stripes, the plurality of stripes being obtained by dividing a target file according to the size of the pre-reading window, the target file being a file containing the target data, and data in the target stripe being used as the pre-reading data.

5. The method of claim 4, wherein, after the determining the target stripe containing the start address from the plurality of stripes, the method further comprises: determining a first cache page, a second cache page and a third cache page from the target stripe, the second cache page being a cache page at a preset position in the target stripe, the first cache page and the third cache page being respectively located within a preset range left and right of the second cache page; when the second cache page is accessed, determining pre-reading data of a next read request according to the first cache page and the second cache page.

6. The method of claim 5, wherein, the determining the pre-reading data of the next read request according to the first cache page and the second cache page when the second cache page is accessed comprises: when the second cache page is accessed, if the first cache page has been accessed and the third cache page has not been accessed, determining that the pre-reading data of the next read request is a first stripe after the target stripe. When the second cache page is accessed, if the first cache page is not accessed and the third cache page is accessed, it is determined that the pre-read data of the next read request is the first strip before the target strip.

7. The method of claim 5, wherein, When the second cache page is accessed, according to the first cache page and the second cache page, the pre-read data of the next read request is determined. When the second cache page is accessed, if the first cache page and the second cache page are both accessed, it is determined that the next read request has no pre-read data. When the second cache page is accessed, if the first cache page and the third cache page are not accessed, it is determined that the next read request has no pre-read data.

8. The method of claim 5, wherein the first cache page, the second cache page and the third cache page are located at one fourth, one half and three fourths of the target strip respectively. When the medium type of the hard disk indicates that the hard disk is a mechanical hard disk, the size of the pre-read window is determined.

9. The method according to any one of claims 1 to 8, characterized in that, When the medium type of the hard disk indicates that the hard disk is a mechanical hard disk, the size of the pre-read window is determined according to a default value. Alternatively, When the medium type of the hard disk indicates that the hard disk is a mechanical hard disk, the size of the pre-read window is determined according to a configuration file issued, and the sizes of the pre-read windows corresponding to different services are different. Comprising:

10. A data access device, characterized by The transceiver module is configured to receive a data access request. The determination module is configured to determine to initiate a read request to the hard disk when the target data requested by the data access request does not exist in the cache and the memory. The processing module is configured to determine the medium type of the hard disk. The read module is configured to determine whether to start the pre-read behavior of the read request when obtaining the target data according to the medium type and obtain the target data. The read module is specifically configured to, when the medium type of the hard disk indicates that the hard disk is a mechanical hard disk, determine the size of the pre-read window, the size of the pre-read window is a preset fixed value, start the pre-read behavior of the read request, initiate a read request with the pre-read behavior started to the hard disk to read pre-read data according to the size of the pre-read window, the pre-read data contains the target data, the read request is a random IO request, and the data amount of the target data requested by the random IO request is less than a preset data amount. The processor executes the computer program to enable the electronic device to implement the method of any one of claims 1 to 9.

11. An electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, ​

Images