Method and system for optimizing direct I / O read performance under Linux system

Abstract

This invention discloses a method and system for optimizing direct I / O read performance under Linux. The method includes: introducing a cache control flag to control cache access; during the file opening phase, determining whether a caching mechanism is enabled; if enabled, when performing a direct I / O read operation, first checking for a cache hit; if a cache hit occurs, reading and returning data directly from the cache; if a cache miss occurs, calling standard direct I / O to read and return data, while simultaneously caching the data asynchronously and setting a cache flag; during a direct I / O write operation, determining whether a caching mechanism is enabled; if enabled, first checking for a cache hit; if a cache hit occurs, invalidating the existing cached data, asynchronously caching the data, and simultaneously calling standard direct I / O to write the data; and after the asynchronous caching and data write operations are completed, setting the cached data to valid. This invention can improve the read performance of direct I / O.

Description

Technical Field

[0001] This invention relates to the field of I / O performance optimization technology, specifically to a method and system for optimizing direct I / O read performance under Linux. Background Technology

[0002] With the rapid development of technology, the performance of central processing units (CPUs), memory, and graphics processing units (GPUs) has been significantly enhanced. However, compared to other hardware components, the improvement in hard drive performance has lagged behind, gradually becoming a bottleneck restricting the overall performance improvement of the system. To solve this problem, operating systems have adopted caching technology, which temporarily stores data about to be written to disk in memory. When subsequent read / write operations occur, if the required data exists in the cache, the operation is performed directly on the cached data, thereby significantly improving the performance of the storage system. However, current caching technology is not suitable for application scenarios with extremely high data consistency requirements, such as database systems. In these specific applications, direct input / output (Direct I / O) technology is usually used to ensure high data consistency. In direct I / O mode, all read and write operations are performed directly from the disk. Although this method is simple to implement, in read-intensive database applications where read operations far outnumber write operations, such as databases in artificial intelligence systems, direct I / O cannot fully utilize the advantages of memory caching.

[0003] In the prior art, Chinese invention patent application number CN201510236213.9 discloses an adaptive direct I / O acceleration method for SSD file systems, which mainly improves the performance of cached I / O by adaptively utilizing direct I / O to shorten the I / O path; Chinese invention patent application number CN202510052540.2 discloses a scheduling method and system for Direct I / O intensive tasks under NUMA architecture, which achieves efficient scheduling and improves task performance by reducing the overhead of accessing disks across NUMA nodes. Neither of these methods can effectively solve the performance problem of direct read I / O caused by the inefficient use of memory cache in read-intensive application scenarios.

[0004] Therefore, there is an urgent need for a new technology to supplement existing direct I / O technology in order to better adapt to read-intensive application scenarios, and to further improve the read performance of direct I / O through memory caching while ensuring data consistency. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a method and system for optimizing direct I / O read performance under Linux, thereby improving direct I / O read performance in read-intensive application scenarios, in order to address the aforementioned problems in the prior art.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for optimizing direct I / O read performance in a Linux system, including the following steps:

[0008] Introduce the cache control flag O_CACHE in the kernel to control cache access;

[0009] Direct I / O read optimization: During the file opening stage, the cache control flag determines whether to enable the caching mechanism. If enabled, when performing a direct I / O read operation, the cache hit is checked first. If the cache hits, the data is read directly from the cache and returned. If the cache misses, standard direct I / O is called to read from memory and return the data. At the same time, the read data is asynchronously cached and a valid cache flag is set to build a caching mechanism to accelerate subsequent data reads.

[0010] Direct I / O write cache: When performing a direct I / O write operation, the cache control flag is used to determine whether the caching mechanism is enabled. If enabled, the cache is first checked for a cache hit. If a cache hit occurs, the existing cached data is set to invalid, the data is asynchronously cached, and the standard direct I / O is called to write the data to the memory. After the asynchronous caching and data write operations are completed, the cached data is set to valid so that the cached data is kept up-to-date.

[0011] Furthermore, the cache control flag O_CACHE is set in the header file include / uapi / asm-generic / fcntl.h, which defines the file open flags in the Linux kernel, and is enabled by a unique flag bit; the cache control flag is enabled by detecting whether the set of open flags iocb->ki_filp->f_flags corresponding to the current I / O operation contains O_CACHE.

[0012] Furthermore, the direct I / O read optimization specifically includes:

[0013] When performing a direct I / O read operation, the system first checks whether the cache control flag is enabled. If the cache control flag is not enabled, the system's original standard direct I / O read process is executed.

[0014] When the cache control flag is enabled, the corresponding cache page index is calculated based on the file offset of the currently read data, and the corresponding cache page is searched in the file's page cache mapping;

[0015] If no corresponding cache page is found, a new cache page is allocated for the currently read data and the cache validity status is initialized to invalid.

[0016] If a cached page is found, the validity status recorded in the cached page is used to determine whether the cached data is valid. If the cached data is valid, the data is read directly from the cached page and returned. If the cached data is invalid or a cache miss occurs, the system's original standard direct I / O read process is executed to complete the data read. After the read is completed, a background asynchronous task is started to write the read data into the cached page to form cached data.

[0017] Furthermore, the `find_get_page(mapping, index)` function searches for the existence of a corresponding cached page in the file's cache mapping `mapping` based on the page index `index`. The cache mapping `mapping` obtains the page cache mapping structure of the file by retrieving the `f_mapping` member of the file object `file`, and the cached page index `index` is obtained by dividing the file offset `iocb->ki_pos` by the page size `PAGE_SHIFT` to get the page number index of that offset.

[0018] Furthermore, the asynchronous cache filling mechanism for asynchronous data caching specifically includes:

[0019] Construct a task structure cache_task for encapsulating asynchronous caching tasks. The task structure includes target file information file, file read / write offset position pos, data to be cached description information iter, cache page for storing cached data page, and task control unit work for submitting tasks to the kernel work queue.

[0020] During the system initialization phase, a global work queue wq is created and initialized to handle cache write tasks;

[0021] When a cache write is triggered, a task structure is allocated and populated with the corresponding file information object, cache page object, file read / write offset object, and data description information object. The task structure is then submitted to the global work queue.

[0022] After receiving the task structure, the global work queue copies the data in the data description information to the specified cache page, establishes an association between the specified cache page and the page cache mapping of the target file, and updates the validity status of the cache page to indicate that the cache write operation is complete.

[0023] Furthermore, the association between the specified cache page and the page cache mapping of the target file is established through the function add_to_page_cache(page, file->f_mapping, pos), where add_to_page_cache() is a kernel-implemented function that establishes the association between a page and a cache mapping, and file->f_mapping represents the page cache address space mapping structure of the current file file in the Linux kernel.

[0024] Furthermore, the direct I / O write cache specifically includes:

[0025] When performing a direct I / O write operation, the system first checks whether the current file's open flag contains a cache control flag. If it does not contain a cache control flag, the system's original standard direct I / O write process is executed.

[0026] If the cache control flag is included, the corresponding page cache mapping relationship is obtained according to the file object of the current write operation, and the corresponding data page index is calculated according to the current file offset position, and the corresponding cache page is found in the page cache mapping;

[0027] When no cache page is found, a new cache page is allocated for the current write operation. When a cache page is found, its validity status is set to invalid, and then a background asynchronous cache write task is started to copy the data to be written to the cache page. At the same time, the system's original standard direct I / O write process is executed synchronously. After the write process is completed, the status count of the cache page is updated to indicate that the write process is complete. After the asynchronous cache write task is completed, the status count is updated again so that the status count is set to indicate that the data in the cache page is valid, thus realizing the coordinated processing of direct I / O write and cache update.

[0028] Furthermore, during the file opening stage, the validity of the cache control flag O_CACHE needs to be verified. If it is detected that the cache control flag O_CACHE is enabled but standard direct I / O is not enabled at the same time, the parameter configuration is determined to be invalid and an error message is returned, terminating the subsequent steps.

[0029] A system for optimizing direct I / O read performance under Linux includes a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to perform the steps of a method for optimizing direct I / O read performance under Linux.

[0030] A computer-readable storage medium storing a computer program / instructions programmed or configured to execute steps of a method for optimizing direct I / O read performance under a Linux system via a processor.

[0031] Compared with the prior art, the advantages of the present invention are as follows:

[0032] This invention maintains the original direct I / O mechanism of the Linux system and introduces a cache control flag to achieve coordinated operation between direct I / O and the caching mechanism. During read operations, the system prioritizes checking the cache and returns the data directly upon a cache hit, thereby reducing disk accesses and significantly lowering I / O latency. When a cache miss occurs, data is read via standard direct I / O and asynchronous cache backfilling is performed, improving subsequent access efficiency. During write operations, cache hits are checked and old cache entries are invalidated promptly. Combined with an asynchronous cache update mechanism, this ensures that cached data is always up-to-date. This invention achieves low-overhead cache acceleration without altering the basic direct I / O process. While improving direct I / O read performance and overall system I / O efficiency, it ensures that data writing is still completed through the standard direct I / O path, thus guaranteeing data persistence and consistency. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating the framework and process of a method for optimizing direct I / O read performance under a Linux system, as described in an embodiment of the present invention.

[0034] Figure 2 This is a flowchart illustrating the optimization of the direct I / O read path in a specific application example.

[0035] Figure 3 This is a flowchart illustrating the cache management implementation in a specific application example.

[0036] Figure 4 This is a flowchart illustrating the optimization of the direct I / O write path in a specific application example. Detailed Implementation

[0037] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0038] The abbreviations and key terms mentioned in this invention are defined as follows:

[0039] Direct I / O: Direct I / O refers to an I / O mode where file read and write operations bypass the system cache and directly transfer data between user space and block devices. Its purpose is to reduce memory copying, decrease cache pollution, or achieve stronger I / O consistency control. For example, databases and logging systems often use direct I / O for easier I / O consistency control. User space enables direct I / O by specifying the O_DIRECT flag in the open() call. Therefore, when opening a file using O_DIRECT, the page cache is bypassed, disabling cache acceleration.

[0040] Caching: Caching is an efficient temporary storage mechanism used to accelerate data access. Caching exists at multiple levels, including the operating system, CPU, disk, and network. Its core idea is to use faster storage media to store copies of recently used or soon-to-be-used data, thereby reducing access latency and overhead.

[0041] like Figure 1 As shown in this embodiment, the method for optimizing direct I / O read performance under the Linux system includes the following steps:

[0042] Introduce the cache control flag O_CACHE in the kernel to control cache access;

[0043] Direct I / O read optimization: During the file opening phase, the cache control flag determines whether the caching mechanism is enabled. If enabled, the cache is checked before performing a direct I / O read operation. If the cache is hit, the data is read directly from the cache and returned. If the cache is not hit, standard direct I / O is called to read from memory and return the data. At the same time, the read data is asynchronously cached and a valid cache flag is set to build a caching mechanism to accelerate subsequent data reads.

[0044] Direct I / O write cache: When performing a direct I / O write operation, the cache control flag determines whether the caching mechanism is enabled. If enabled, the cache is first checked for a cache hit. If a cache hit occurs, the existing cached data is invalidated, the data is asynchronously cached, and the standard direct I / O is called to write the data to memory. After the asynchronous caching and data write operations are completed, the cached data is made valid to keep the cached data up-to-date.

[0045] In this embodiment, the cache control flag O_CACHE is set in the header file include / uapi / asm-generic / fcntl.h, which defines the file open flags in the Linux kernel, and a unique flag bit indicates that the cache control function is enabled. The cache control flag is enabled by checking whether the set of open flags for the file corresponding to the current I / O operation contains O_CACHE.

[0046] In this embodiment, during the file opening stage, it is also necessary to verify the validity of the cache control flag O_CACHE. If it is detected that the cache control flag O_CACHE is enabled but standard direct I / O is not enabled at the same time, the parameter configuration is determined to be invalid and an error message is returned, terminating the subsequent steps.

[0047] In specific application embodiments, the implementation process is simplified by using cached pages as the unit, such as... Figure 2 As shown, the steps for setting the O_CACHE flag, S1, are as follows:

[0048] S101, Introduce a new file open flag O_CACHE: Add the file open flag in the header file include / uapi / asm-generic / fcntl.h: #define O_CACHE 0x40000000;

[0049] S102, Verify the validity of O_CACHE when opening a file: Add a check for the validity of the O_CACHE flag in the `build_open_flags()` function in the `fs / open.c` file. Specifically, if it is detected that the O_CACHE flag is set but the O_DIRECT flag (indicating whether standard direct I / O is enabled) is not set simultaneously, return -EINVAL to indicate that the parameter configuration is incorrect.

[0050] In this embodiment, direct I / O read optimization specifically includes:

[0051] When performing a direct I / O read operation, first determine whether the cache control flag is enabled. If the cache control flag is not enabled, execute the system's original standard direct I / O read process.

[0052] When the cache control flag is enabled, the corresponding cache page index is calculated based on the file offset of the currently read data, and the corresponding cache page is searched in the file's page cache mapping;

[0053] If no corresponding cache page is found, a new cache page is allocated for the currently read data and the cache validity status is initialized to invalid.

[0054] If a cached page is found, the validity status recorded in the cached page is used to determine whether the cached data is valid. If the cached data is valid, the data is read directly from the cached page and returned. If the cached data is invalid or a cache miss occurs, the system's original standard direct I / O read process is executed to complete the data read. After the read is completed, a background asynchronous task is started to write the read data into the cached page to form cached data.

[0055] In this embodiment, the find_get_page(mapping, index) function is used to search for the existence of a corresponding cached page in the file's cache mapping based on the page index. The cache mapping obtains the page cache mapping structure of the file by retrieving the f_mapping member of the file object file. The cached page index is obtained by dividing the file offset iocb->ki_pos by the page size PAGE_SHIFT to get the page number index of the offset.

[0056] In specific application embodiments, such as Figure 2 As shown, the direct I / O read path optimization is implemented in the kernel's `generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter)`. Its goal is to prioritize reading data from the cache during direct I / O reads. If a cache miss occurs, it reverts to the kernel's original standard direct I / O read process and performs cache backfilling. Specifically, it includes the following steps S2:

[0057] S201: Check if the O_CACHE flag is enabled;

[0058] Before performing a direct I / O read operation, it first checks whether iocb->ki_filp->f_flags contains O_CACHE. If this flag is not enabled, the kernel's original standard direct I / O read process path is followed; if O_CACHE is enabled, the process proceeds to the subsequent enhancement step S202.

[0059] S202: Check if the cache is hit. If it is hit, read the cached data and return it, ending the direct I / O read process; if it is not hit, proceed to step S203.

[0060] S2021, call `struct page *page = find_get_page(mapping, index)` to check if there is cached data in the cache. `find_get_page()` is a kernel-implemented function that searches for and retrieves a specific page cache in the cache mapping. `mapping`, i.e., the cache mapping, is obtained through `file->f_mapping`, and `index` is calculated using `iocb->ki_pos>>PAGE_SHIFT`. If the returned page is NULL, it means there is no cached memory, proceed to step S2022; otherwise, proceed to step S2024.

[0061] S2022, call page = alloc_page() to allocate space;

[0062] S2023, call page->private = 1 to set the cache validity, exit step S202. Since for read data cache, the data is considered valid once the asynchronous cache is written, it is set to 1 in advance;

[0063] S2024, check if the cached content is valid by checking page->private. If page->private is less than 2, it means that the cached content is invalid and the cache misses, so exit step S202.

[0064] S2025, cache hit, call copy_page_to_iter(page,..., iter) to read data from the cache, end step S2; copy_page_to_iter() is a kernel-implemented function that copies the page cache to iter;

[0065] S203: The kernel's original standard direct I / O read procedure is invoked to complete the data read;

[0066] S204: Start asynchronous cache writing thread to cache the data read, cache the read data asynchronously and set it to valid, and synchronously return the read data to user space;

[0067] The asynchronous cache write is initiated by calling async_cache_write(file, page, iter, iocb->ki_pos), and the function async_cache_write() is specifically implemented in step S303.

[0068] It is understandable that in step S202, when a cache hit occurs, it indicates that the data in the cache is exactly the same as the data in the storage device. In this case, data is retrieved directly from the cache, avoiding reading data from the less efficient storage device, which is crucial for improving direct I / O read performance.

[0069] In this embodiment, the asynchronous cache filling mechanism for asynchronous data caching specifically includes:

[0070] Construct a task structure `cache_task` to encapsulate asynchronous caching tasks. The task structure contains target file information `file`, file read / write offset position `pos`, description information of data to be cached `iter`, cache page for storing cached data `page`, and task control unit `work` for submitting tasks to the kernel work queue.

[0071] During the system initialization phase, a global work queue wq is created and initialized to handle cache write tasks;

[0072] When a cache write is triggered, a task structure is allocated and populated with the corresponding file information object, cache page object, file read / write offset object, and data description information object. The task structure is then submitted to the global work queue.

[0073] After receiving the task structure, the global work queue copies the data in the data description information to the specified cache page, establishes an association between the specified cache page and the page cache mapping of the target file, and updates the validity status of the cache page to indicate that the cache write operation is complete.

[0074] In this embodiment, the association between a specified cache page and the page cache mapping of the target file is established by the function add_to_page_cache(page, file->f_mapping, pos), where add_to_page_cache() is a kernel-implemented function that establishes an association between a page and a cache mapping, and file->f_mapping represents the page cache address space mapping structure of the current file file in the Linux kernel.

[0075] In specific application embodiments, such as Figure 3 As shown, the cache management design and implementation steps S3 are as follows:

[0076] S301: Define the data structures required by the cache module;

[0077] The data structure `cache_task` is defined to encapsulate all context information of an asynchronous cache task so that it can be submitted to the background cache write thread for processing. It contains the fields `struct file *file` to represent the target file for the data to be cached, `loff_t pos` to represent the file offset of this read / write operation and to determine the cache position, `struct iov_iter *iter` to describe the data to be written to the cache, `struct page *page` to store the allocated cache memory, and `struct work_struct work` to submit the task to the kernel work queue.

[0078] Define a global cache work queue: `struct workqueue_struct *cache_wq`;

[0079] S302: Initialize the global cache work queue;

[0080] The `cache_wq = alloc_workqueue("cache_writer", WQ_UNBOUND | WQ_HIGHPRI, 0)` function is called to complete the initialization and creation of the work queue.

[0081] S303: Implementation of asynchronous cache write interface;

[0082] The cache write interface is implemented as async_cache_write(struct *file, struct *page, struct iov_iter *from, loff_t pos), and the specific process is as follows:

[0083] Allocate task structure cache_task = kzalloc(sizeof(*cache_task), GFP_KERNEL);

[0084] Run `cache_task->file = file` to pass the file to the cache asynchronous write thread;

[0085] Run `cache_task->page = page` to pass the page cache space to the cache asynchronous write thread;

[0086] Run `cache_task->pos = pos` to pass the file offset to the cache asynchronous write thread;

[0087] Run `cache_task->iter = from` to pass the data to be cached to the cache asynchronous write thread;

[0088] Initialize the task using INIT_WORK(&task->work, cache_write_worker);

[0089] Submit the task: queue_work(cache_wq,&cache_task->work);

[0090] S304: Implementation of the asynchronous write thread handling function for the cache;

[0091] The `copy_page_from_iter(page,...,iter)` function copies data to the cache. `copy_page_from_iter()` is a kernel-implemented function that reads data from `iter` and copies it to the cache of the specified page.

[0092] The function `add_to_page_cache(page, file->f_mapping, pos...)` inserts the cache into the cache pool; `add_to_page_cache()` is a kernel-implemented function that establishes an association between a page and a cache mapping.

[0093] Perform the page->private++ operation to set the validity of cached data, and increment the private value by 1 to indicate that the asynchronous cache write operation is complete.

[0094] It is understandable that in step S303, the asynchronous cache write interface acts as a bridge, decoupling the synchronous write process from the asynchronous cache mechanism. This interface does not directly operate on the cache, but rather transforms cache requests into tasks, which are then handled by the asynchronous write thread. The main function of the asynchronous cache write interface is to collect necessary context information, construct complete task information (including file, data, offset, etc.) based on the current I / O data, and encapsulate it into a structure. This structure carries all the data required for the asynchronous cache thread to process, and finally safely submits the task to the kernel work queue, thereby achieving non-blocking of the cache update process. In step S303, it is essential to ensure that referenced resources (such as files, data, etc.) are not released or overwritten before the asynchronous thread completes processing; reference counting may be necessary to guarantee this. In step S304, the asynchronous write thread processing function is the core processing logic for executing the asynchronous cache filling task. It is scheduled and executed by the work queue, and its main responsibility is to copy data to the cache, ensure the cache state is correctly marked, and finally insert the cached data into the cache management system.

[0095] In this embodiment, direct I / O write to the cache specifically includes:

[0096] When performing a direct I / O write operation, the system first checks whether the current file's open flag contains a cache control flag. If it does not contain a cache control flag, the system's original standard direct I / O write process is executed.

[0097] If a cache control flag is included, the corresponding page cache mapping relationship is obtained based on the file object of the current write operation, and the corresponding data page index is calculated based on the current file offset position, and the corresponding cache page is found in the page cache mapping.

[0098] When no cache page is found, a new cache page is allocated for the current write operation. When a cache page is found, its validity status is set to invalid, and then a background asynchronous cache write task is started to copy the data to be written to the cache page. At the same time, the system's original standard direct I / O write process is executed synchronously. After the write process is completed, the status count of the cache page is updated to indicate that the write process is complete. After the asynchronous cache write task is completed, the status count is updated again to set the status count to indicate that the data in the cache page is valid, thus realizing the coordinated processing of direct I / O write and cache update.

[0099] In specific application embodiments, such as Figure 4 As shown, the goal of direct I / O write path optimization is to add cache access capabilities to the standard kernel direct I / O write process. The relevant optimizations are concentrated in the kernel's `generic_file_direct_write(struct kiocb *iocb, struct iov_iter *from)` function. Step S4 of the direct I / O write path optimization implementation is as follows:

[0100] S401: Check if the O_CACHE flag is enabled;

[0101] Before performing a direct I / O write operation, it first checks whether iocb->ki_filp->f_flags contains O_CACHE. If this flag is not enabled, the kernel's original standard direct I / O write process is followed; if O_CACHE is enabled, the process proceeds to the subsequent enhancement step S402.

[0102] S402: Check if the cache is hit. If it is hit, the cached data is set to invalid.

[0103] S4021, call struct page *page = find_get_page(mapping, index) to check if there is already cached data in the cache. The mapping is obtained through file->f_mapping, the file is obtained through iocb->ki_filp, and the index is calculated through iocb->ki_pos>>PAGE_SHIFT. If the returned page is NULL, it means that there is no cached data. Proceed to step S4022; otherwise, proceed to step S4023.

[0104] S4022, call page = alloc_page() to allocate cache space, and end step S402;

[0105] S4023, set page->private to 0 to invalidate cached data;

[0106] S403: Start an asynchronous cache write thread to cache the data asynchronously;

[0107] An asynchronous cache write operation is initiated by calling async_cache_write(file, page, from, iocb->ki_pos), the detailed implementation of which is located in step S303;

[0108] S404: Synchronously executes the kernel's original standard direct I / O write process;

[0109] S405: Mark the validity of cached data after the kernel standard direct I / O write process is completed and the data is asynchronously cached;

[0110] The `page->private++` operation increments the `private` value by 1 to indicate that the kernel's standard direct I / O write process is complete. Similarly, `page->private++` is executed after an asynchronous cache write operation completes. When the value of `page->private` reaches 2, it means that the data in the cache is valid.

[0111] It is understandable that in step S402, given the asynchronous write mechanism for cached data, to ensure that direct I / O read operations retrieve only the latest data, matching data in the cache must be marked as invalid before performing direct I / O writes to avoid reading outdated data from the cache during direct I / O. In step S403, an asynchronous write caching mechanism is implemented to reduce the negative impact of data cache writes on the performance of direct I / O writes. In step S405, cached data is only valid and can only achieve a cache hit during direct I / O read operations when data maintains consistency between the cache and storage.

[0112] This invention introduces a combination of the O_CACHE and O_DIRECT flags used when opening files to indicate whether a direct I / O caching mechanism is enabled during file opening. This flag can be used to open files in read-intensive application scenarios. Secondly, the processing logic for direct I / O write paths has been improved. When performing a direct I / O write operation, the system first checks whether the file is opened with the O_CACHE flag. If this flag is not set, the kernel's default direct I / O write process is used; if O_CACHE is set, the written data will be asynchronously cached, and the standard direct I / O write operation will be performed synchronously to ensure data persistence. Finally, the processing logic for the direct I / O read path was also optimized. When performing a direct I / O data read operation on a file, it first checks whether the file is opened in O_CACHE mode. If this flag is not set, the standard direct I / O read process is followed. If O_CACHE is set, it first checks whether there is valid data in the cache. If the cache is hit, the content in the cache is returned directly, avoiding reading data from inefficient storage devices. If the cache is not hit, the data is read through standard direct I / O, and the read result is asynchronously written to the cache for later use. This invention addresses the problem that existing direct I / O technologies cannot efficiently utilize memory caching to accelerate direct read I / O performance. It proposes an optimization method for direct I / O read performance, better adapting to read-intensive application scenarios. While ensuring the reliability of data persistence to disk and data consistency, it further improves the read performance of direct I / O through memory caching.

[0113] The present invention further provides a system for optimizing direct I / O read performance under a Linux system, comprising a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the steps of a method for optimizing direct I / O read performance under a Linux system.

[0114] The present invention further provides a computer-readable storage medium storing a computer program / instructions, the computer program / instructions being programmed or configured to execute, via a processor, steps of a method for optimizing direct I / O read performance under a Linux system.

[0115] The system and medium of the present invention, corresponding to the methods described above, also have the advantages described above.

[0116] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0117] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for optimizing direct I / O read performance under a Linux system, characterized in that, Including the following steps: Introduce the cache control flag O_CACHE in the kernel to control cache access; Direct I / O read optimization: During the file opening stage, the cache control flag determines whether to enable the caching mechanism. If enabled, when performing a direct I / O read operation, the cache hit is checked first. If the cache hits, the data is read directly from the cache and returned. If the cache misses, standard direct I / O is called to read from memory and return the data. At the same time, the read data is asynchronously cached and a valid cache flag is set to build a caching mechanism to accelerate subsequent data reads. Direct I / O write cache: When performing a direct I / O write operation, the cache control flag is used to determine whether the cache mechanism is enabled. If it is enabled, the cache is first checked for a cache hit. If a cache hit occurs, the original cached data is set to invalid. The data is asynchronously cached while standard direct I / O is called to write the data to the memory. After the asynchronous caching and data writing operations are completed, the cached data is set to valid so that the cached data is kept up-to-date. Specifically, the direct I / O read optimization includes: When performing a direct I / O read operation, the system first checks whether the cache control flag is enabled. If the cache control flag is not enabled, the system's original standard direct I / O read process is executed. When the cache control flag is enabled, the corresponding cache page index is calculated based on the file offset of the currently read data, and the corresponding cache page is searched in the file's page cache mapping. If no corresponding cache page is found, a new cache page is allocated for the currently read data and the cache validity status is initialized to invalid. If a cache page is found, the validity status recorded in the cache page is used to determine whether the cached data is valid. If the cached data is valid, the data is read directly from the cache page and returned. If the cached data is invalid or is not found, the original standard direct I / O read process of the system is executed to complete the data reading. After the reading is completed, a background asynchronous task is started to write the read data into the cache page to form cached data. The asynchronous cache filling mechanism for asynchronous data caching specifically includes: Construct a task structure cache_task for encapsulating asynchronous caching tasks. The task structure includes target file information file, file read / write offset position pos, data to be cached description information iter, cache page for storing cached data page, and task control unit work for submitting tasks to the kernel work queue. During system initialization, a global work queue wq is created and initialized to handle cache write tasks; When a cache write is triggered, a task structure is allocated and populated with the corresponding file information object, cache page object, file read / write offset object, and data description information object. The task structure is then submitted to the global work queue. After receiving the task structure, the global work queue copies the data in the data description information to the specified cache page, establishes an association between the specified cache page and the page cache mapping of the target file, and updates the validity status of the cache page to indicate that the cache write operation is complete.

2. The method for optimizing direct I / O read performance under Linux system according to claim 1, characterized in that, The cache control flag O_CACHE is set in the header file include / uapi / asm-generic / fcntl.h, which defines file open flags in the Linux kernel, and is enabled by a unique flag bit. The cache control flag is enabled by checking whether the set of open flags for the file corresponding to the current I / O operation contains O_CACHE.

3. The method for optimizing direct I / O read performance under Linux system according to claim 1, characterized in that, The `find_get_page(mapping, index)` function searches for the corresponding cached page in the file's cache mapping `mapping` based on the page index `index`. The cache mapping `mapping` obtains the file's page cache mapping structure by retrieving the `f_mapping` member of the file object `file`. The cached page index `index` is obtained by dividing the file offset `iocb->ki_pos` by the page size `PAGE_SHIFT` to get the page number index of that offset.

4. The method for optimizing direct I / O read performance under Linux system according to claim 1, characterized in that, The association between a specified cache page and the page cache mapping of the target file is established by the function add_to_page_cache(page, file->f_mapping, pos). Here, add_to_page_cache() is a kernel-implemented function that establishes the association between a page and a cache mapping, and file->f_mapping represents the page cache address space mapping structure of the current file file in the Linux kernel.

5. The method for optimizing direct I / O read performance under Linux system according to claim 1, characterized in that, The direct I / O write cache specifically includes: When performing a direct I / O write operation, the system first checks whether the current file's open flag contains a cache control flag. If it does not contain a cache control flag, the system's original standard direct I / O write process is executed. If the cache control flag is included, the corresponding page cache mapping relationship is obtained according to the file object of the current write operation, and the corresponding data page index is calculated according to the current file offset position, and the corresponding cache page is found in the page cache mapping; When no cache page is found, a new cache page is allocated for the current write operation. When a cache page is found, its validity status is set to invalid, and then a background asynchronous cache write task is started to copy the data to be written to the cache page. At the same time, the system's original standard direct I / O write process is executed synchronously. After the write process is completed, the status count of the cache page is updated to indicate that the write process is complete. After the asynchronous cache write task is completed, the status count is updated again so that the status count is set to indicate that the data in the cache page is valid, thus realizing the coordinated processing of direct I / O write and cache update.

6. The method for optimizing direct I / O read performance under Linux system according to claim 1, characterized in that, During the file opening phase, the validity of the cache control flag O_CACHE also needs to be verified. If it is detected that the cache control flag O_CACHE is enabled but standard direct I / O is not enabled at the same time, the parameter configuration is determined to be invalid and an error message is returned, terminating the subsequent steps.

7. A system for optimizing direct I / O read performance under Linux, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to perform the steps of the method for optimizing direct I / O read performance under the Linux system as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program / instructions, characterized in that, The computer program / instructions are programmed or configured to execute, via a processor, the steps of the method for optimizing direct I / O read performance under the Linux system as described in any one of claims 1 to 6.

Images