Data reading methods, data writing methods

Abstract

This specification provides a data reading method and a data writing method. The data reading method includes: in response to a data reading request sent by an application, obtaining the current caching strategy; if the current caching strategy is determined to be a first caching strategy, allocating a target cache page that is not added to the page management list, and forwarding the data reading request to the user space file system service process; obtaining and parsing the response data returned from the user space file system service process to obtain the data to be read; writing the data to be read to the target cache page and copying it to user space, and releasing the target cache page. By enabling the first caching strategy, the data to be read is directly written to the target cache page and released after completion, significantly reducing the application blocking phenomenon caused by direct memory reclamation.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data reading method and a data writing method. Background Technology

[0002] In existing operating systems, file read / write operations typically improve performance through a page cache mechanism. This mechanism first reads and writes data to the page cache in memory, leveraging the high speed of memory to accelerate access. However, this memory-based caching approach has inherent drawbacks. As the operating system runs longer, the page cache continuously consumes a large amount of memory, reducing the operating system's free memory. When memory resources are scarce, new cache read / write operations must wait for the operating system to perform direct memory reclamation to free up memory space before they can proceed. This synchronous operation blocks the application's execution flow. Especially in high-performance storage device scenarios, due to the extremely high I / O (Input / Output) performance of high-performance storage devices, the page cache is quickly filled, leading to frequent memory reclamation operations, which in turn causes periodic fluctuations in application performance, affecting service quality. Existing caching architectures face severe challenges in meeting the demands of high-performance storage. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a data reading method. One or more embodiments of this specification also relate to a data reading device, a data writing method, a data writing device, a computing device, a computer-readable storage medium, and a computer program product, in order to solve the technical defects existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data reading method is provided, applied to a user-space file system kernel module, comprising:

[0005] In response to a data read request sent by an application, the current caching strategy is obtained, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0006] If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the page management list is allocated, and the data read request is forwarded to the user space file system service process.

[0007] Obtain and parse the response data returned from the user space file system service process to obtain the data to be read;

[0008] Write the data to be read into the target cache page and copy it to the user space, then release the target cache page.

[0009] According to a second aspect of the embodiments of this specification, a data reading device is provided, applied to a user-space file system kernel module, comprising:

[0010] The first strategy acquisition module is configured to acquire the current caching strategy in response to a data read request sent by the application, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0011] The first cache allocation module is configured to allocate a target cache page that is not added to the page management list when the current cache policy is determined to be the first cache policy, and forward the data read request to the user space file system service process.

[0012] The first data acquisition module is configured to acquire and parse the response data returned from the user space file system service process to obtain the data to be read;

[0013] The first cache release module is configured to write the data to be read into the target cache page and copy it to the user space, thereby releasing the target cache page.

[0014] According to a third aspect of the embodiments of this specification, a data writing method is provided, applied to a user-space file system kernel module, comprising:

[0015] In response to a data write request sent by an application, the current caching strategy is obtained, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0016] If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the page management list is allocated, and the data to be written is written to the target cache page;

[0017] Return a write success instruction to the application and forward the data write request to the user space file system service process;

[0018] Obtain and parse the write response returned from the user space file system service process. If the write response indicates that the data to be written has been successfully written back, release the target cache page.

[0019] According to a fourth aspect of the embodiments of this specification, a data writing apparatus is provided, applied to a user-space file system kernel module, comprising:

[0020] The second strategy acquisition module is configured to acquire the current caching strategy in response to a data write request sent by the application, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0021] The second cache allocation module is configured to allocate a target cache page that is not added to the page management list when the current cache strategy is determined to be the first cache strategy, and to write the data to be written to the target cache page.

[0022] The request forwarding module is configured to return a write success instruction to the application and forward the data write request to the user space file system service process;

[0023] The second cache release module is configured to obtain and parse the write response returned from the user space file system service process, and release the target cache page if it is determined from the write response that the data to be written has been successfully written back.

[0024] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0025] Memory and processor;

[0026] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the above-mentioned data reading method or data writing method.

[0027] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions, which, when executed by a processor, implement the steps of the above-described data reading method or data writing method.

[0028] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data reading method or data writing method.

[0029] One embodiment of this specification implements a data reading method applied to a user-space file system kernel module, comprising: responding to a data reading request sent by an application, obtaining a current caching strategy, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure; if the current caching strategy is determined to be a first caching strategy, allocating a target cache page that is not added to the page management list, and forwarding the data reading request to the user-space file system service process; obtaining and parsing the response data returned from the user-space file system service process to obtain the data to be read; writing the data to be read into the target cache page and copying it to user space, and releasing the target cache page.

[0030] Specifically, this data reading method achieves dynamic decision-making and execution of caching strategies through periodic detection of data processing load pressure by the FUSE kernel module. When the first caching strategy is determined to be enabled, the FUSE kernel module can directly write the data to be read into the target cache page that is not added to the page management list, and release the target cache page after completion, significantly reducing application blocking caused by direct memory reclamation. Especially in high-throughput I / O scenarios of high-performance storage devices, it can provide applications with a stable, low-latency data access experience, significantly improving the overall service quality and resource utilization of the operating system. Attached Figure Description

[0031] Figure 1 This specification illustrates a schematic diagram of a data processing architecture provided in one embodiment.

[0032] Figure 2 A flowchart of a data reading method provided in one embodiment of this specification is shown;

[0033] Figure 3 This diagram illustrates the allocation mechanism of the target cache page in a data reading method according to an embodiment of this specification;

[0034] Figure 4 This diagram illustrates two independent control paths executing a first caching strategy in a data reading method provided in one embodiment of this specification;

[0035] Figure 5 A schematic diagram illustrating the processing steps of a data reading method according to an embodiment of this specification is shown;

[0036] Figure 6 A flowchart of a data writing method provided in one embodiment of this specification is shown;

[0037] Figure 7 A schematic diagram illustrating the processing steps of a data writing method according to an embodiment of this specification is shown;

[0038] Figure 8 This specification shows a schematic diagram of the structure of a data reading device according to one embodiment;

[0039] Figure 9 This specification shows a schematic diagram of the structure of a data writing device according to one embodiment;

[0040] Figure 10 A structural block diagram of a computing device provided in one embodiment of this specification is shown. Detailed Implementation

[0041] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0042] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0044] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0045] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0046] Page cache: The page cache used by the operating system kernel to cache file data.

[0047] DirectReclaim: Direct memory reclamation. When the operating system is short on memory, the application's I / O context needs to release free memory through direct memory reclamation before performing I / O reads and writes. Since DirectReclaim is a synchronous operation, it will cause I / O jitter and affect performance.

[0048] QoS: Quality of Service.

[0049] FUSE: FilesysteminUserspace, a kernel component that allows users to create custom file systems in user space.

[0050] LRU: Least Recently Used, a kernel scheme for managing memory page access.

[0051] CPU: Central Processing Unit.

[0052] DirectIO: Direct I / O bypasses the page cache and directly reads and writes data to the disk. It is often used in database scenarios.

[0053] BufferIO: Buffered I / O. Buffered reads first read data from the disk into the page cache, then copy it to the user-space buffer. Buffered writes first write data to the page cache, then write it back to the disk. The aim is to reduce disk access by utilizing the memory page cache, thereby improving read and write performance.

[0054] RDMA: Remote Direct Memory Access, is a high-performance network communication technology that allows one computer to directly access the memory of another computer without the intervention of the operating system kernel and the participation of the CPU, thereby achieving ultra-low latency and high throughput data transmission.

[0055] FUSE is a widely used general-purpose kernel component that allows users to create custom file systems in user space. A new parallel file system client architecture built on FUSE fully leverages the advantages of user-space architecture: deployment requires no kernel compilation or system restart, significantly improving operational flexibility; the inherent isolation of user-space processes provides fault and security isolation guarantees for multi-tenant scenarios; and it supports diverse user-space caching strategies. Currently, projects and open-source frameworks using this type of file system commonly employ cache read / write, but cache read / write performance degrades drastically and becomes uncontrollable when memory usage is high. This problem is mainly caused by the following factors:

[0056] First, performance bottlenecks caused by direct memory reclamation: Cache read / write operations require data to be read and written to the kernel page cache first. When the page cache memory usage is high, subsequent read / write operations will trigger a direct memory reclamation process within the application process context. This reclamation process is synchronous, meaning the application must wait for the reclamation to complete before continuing execution, directly leading to a decrease in the application's read / write performance and a loss of service quality. This problem is particularly prominent in high-speed devices or high-performance distributed storage systems, as these devices can generate memory pressure more quickly.

[0057] Second, lock contention caused by page cache management overhead: When page cache usage is too high, the overhead of the kernel managing these cached pages becomes significant. Specifically, when allocating and releasing page cache, pages need to be frequently added to or removed from the LRU list. The LRU list is an important data structure used by the kernel to manage memory page reclamation, and operations on these lists require acquiring a global lock. Therefore, under high-pressure scenarios, intense lock contention will occur, further degrading system performance.

[0058] Third: Single-threaded write-back architecture limitations and bandwidth: In the kernel's write-back framework, each storage device is equipped with only a single write-back thread responsible for writing dirty pages to the storage device. This design limits the write-back bandwidth to the processing power of a single CPU core. Since the kernel's dirty page balancing mechanism limits the application's write rate when dirty pages accumulate to a certain threshold, this single-threaded write-back architecture effectively becomes a write performance bottleneck, directly restricting the maximum bandwidth that the application can achieve in cache write scenarios.

[0059] Based on this, the following solutions exist:

[0060] HybridIO dynamically switches between BufferIO and DirectIO through internal file system strategies, selecting the optimal I / O path based on the actual runtime state to improve performance. Compared to directly using DirectIO, HybridIO avoids requiring applications to modify their read / write methods while maintaining standard read / write I / O semantics.

[0061] However, there is a risk of contention when dynamically switching between BufferIO and DirectIO for the same file. When a file is accessed simultaneously through two different paths, data inconsistency may occur due to cache inconsistency. Secondly, DirectIO has an inherent drawback: it cannot support the mmap (mappagesofmemory) system call. Some frameworks, such as PyTorch, load model files via mmap; switching to DirectIO will not cover this use case, limiting the applicability of the solution.

[0062] UncachedIO (unbuffered I / O): While still using the page cache as a data transfer buffer during read and write operations, the corresponding page cache is immediately invalidated and reclaimed. Specifically, for read operations, the corresponding page cache is invalidated and reclaimed immediately after data is copied from the page cache to the user buffer; for write operations, a write-back is triggered immediately after data is written to the page cache, and the corresponding page cache is invalidated and reclaimed immediately upon completion of the write-back I / O. This not only preserves the standard semantics of buffer read and write operations and avoids the destructive modifications to application interfaces caused by DirectIO, but also effectively alleviates memory pressure and reduces direct memory reclamation and LRU lock contention issues by promptly reclaiming the page cache.

[0063] However, UncachedIO requires applications to pass in special flags when reading and writing files, which means that existing applications need to be adapted and modified, increasing deployment costs. More importantly, because the page cache is immediately invalidated, the cache cannot be used to speed up operations in scenarios such as repeatedly reading the same data, resulting in significant performance degradation. While this "use-and-discard" caching strategy solves the memory consumption problem, it sacrifices the core value of the caching mechanism—the performance improvement brought by data reuse.

[0064] Writethrough: For buffer write scenarios, it bypasses the constraints of a single write-back thread by initiating the write-back operation directly within the application thread context, enabling multi-threaded parallel write-back and effectively improving write bandwidth. It maintains compatibility with BufferIO semantics and requires no modifications to the application layer.

[0065] However, the Writethrough scheme still uses the Page cache as a write buffer and does not perform invalidation and reclamation operations on the Page cache, causing the Page cache to continuously occupy system memory. This makes it unable to solve the direct memory reclamation problem caused by excessive Page cache usage, nor can it alleviate the system burden of high LRU list traversal overhead. Therefore, while improving write performance, this scheme fails to fundamentally solve the system performance bottleneck caused by memory management overhead.

[0066] To address the aforementioned technical problems, this specification provides a data reading method in its embodiments. One or more embodiments of this specification also relate to a data reading device, a data writing method, a data writing device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0067] See Figure 1 , Figure 1 A schematic diagram of a data processing architecture provided by one embodiment of this specification is shown.

[0068] Figure 1 This is a schematic diagram of a data processing architecture. The data reading and data writing methods provided in one or more embodiments of this specification are all implemented based on this data processing architecture.

[0069] This data processing architecture diagram describes the complete flow of data from the application to the remote distributed storage system.

[0070] Specifically, the application initiates an I / O request through system calls (such as read and write); the data corresponding to the I / O request is first written to the kernel's page cache; then it is forwarded through the FUSE kernel module (i.e., the user-space file system kernel module). The FUSE kernel module acts as a communication channel between the kernel and the user-space file system, forwarding the I / O request to the FUSE service process; the FUSE service process further uses the RDMA (Remote Direct Memory Access) network protocol to perform efficient and low-latency data read and write interactions with the remote distributed storage system.

[0071] See Figure 2 , Figure 2 A flowchart of a data reading method provided in one embodiment of this specification is shown. This data reading method is applied to a user-space file system kernel module and specifically includes the following steps.

[0072] Step 202: In response to a data read request sent by the application, obtain the current caching policy.

[0073] The data reading method provided in one or more embodiments of this specification can be applied to various scenarios requiring high-performance data reading and writing, including but not limited to large model training scenarios (such as large-scale data reading and writing), high-performance computing scenarios (such as scientific computing, gene sequencing, etc.), large-scale data analysis scenarios, video processing and rendering scenarios, etc., which will not be exhaustive here. For ease of understanding, the following examples will focus on the application of this data reading method to large model training scenarios for detailed description.

[0074] Specifically, the step of obtaining the current caching strategy in response to a data read request sent by the application includes:

[0075] Receive and parse data read requests sent by the application to obtain the file descriptor of the target file containing the data to be read;

[0076] If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

[0077] A data read request can be understood as a file data retrieval request initiated by an application through a system call interface provided by the operating system (such as read()). This data read request enters kernel mode through the read() system call, triggering a switch from user space to kernel space.

[0078] A file descriptor can be understood as a unique identifier for an open file instance within a process. This file descriptor can be understood as an index of the process's file descriptor table, pointing to the fuse_file structure maintained by the kernel. This structure contains the file's open mode (such as read-only, write-only, etc.), the current read / write position, etc. In the embodiments of this specification, this structure also contains a target cache flag returned from the user-space file system service process when the target file is opened. This target cache flag is used to determine the caching strategy for the data read request.

[0079] Specifically, the FUSE kernel module receives and parses data read requests sent by applications to obtain the file descriptor of the target file containing the data to be read. This can be understood as the FUSE kernel module receiving the data read request from the VFS (Virtual File System) layer, parsing the request, extracting the corresponding file descriptor, and then using this file descriptor as an index to search the current process's file descriptor table. The lookup table identifies the `fuse_file` structure associated with that file descriptor, and the presence of the target cache flag is determined based on this `fuse_file` structure. In practical applications, after the application sends the data read request, the file identification information (such as file path, unique identifier, handle, etc.) contained in the request can be obtained, and then the file descriptor of the target file can be obtained based on this textual identification information.

[0080] Secondly, if it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

[0081] The current caching strategy is dynamically determined based on the periodically detected data processing load pressure.

[0082] Specifically, the target cache flag can be understood as a caching policy identifier set by the user-space file system service process and returned to the kernel when the target file is opened. For example, the target cache flag could be the FOPEN_DONTCACHE (do not cache when opening a file) flag. This target cache flag is generally stored as a flag in the fuse_file structure to instruct the FUSE kernel module that all subsequent I / O operations on this file descriptor bypass the page cache mechanism.

[0083] Periodic checks can be understood as checks performed by the FUSE kernel module at preset time intervals.

[0084] Data processing load pressure can be understood as a supply and demand indicator of the system's I / O subsystem processing capacity. It is calculated using the page cache dirtying rate (the number of newly added dirty pages per unit time, reflecting the intensity of data generation in the workload) and the page write-back rate (the number of dirty pages successfully written back to storage per unit time, characterizing the storage system's processing capacity). In practical applications, data processing load pressure can also be quantified by the dynamic backlog of the I / O request queue. Specifically, it is the ratio between the number of I / O requests submitted to the block device layer per unit time and the number of I / O requests actually completed, weighted and adjusted in conjunction with the average I / O response latency.

[0085] Specifically, before obtaining the current caching strategy, the process also includes:

[0086] Memory statistics are obtained according to a preset time period, and page modification rate and page write-back rate are calculated based on the memory statistics.

[0087] The data processing load pressure is determined based on the page modification rate and the page write-back rate.

[0088] Based on the data processing load pressure, determine the first caching strategy or the second caching strategy;

[0089] The step of obtaining the current caching strategy includes:

[0090] Determine the current time point corresponding to receiving the data read request, and determine the first caching strategy or the second caching strategy within the time period corresponding to the current time point as the current caching strategy.

[0091] The preset time period can be set according to actual needs. For example, the preset time period can be set to 200ms, that is, 200ms is one time period.

[0092] Memory statistics can be understood as a set of dynamic data about the operating system's memory usage provided by the operating system kernel, including but not limited to the number of dirty pages (pages that have been modified but not yet written back to memory) and the number of pages currently being written back.

[0093] Page modification rate, or page cache dirty rate, is the rate at which the application writes data to the page cache, which is also the rate at which dirty pages are generated. In other words, the page modification rate is the number of new dirty pages added per unit of time, reflecting the pressure on the application to write data. The calculation formula is (current number of dirty pages - last number of dirty pages) / time interval, used to represent the frequency of application modifications to the page cache and the speed of data generation.

[0094] Page write-back rate, which can be understood as the rate at which dirty pages are flushed back, is the rate at which the kernel flushes dirty pages to the storage device. Specifically, it's the number of pages successfully persisted to storage per unit of time, calculated as (current write-back count - previous write-back count) / time interval. This page write-back rate reflects the storage system's maximum processing capacity and actual throughput.

[0095] Data processing load pressure can be understood as a quantitative indicator of the supply and demand relationship of system I / O processing capacity, determined by comparing the page modification rate and the page write-back rate. If the dirty page rate consistently exceeds the flush rate (i.e., the page modification rate is greater than the page write-back rate), it means that dirty pages are accumulating, and the operating system cannot write data to the storage device in a timely manner, resulting in high I / O pressure. If the dirty page rate approaches or reaches the single-threaded flush bandwidth limit, it indicates that the current I / O pressure is already very high. Therefore, when the I / O pressure is detected to have reached the single-threaded bandwidth limit, the FUSE kernel module can decide to enable the dontcache strategy (i.e., the first caching strategy). Enabling dontcache can prevent the accumulation of dirty pages in the page cache, reducing memory pressure.

[0096] If the page modification rate is less than or equal to the page write-back rate, then the current caching strategy can be determined to be the second caching strategy, which is the standard read-write strategy with kernel page caching that does not use the first caching strategy, i.e., the operating system's default strategy.

[0097] Specifically, the FUSE kernel module continuously collects key statistical information from the kernel memory management subsystem at a preset time period. By calculating the rate of change in the number of dirty pages and write-back pages within adjacent periods, it quantifies the system's page modification rate and page write-back rate. Secondly, based on a comparative analysis of the page modification rate and page write-back rate, it dynamically assesses the current system's data processing load: when the page modification rate consistently exceeds the page write-back rate, the system is determined to be under high I / O pressure, and the first caching strategy (dontcache) is activated; otherwise, the second caching strategy (writeback) is adopted.

[0098] In practical applications, when the FUSE kernel module receives a data read request, it determines which time period the time point corresponding to the data read request falls within. Then, within that time period, it uses either the first caching strategy or the second caching strategy determined by the data processing load pressure as the current caching strategy for processing the data read request.

[0099] The data reading method provided in one or more embodiments of this specification allows the FUSE kernel module to automatically and dynamically switch seamlessly between a first caching strategy and a second caching strategy based on real-time load performance metrics (memory statistics), without any manual intervention or preset. By periodically collecting memory statistics and calculating page modification rate and page write-back rate, the FUSE kernel module intelligently determines the current system state: when it detects that the page modification rate is consistently higher than the page write-back rate (indicating that the data generation speed far exceeds the storage system's persistence capability, and memory is about to become a bottleneck), the FUSE kernel module automatically determines to enable the first caching strategy, avoiding the risk of memory overflow caused by a large number of dirty pages accumulating in the kernel page cache, and eliminating unnecessary data copying, ensuring the stability and response speed of the operating system. Conversely, when the load is stable and write-back capability is sufficient, the FUSE kernel module automatically switches back to the second caching strategy to utilize caching to improve read and write efficiency. This gives the storage system high adaptability, enabling it to intelligently match drastically fluctuating I / O loads in scenarios such as AI training, maximizing overall throughput performance while ensuring data reliability.

[0100] Step 204: If the current caching strategy is determined to be the first caching strategy, allocate a target cache page that is not added to the page management list, and forward the data read request to the user space file system service process.

[0101] The first caching strategy can be understood as a no-caching strategy (i.e., the dontcache strategy). Based on this first caching strategy, the FUSE kernel module can determine that when processing I / O requests for a specific file or file system, it can completely bypass the kernel's own page caching mechanism.

[0102] The target cache page can be understood as a temporary kernel memory page in the kernel. After data enters the kernel from the user space file system (via / dev / fuse), it needs to be temporarily stored in a place before it can be copied back to the user space buffer of the application that initiated the data read request. This temporary page is the target cache page.

[0103] The page management list can be understood as a data structure in the kernel used to manage the lifecycle of page caches; for example, the page management list might be an LRU list. Not adding a page to the page management list can be understood as not adding it to the LRU list. In practice, the LRU list is the data structure used by the kernel to track and manage all cached pages. Adding a page to the list tells the memory management system: "This data is valid cached; please keep it safe and reclaim or write it back when needed." Not adding it explicitly indicates that this page is "disposable" and not intended for persistent cache. The kernel's memory management subsystem will ignore the existence of this target cache page.

[0104] The user-space file system service process can be understood as a file system daemon process running in user space, based on the FUSE framework. This user-space file system service process is the FUSE service process, which establishes a bidirectional communication channel with the FUSE kernel module through the / dev / fuse character device.

[0105] Forwarding data read requests to the user-space file system service process can be understood as the FUSE kernel module converting the VFS layer's data read request into a FUSE protocol message and transmitting it to user space through a communication channel. Specifically, the FUSE kernel module encapsulates the data read request into a FUSE_READ request message according to the FUSE protocol format, and then forwards the encapsulated FUSE_READ request message to the FUSE service process through the / dev / fuse character device.

[0106] In practice, the FUSE kernel module determines the corresponding fuse_file structure based on the file descriptor, checks the open flag field in the fuse_file structure to determine whether the target cache flag is set, and if the target cache flag is not set, it dynamically determines the current caching strategy based on the periodically detected data load pressure. If the current caching strategy is determined to be the first caching strategy, it allocates a target cache page that is not added to the page management list and forwards the data read request to the user space file system service process.

[0107] Step 206: Obtain and parse the response data returned from the user space file system service process to obtain the data to be read.

[0108] The response data can be understood as the message returned by the FUSE service process to the FUSE kernel module through the / dev / fuse character device after receiving and processing the data read request. This response data follows the FUSE protocol format and consists of a header and a body. The header may include data attribute information of the data to be read, and the body may include the data content of the data to be read, i.e., the file content obtained from the distributed storage system.

[0109] In practice, the response data returned by the user space file system service process is obtained and parsed to obtain the data to be read. This can be understood as follows: after the FUSE service process completes the data read request, it writes the encapsulated response data to the / dev / fuse character device. The FUSE kernel module obtains the response data, parses it, and extracts the data to be read from it.

[0110] Step 208: Write the data to be read into the target cache page and copy it to the user space, then release the target cache page.

[0111] User space can be understood as the memory address space where the application runs. It is isolated from the kernel space, and the application's data buffer is located in this user space.

[0112] Specifically, after the FUSE kernel module parses the data to be read from the response data, it first fills the data into a special target cache page allocated by the memory management subsystem. This target cache page has data carrying capacity but is marked as not to be added to the system page management list, thus avoiding LRU lock contention. Then, the data to be read in this target cache page is transferred to the application's user-space buffer via a safe copy operation. After the data delivery is complete, the target cache page is released, reclaiming memory resources.

[0113] The data reading method provided in one or more embodiments of this specification achieves dynamic decision-making and execution of caching strategies through periodic detection of data processing load pressure by the FUSE kernel module. When it is determined that the first caching strategy is enabled, the FUSE kernel module can directly write the data to be read into the target cache page that is not added to the page management list and release it after completion, significantly reducing application blocking caused by direct memory reclamation. Especially in high-throughput I / O scenarios of high-performance storage devices, it can provide applications with a stable, low-latency data access experience, significantly improving the overall service quality and resource utilization of the operating system.

[0114] In another implementation, when the FUSE kernel module determines the existence of a target cache marker based on the file descriptor, it can directly enter the first caching strategy to complete the data service request. The specific implementation is as follows:

[0115] After obtaining the file descriptor of the target file containing the data to be read, the process further includes:

[0116] If the target cache tag is found in the file descriptor, a target cache page is allocated that is not added to the page management list, and the data read request is forwarded to the user space file system service process.

[0117] In practice, when the FUSE kernel module detects the pre-set FOPEN_DONTCACHE flag (i.e., the target cache flag) in the file descriptor, it immediately initiates the optimization process: First, it allocates an independently managed target cache page. This target cache page has data carrying capacity but is not added to the page management list, thereby avoiding LRU lock contention; second, it forwards the data read request to the FUSE service process to obtain the data to be read; after receiving the reply data, it writes the data to be read into the target cache page and copies it to the application user space through a secure copy mechanism; then, it releases the temporary target cache page, completing resource reclamation.

[0118] For the communication method between the FUSE kernel module and the FUSE service process, please refer to the above embodiments, and it will not be repeated here.

[0119] The data reading method provided in one or more embodiments of this specification enables a dedicated caching strategy when the FUSE kernel module detects a target cache marker in a file descriptor. By allocating a target cache page that is not added to the page management list, it avoids global lock contention and significantly improves system scalability in high-concurrency scenarios. Furthermore, the data transfer path adopts a streamlined "allocation-write-copy-release" pipeline, ensuring that the data to be read can quickly pass through the kernel without causing memory residency. Suitable for streaming data access in scenarios such as AI training, it not only avoids memory pressure caused by cache pollution but also enables immediate resource reclamation, providing stable data reading performance for applications.

[0120] See Figure 3 , Figure 3 This diagram illustrates the allocation mechanism of the target cache page in a data reading method provided in one embodiment of this specification.

[0121] Figure 3 This document illustrates the management mechanism for target cache pages in the data reading method provided in this embodiment. Normally, when allocating a page cache (target cache page), it needs to be added to the LRU list, and when an invalid page cache is used, it needs to be removed from the LRU list. Both of these operations involve accessing the global LRU list, and frequent page cache allocation and release can lead to intense contention for this LRU list, impacting operating system performance. Therefore, in the data reading method provided in this embodiment, by allocating an independent target cache page, the target cache page bypasses the global LRU list mechanism (i.e., the target cache page is neither added to nor removed from the LRU list during allocation), thereby eliminating the lock contention overhead and significantly improving system performance in high-concurrency I / O scenarios.

[0122] Furthermore, to achieve smoother and more intelligent memory reclamation and improve operating system performance and stability, a delayed release task is created when releasing the target cache page. The specific implementation is as follows:

[0123] The step of writing the data to be read into the target cache page and copying it to user space, and then releasing the target cache page, includes:

[0124] The data to be read is written to the target cache page and copied to the user space buffer of the application.

[0125] Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

[0126] The delayed release task can be understood as an internal task created by the FUSE kernel module. This internal task contains the information required for the release operation (such as the address of the target cache page) and sets a preset delay time.

[0127] The preset delay time can be understood as a predefined time interval (such as 10 minutes or 20 minutes). This preset delay time determines the time interval between the creation of the delayed release task and the actual execution of the release of the target cache page.

[0128] Specifically, after acquiring the data to be read, the FUSE kernel module first writes the data to the target cache page and then transfers it to the application's user-space buffer via a safe copy operation. Next, after confirming that the copy of the data from the target cache page to user space is complete, the FUSE kernel module does not immediately perform a memory release operation. Instead, it creates a delayed release task for that target cache page. This delayed release task encapsulates the reference information of the target cache page and a preset delay time, and is submitted for asynchronous execution via the kernel work queue. When the preset delay time arrives, the work queue scheduler triggers the task callback function, executing the page release operation for the target cache page and returning the physical memory page completely to the memory manager.

[0129] The data reading method provided in one or more embodiments of this specification achieves smooth and intelligent memory reclamation by introducing a delayed release task and a preset delay time, thereby improving operating system performance and stability. Specifically, the introduction of the delayed release task means that the target cache page is not released immediately after data is read or written. Instead, the target cache page is allowed to remain in memory for a preset delay time to cope with sudden I / O and reduce repeated read overhead. For example, in scenarios such as AI training, the workload often exhibits significant fluctuations. Temporary data written in one computation step may be read immediately by the next step in the next second. If the same read request arrives within the delay window, the data can be quickly retrieved directly from the target cache page in the kernel, avoiding triggering remote storage read operations again, reducing I / O latency and improving overall throughput. Furthermore, it can avoid instantaneous resource contention, i.e., the immediate and concentrated release of a large number of target cache pages can bring instantaneous pressure to the memory management subsystem, which may lead to CPU utilization spikes and system response delays. Delayed release allows this centralized release operation to be performed at different times, effectively smoothing the system load, avoiding performance fluctuations caused by resource contention, and ensuring the smoothness and stability of application operation.

[0130] In practice, the target cache is set by the FUSE service process for the target file when the application opens it. The specific implementation method is as follows:

[0131] Before receiving and parsing the data read request sent by the application, the process also includes:

[0132] Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process;

[0133] The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process for the target file according to a preset setting rule.

[0134] The open request can be understood as a file access initialization operation initiated by the application through the open() system call.

[0135] Preset setting rules can be understood as static policy rules predefined within the FUSE service process, such as setting rules based on file type (i.e., determining whether to set the target cache mark for the target file based on the file type) and setting rules based on path characteristics (i.e., determining whether to set the target cache mark for the target file based on the path characteristics), etc.

[0136] Recording the target cache mark to the file descriptor of the target file can be understood as persistently recording the target cache mark to the kernel data structure of the file descriptor of the target file.

[0137] Specifically, the implementation of setting the cache flag for the Open operation under the FUSE architecture is as follows: The FUSE kernel module receives the open() system call from the application at the VFS layer, encapsulates the request into a FUSE_OPEN message in the FUSE protocol format, and writes the message to the user space communication queue through the / dev / fuse character device. The FUSE service process retrieves the FUSE_OPEN message from the queue, performs static policy judgment based on preset rules (file path matching, file extension recognition, etc.), and if it determines that the non-caching policy needs to be enabled, it sets the FOPEN_DONTCACHE flag (target cache flag) in the fuse_file structure to be returned. The FUSE kernel module receives and parses the reply message returned by the FUSE service process for the open request. If it detects that the FOPEN_DONTCACHE flag is set in the reply message, it persists the FOPEN_DONTCACHE flag in the fuse_file->open_flags field of the struct fuse_file structure created for the target file. Then, it returns the file descriptor corresponding to the fuse_file structure to the application, completing the Open operation. For all subsequent read / write operations through this file descriptor, the FUSE kernel module can quickly determine whether to use a non-cached path by checking fuse_file->open_flags.

[0138] The data reading method provided in one or more embodiments of this specification advances the caching strategy decision to the file opening stage and persists it in the file descriptor. When the application subsequently initiates data read / write, the FUSE kernel module only needs one efficient kernel-level mark check to determine whether to adopt the non-caching strategy, thus avoiding additional communication with user space and complex dynamic decision-making overhead.

[0139] Meanwhile, regarding the determination of the dynamic caching strategy, users can enable this scheme by attaching parameters to `dontcache` (no caching strategy / first caching strategy). The specific implementation method is as follows:

[0140] Before receiving and parsing the data read request sent by the application, the process also includes:

[0141] In response to the mount command, receive and parse the mount parameters carried in the mount command;

[0142] If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

[0143] The mount command can be understood as the file system mount command provided by the operating system, including but not limited to mount parameters, mount point and other information. The mount command is called to enter the kernel and trigger the file system initialization process.

[0144] Mount parameters can be understood as the configuration parameters specified after the -o option in the mount command, which are used to control the behavior of the file system. In the FUSE architecture, mount parameters can be understood as the dontcache cache mount parameters.

[0145] Cache mount parameters can be understood as parameters that enable dynamic caching strategy management. For example, the dontcache mount parameter is used to instruct the file system to dynamically adjust cache behavior based on system load, and is used to start the dynamic adaptive caching decision mechanism of the FUSE service process.

[0146] In practice, the system administrator mounts the file system using the command `mount -o dontcache...`. During the kernel mount process, the FUSE kernel module receives and parses these mount parameters. When the `dontcache` parameter is detected, the FUSE kernel module determines that dynamic caching strategy management needs to be enabled on this file system instance. Based on the parsed cache mount parameters, the FUSE kernel module internally initiates a dedicated periodic detection mechanism. This mechanism runs automatically according to a preset time period (e.g., 200ms), completely independent of the user-space file system service process. In each detection cycle, the FUSE kernel module directly reads the statistics exposed by the kernel memory management subsystem, calculates the page modification rate and page write-back rate, and dynamically determines the global caching strategy (i.e., decides whether to enable `dontcache`) based on this data and the processing load pressure. This decision result is then stored in kernel memory (e.g., in a global flag).

[0147] The data reading method provided in one or more embodiments of this specification achieves intelligent management of system resources by triggering a dynamic caching strategy mechanism through mounted parameters. It transforms static cache configuration into adaptive decisions based on real-time load, enabling the FUSE kernel module to dynamically adjust cache behavior according to actual I / O pressure. Under high load scenarios, it automatically enables direct I / O mode, effectively avoiding high-frequency memory reclamation triggered by rapidly filling the page cache, thus eliminating performance fluctuations in the application. Under normal load, it retains the standard caching mechanism, fully utilizing memory acceleration to improve data access performance. The entire process is completely transparent to the application, achieving continuous performance optimization without manual intervention, significantly reducing system management complexity.

[0148] See Figure 4 , Figure 4 This diagram illustrates two independent control paths executing a first caching strategy in a data reading method provided in one embodiment of this specification.

[0149] Figure 4 The diagram illustrates two independent control paths that trigger the first caching strategy. The left path implements a dynamic caching strategy. System administrators pass caching control parameters to the FUSE kernel module using the `mount -o dontcache` command when mounting the file system. This FUSE kernel module then enables an adaptive algorithm, periodically monitoring the system's dirty page rate and dirty page write-back rate, and dynamically deciding whether to enable the first caching strategy based on the system I / O pressure reflected by these two metrics. The right path implements a static caching strategy. When an application opens a file, the FUSE service process internally sets an open flag (FOPEN_DONTCACHE) for that file according to preset rules. This flag is communicated to the FUSE kernel module via the FUSE protocol, thereby forcing the first caching strategy to be enabled for all subsequent operations on that file.

[0150] See Figure 5 , Figure 5 This diagram illustrates the processing steps of a data reading method provided in one embodiment of this specification.

[0151] Specifically, the FUSE kernel module receives data read requests sent by the application from the VFS layer. The FUSE kernel module first parses the data read request, obtains its corresponding file descriptor, and checks whether the FOPEN_DONTCACHE static flag is pre-set in the kernel data structure of the file descriptor.

[0152] If a static tag exists, the FUSE kernel module can immediately determine to use the dontcache strategy, allocate a target cache page that is not added to the LRU management list (i.e., allocate a page cache), encapsulate the data read request into a FUSE protocol message, forward it to the user-space FUSE service process through the / dev / fuse character device, and wait for its reply.

[0153] If the static tag does not exist, the FUSE kernel module determines the current caching strategy based on its dynamic monitoring algorithm. If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the LRU management list is allocated (i.e., page cache is allocated), and the data read request is encapsulated into a FUSE protocol message and forwarded to the FUSE service process in user space through the / dev / fuse character device, and then waits for its reply.

[0154] After the FUSE kernel module forwards the data read request to the FUSE service process, the FUSE service process retrieves the data to be read from remote storage and returns the data. The FUSE kernel module then writes the received data to be read into the previously allocated target cache page (i.e., fills the page cache). Subsequently, the data to be read is copied from the target cache page to a user-space buffer provided by the application (i.e., fills the user-space buffer).

[0155] For all read requests completed through the target cache page, the FUSE kernel module schedules a delayed worker task. This causes the target cache page to be asynchronously invalidated and released after a short delay (i.e., invalidating the page cache). This ensures efficient data reading while effectively controlling kernel memory levels through delayed release, avoiding performance fluctuations caused by frequent memory reclamation.

[0156] See Figure 6 , Figure 6 A flowchart of a data writing method provided in one embodiment of this specification is shown. This data writing method is applied to a user-space file system kernel module and specifically includes the following steps.

[0157] Step 602: In response to a data write request sent by the application, obtain the current caching policy.

[0158] The current caching strategy is dynamically determined based on the periodically detected data processing load pressure.

[0159] Step 604: If the current caching strategy is determined to be the first caching strategy, allocate a target cache page that is not added to the page management list, and write the data to be written to the target cache page.

[0160] Step 606: Return a write success instruction to the application and forward the data write request to the user space file system service process.

[0161] Step 608: Obtain and parse the write response returned from the user space file system service process. If the write response confirms that the data to be written has been successfully written back, release the target cache page.

[0162] Specifically, the implementation of this data write request is basically the same as that of the data read request. The only difference is that the data read request needs to wait for the FUSE service process to return the data to be read before it can write the data to be read to the target cache page, while the data write request can write the data to be written to the target cache page after the target cache page is allocated, without waiting for the FUSE service process to reply.

[0163] In specific implementation, the step of obtaining the current caching strategy in response to a data write request sent by the application includes:

[0164] Receive and parse the data write request sent by the application, and obtain the file descriptor of the target file to be written;

[0165] If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

[0166] A data write request can be understood as a request initiated by an application through a system call (such as write()) to write data to a file.

[0167] A write success instruction can be understood as a success instruction returned to the application immediately after the FUSE kernel module writes the data to be written to the target cache page, without waiting for the data to be written to actually be written to the underlying storage.

[0168] A write response can be understood as the response returned by the FUSE service process to the FUSE kernel module after processing the write request.

[0169] In practice, when an application initiates a data write request, the FUSE kernel module receives the request through the VFS layer. It first checks the static cache marker (i.e., the target cache marker) in the file descriptor obtained by parsing the write request. If no static cache marker exists, it assesses the current data processing pressure based on periodically collected system load metrics (page modification rate and write-back rate) and dynamically decides the current caching strategy based on this pressure. If the current caching strategy is determined to be the first caching strategy, the data to be written is copied from user space to the target cache page, and a write success message is immediately returned to the application, achieving asynchronous write semantics. Simultaneously, the FUSE kernel module forwards the actual data write request to the FUSE service process for persistence processing. Once the write response returned by the FUSE service process is parsed and the write-back of the data is confirmed to be successful, the FUSE kernel module releases the target cache page and reclaims memory resources.

[0170] The data writing method provided in one or more embodiments of this specification achieves intelligent management of memory resources while ensuring write performance through a dynamic caching strategy management mechanism. Specifically, after data is written to the page cache and its persistence is confirmed, the system dynamically decides whether to immediately release the page cache based on the system I / O pressure. This enables timely reclamation of cache space for persisted data under high load scenarios, effectively avoiding memory pressure caused by excessive accumulation of page cache. Furthermore, by determining the strategy based on periodically detected data processing load pressure, it retains the acceleration benefits of standard caching mechanisms under normal load while ensuring system stability and responsiveness under high load.

[0171] If a target cache tag is found in the file descriptor, a target cache page that is not added to the LRU list can be directly allocated, and the data to be written can be written to that target cache page. The FUSE service process only needs to send asynchronous data write requests to achieve asynchronous write-back. The specific implementation method is as follows:

[0172] After obtaining the file descriptor of the target file from which the data to be written is obtained, the process further includes:

[0173] If the target cache tag is found in the file descriptor, a target cache page that is not added to the page management list is allocated, and the data to be written is written to the target cache page.

[0174] In practice, upon receiving the data write request, the FUSE kernel module first checks the `struct fuse_file` structure associated with the file descriptor. If it detects that the `FOPEN_DONTCACHE` target cache flag is set in the `open_flags` field, it determines to adopt the first caching strategy, allocates a target cache page that is not added to the page management list, writes the data to be written to this target cache page, and immediately returns a write success instruction to the application. This achieves asynchronous write semantics, allowing the application to continue execution without waiting for the actual data write persistence to complete. Simultaneously, the FUSE kernel module encapsulates the data write request into a FUSE protocol message and forwards it to the FUSE service process via ` / dev / fuse`. The FUSE service process then writes the data to the remote distributed storage system using protocols such as RDMA. After receiving the write response from the FUSE service process, the FUSE kernel module parses the response and, after confirming that the FUSE service process has completed its write-back operation, releases the target cache page.

[0175] The data writing method provided in one or more embodiments of this specification achieves a high-performance write path through static tag prediction and dedicated cache page management. By allocating dedicated target cache pages that are not added to the LRU list, the write throughput in high-concurrency scenarios is significantly improved; and the "use and release" mechanism ensures that temporary data does not occupy memory resources for a long time, avoiding memory pressure caused by cache pollution; at the same time, by decoupling the asynchronous semantics of returning immediately after a successful write from background persistence, both low-latency response of the application and data consistency are ensured through the asynchronous write-back confirmation mechanism.

[0176] The specific implementation of releasing the target cache page, determining the cache policy identifier, and the specific implementation of the static cache policy and dynamic cache policy can all be referred to the above-described data reading method embodiments, and will not be repeated here.

[0177] Optionally, releasing the target cache page includes:

[0178] Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

[0179] Optionally, the caching policy identifier is determined through the following steps:

[0180] The user space file system service process acquires memory statistics information according to a preset time period, calculates the page modification rate and page write-back rate based on the memory statistics information, and determines the data processing load pressure based on the page modification rate and the page write-back rate.

[0181] Based on the data processing load pressure, determine the cache strategy identifier of the first cache strategy or the cache strategy identifier of the second cache strategy.

[0182] Optionally, before receiving and parsing the data write request sent by the application, the method further includes:

[0183] Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process;

[0184] The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process for the target file according to a preset setting rule.

[0185] Optionally, before receiving and parsing the data write request sent by the application, the method further includes:

[0186] In response to the mount command, receive and parse the mount parameters carried in the mount command;

[0187] If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

[0188] See Figure 7 , Figure 7 This diagram illustrates a data writing method according to an embodiment of this specification.

[0189] Specifically, the FUSE kernel module receives data write requests sent by the application from the VFS layer, parses the data write request, obtains the corresponding file descriptor, and checks whether the FOPEN_DONTCACHE static flag is pre-set in the kernel data structure of the file descriptor.

[0190] If a static flag exists, the FUSE kernel module can determine to use the dontcache strategy, allocate a target cache page that is not added to the LRU management list (i.e., allocate a page cache), copy the data to be written from the user-space buffer to the target cache page (i.e., fill the page cache), immediately return a write success instruction to the application, and encapsulate the data write request into a FUSE protocol message, which is then forwarded to the FUSE service process via / dev / fuse.

[0191] If the static tag does not exist, the FUSE kernel module determines the current caching strategy based on its dynamic monitoring algorithm. If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the LRU management list is allocated (i.e., page cache is allocated), and the data to be written is copied to the target cache page. The write success instruction is immediately returned to the application, and the data write request is encapsulated into a FUSE protocol message and forwarded to the FUSE service process through / dev / fuse.

[0192] Specifically, after receiving the data write request, the FUSE service process can write the data to be written to the remote distributed storage system through protocols such as RDMA (i.e., write back dirty pages), and after persistence is completed, return a write reply to the FUSE kernel module (i.e., write back is complete).

[0193] After confirming a successful write, the FUSE kernel module immediately releases the target cache page that is not added to the LRU list. The release of the target cache page is achieved by scheduling a delayed job task, which is executed asynchronously after a short delay (i.e., scheduling a delayed job task to invalidate the cached page).

[0194] The data writing method provided in one or more embodiments of this specification, through asynchronous write semantics, immediately returns a write success instruction to the application after the data enters the kernel cache, ensuring that the application is not blocked and significantly improving system response speed and overall throughput. Secondly, combined with an intelligent resource reclamation mechanism, cache pages are released in a timely manner after data persistence is completed according to static marking or dynamic strategies, effectively avoiding the invalid occupation of memory resources. At the same time, by utilizing dual-path optimization, performance can be improved by achieving zero-decision through static paths, while the system's adaptability under different loads can be ensured by using dynamic paths, significantly improving system performance, stability, and resource utilization.

[0195] Corresponding to the above method embodiments, this specification also provides embodiments of a data reading device. Figure 8 A schematic diagram of a data reading device according to one embodiment of this specification is shown. Figure 8 As shown, this device is applied to the user-space file system kernel module and includes:

[0196] The first strategy acquisition module 802 is configured to acquire the current caching strategy in response to a data read request sent by the application, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0197] The first cache allocation module 804 is configured to allocate a target cache page that is not added to the page management list when the current cache policy is determined to be the first cache policy, and forward the data read request to the user space file system service process.

[0198] The first data acquisition module 806 is configured to acquire and parse the response data returned from the user space file system service process to obtain the data to be read;

[0199] The first cache release module 808 is configured to write the data to be read into the target cache page and copy it to the user space, thereby releasing the target cache page.

[0200] Optionally, the first strategy acquisition module 802 is further configured to:

[0201] Receive and parse data read requests sent by the application to obtain the file descriptor of the target file containing the data to be read;

[0202] If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

[0203] Optionally, the device further includes:

[0204] The third cache allocation module is configured as follows:

[0205] If a target cache tag is found in the file descriptor, a target cache page is allocated that is not added to the page management list, and the data read request is forwarded to the user space file system service process.

[0206] Optionally, the first cache release module 808 is further configured to:

[0207] The data to be read is written to the target cache page and copied to the user space buffer of the application.

[0208] Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

[0209] Optionally, the device further includes:

[0210] The first strategy determination module is configured as follows:

[0211] Memory statistics are obtained according to a preset time period, and page modification rate and page write-back rate are calculated based on the memory statistics.

[0212] The data processing load pressure is determined based on the page modification rate and the page write-back rate.

[0213] Based on the data processing load pressure, determine the first caching strategy or the second caching strategy;

[0214] The first strategy acquisition module 802 is further configured to:

[0215] Determine the current time point corresponding to receiving the data read request, and determine the first caching strategy or the second caching strategy within the time period corresponding to the current time point as the current caching strategy.

[0216] Optionally, the device further includes:

[0217] The first static tag setting module is configured as follows:

[0218] Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process;

[0219] The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process for the target file according to a preset setting rule.

[0220] Optionally, the device further includes:

[0221] The first dynamic tag setting module is configured as follows:

[0222] In response to the mount command, receive and parse the mount parameters carried in the mount command;

[0223] If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

[0224] The above is an illustrative scheme of a data reading device according to this embodiment. It should be noted that the technical solution of this data reading device and the technical solution of the data reading method described above belong to the same concept. For details not described in detail in the technical solution of the data reading device, please refer to the description of the technical solution of the data reading method described above.

[0225] Corresponding to the above method embodiments, this specification also provides embodiments of a data writing device. Figure 9 A schematic diagram of a data writing device according to one embodiment of this specification is shown. Figure 9 As shown, this device is applied to the user-space file system kernel module and includes:

[0226] The second strategy acquisition module 902 is configured to acquire the current caching strategy in response to a data write request sent by the application, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure;

[0227] The second cache allocation module 904 is configured to allocate a target cache page that is not added to the page management list when the current cache strategy is determined to be the first cache strategy, and to write the data to be written into the target cache page.

[0228] The request forwarding module 906 is configured to return a write success instruction to the application and forward the data write request to the user space file system service process.

[0229] The second cache release module 908 is configured to acquire and parse the write response returned from the user space file system service process, and release the target cache page if it is determined from the write response that the data to be written has been successfully written back.

[0230] Optionally, the second strategy acquisition module 902 is further configured as follows:

[0231] Receive and parse the data write request sent by the application, and obtain the file descriptor of the target file to be written;

[0232] If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

[0233] Optionally, the device further includes:

[0234] The fourth cache allocation module is configured as follows:

[0235] If a target cache tag is found in the file descriptor, a target cache page that is not added to the page management list is allocated, and the data to be written is written to the target cache page.

[0236] Optionally, the second cache release module 908 is further configured to:

[0237] Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

[0238] Optionally, the device further includes:

[0239] The second strategy determination module is configured as follows:

[0240] Memory statistics are obtained according to a preset time period, and page modification rate and page write-back rate are calculated based on the memory statistics.

[0241] The data processing load pressure is determined based on the page modification rate and the page write-back rate.

[0242] Based on the data processing load pressure, determine the first caching strategy or the second caching strategy;

[0243] The second strategy acquisition module 902 is further configured as follows:

[0244] Determine the current time point corresponding to receiving the data write request, and determine the first caching strategy or the second caching strategy within the time period corresponding to the current time point as the current caching strategy.

[0245] Optionally, the device further includes:

[0246] The second static tag setting module is configured as follows:

[0247] Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process;

[0248] The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process for the target file according to a preset setting rule.

[0249] Optionally, the device further includes:

[0250] The second dynamic tag setting module is configured as follows:

[0251] In response to the mount command, receive and parse the mount parameters carried in the mount command;

[0252] If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

[0253] The above is an illustrative scheme of a data writing device according to this embodiment. It should be noted that the technical solution of this data writing device and the technical solution of the data writing method described above belong to the same concept. For details not described in detail in the technical solution of the data writing device, please refer to the description of the technical solution of the data writing method described above.

[0254] See Figure 10 , Figure 10 This specification illustrates a structural block diagram of a computing device according to one embodiment. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0255] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0256] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0257] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0258] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data reading method or data writing method.

[0259] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the data reading method or data writing method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data reading method or data writing method described above.

[0260] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data reading method or data writing method.

[0261] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the data reading method or data writing method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data reading method or data writing method described above.

[0262] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data reading method or data writing method.

[0263] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the data reading method or data writing method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data reading method or data writing method described above.

[0264] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0265] The computer program / instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0266] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0267] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0268] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data reading method, applied to a user-space file system kernel module, comprising: In response to a data read request sent by an application, the current caching strategy is obtained, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure; If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the page management list is allocated, and the data read request is forwarded to the user space file system service process. Obtain and parse the response data returned from the user space file system service process to obtain the data to be read; Write the data to be read into the target cache page and copy it to the user space, then release the target cache page.

2. The data reading method according to claim 1, wherein obtaining the current caching strategy in response to a data reading request sent by the application includes: Receive and parse data read requests sent by the application to obtain the file descriptor of the target file containing the data to be read; If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

3. The data reading method according to claim 2, after obtaining the file descriptor of the target file containing the data to be read, further includes: If the target cache tag is found in the file descriptor, a target cache page is allocated that is not added to the page management list, and the data read request is forwarded to the user space file system service process.

4. The data reading method according to any one of claims 1-3, wherein writing the data to be read into the target cache page and copying it to the user space, and releasing the target cache page, comprises: The data to be read is written to the target cache page and copied to the user space buffer of the application. Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

5. The data reading method according to any one of claims 1-3, further comprising, before obtaining the current caching strategy: Memory statistics are obtained according to a preset time period, and page modification rate and page write-back rate are calculated based on the memory statistics. The data processing load pressure is determined based on the page modification rate and the page write-back rate. Based on the data processing load pressure, determine the first caching strategy or the second caching strategy; The step of obtaining the current caching strategy includes: Determine the current time point corresponding to receiving the data read request, and determine the first caching strategy or the second caching strategy within the time period corresponding to the current time point as the current caching strategy.

6. The data reading method according to claim 2 or 3, further comprising, before receiving and parsing the data reading request sent by the application: Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process; The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process according to a preset setting rule for the target file.

7. The data reading method according to claim 2 or 3, further comprising, before receiving and parsing the data reading request sent by the application: In response to the mount command, receive and parse the mount parameters carried in the mount command; If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

8. A data writing method, applied to a user-space file system kernel module, comprising: In response to a data write request sent by an application, the current caching strategy is obtained, wherein the current caching strategy is dynamically determined by periodically detected data processing load pressure; If the current caching strategy is determined to be the first caching strategy, a target cache page that is not added to the page management list is allocated, and the data to be written is written to the target cache page. Return a write success instruction to the application and forward the data write request to the user space file system service process; Obtain and parse the write response returned from the user space file system service process. If the write response indicates that the data to be written has been successfully written back, release the target cache page.

9. The data writing method according to claim 8, wherein obtaining the current caching strategy in response to a data write request sent by the application includes: Receive and parse the data write request sent by the application, and obtain the file descriptor of the target file to be written; If it is determined that there is no target cache tag in the file descriptor, the current caching policy is obtained.

10. The data writing method according to claim 9, further comprising, after obtaining the file descriptor of the target file from which the data to be written is to be obtained: If the target cache tag is found in the file descriptor, a target cache page that is not added to the page management list is allocated, and the data to be written is written to the target cache page.

11. The data writing method according to any one of claims 8-10, wherein releasing the target cache page comprises: Create a delayed release task for the target cache page, and release the target cache page according to the preset delay time carried in the delayed release task.

12. The data writing method according to any one of claims 8-10, further comprising, before obtaining the current caching strategy: Memory statistics are obtained according to a preset time period, and page modification rate and page write-back rate are calculated based on the memory statistics. The data processing load pressure is determined based on the page modification rate and the page write-back rate. Based on the data processing load pressure, determine the first caching strategy or the second caching strategy; The step of obtaining the current caching strategy includes: Determine the current time point corresponding to receiving the data write request, and determine the first caching strategy or the second caching strategy within the time period corresponding to the current time point as the current caching strategy.

13. The data writing method according to claim 9 or 10, further comprising, before receiving and parsing the data write request sent by the application: Receive the open request for the target file sent by the application, and forward the open request to the user space file system service process; The response message returned by the user space file system service process is obtained and parsed. If the response message carries the target cache tag, the target cache tag is recorded in the file descriptor of the target file. The target cache tag is set by the user space file system service process according to a preset setting rule for the target file.

14. The data writing method according to claim 9 or 10, further comprising, before receiving and parsing the data write request sent by the application: In response to the mount command, receive and parse the mount parameters carried in the mount command; If the mounting parameters are determined to be cache mounting parameters, a periodic detection mechanism is initiated to dynamically determine the caching strategy based on the periodically detected data processing load pressure.

15. A computing device, comprising: A memory and a processor, the memory and the processor being connected via a bus; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1-14.

16. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-14.

17. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-14.

Images