A Client Local Persistent Cache Optimization Method Based on File Page Exchange

Abstract

The present invention relates to a method for optimizing the client local persistent cache based on file page swapping, which is mainly aimed at the characteristics of unstable network and limited device hardware conditions in the consumer-grade scenario, and performs performance optimization for cross-device access of the client of the distributed file system, including: refining the local persistent cache granularity, optimizing the local persistent cache management, and streamlining the local persistent cache call stack. The basic idea is to asynchronously swap the file pages obtained by the client through network requests to the swap partition of the external storage, and when these file pages need to be accessed again, swap them into the memory from the swap partition, so as to reduce network requests and lower access latency. At the same time, taking the file page as the minimum cache granularity and supplemented by a cache management algorithm can save device storage space and improve the cache hit rate. Using the idea of the swap partition and directly using the read and write of the specific file system to perform the swapping in and out of the cache can reduce the overhead between levels and improve the system performance.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed file system performance, and particularly to an optimization method for client local persistent caching based on file page swapping, which optimizes cross-device access for mobile terminal devices according to the characteristics of consumer scenarios. Background Art

[0002] In the past few decades, consumer mobile terminal devices have developed rapidly, the user scale has been continuously expanding, and the user data generated by users in daily use has also been increasing, which has increased the storage pressure on the devices. In order to meet the needs of user data storage and backup, and ensure the efficiency and performance of data access, and ultimately give users a good user experience, the home network attached storage (NAS) service has become more and more popular.

[0003] The home NAS service enables users to upload and back up the user data generated on mobile terminal devices to the NAS server via a wireless local area network, and access and modify it via a WiFi connection when needed. The NAS server establishes connections with various consumer mobile terminal devices to form a distributed system with a star structure (server-client model), which is managed by a relevant distributed file system (DFS). As a client, the consumer mobile terminal obtains the required file pages through network requests and temporarily stores them in the local page cache (belonging to memory). These pages will then be returned to the user-mode application through the virtual file system for the user to use. However, limited by the unstable network conditions and limited device hardware conditions (such as memory space, external storage space, etc.) in consumer scenarios, such cross-device remote file access will be greatly affected in terms of performance and reliability. Through experiments, it is found that when the network quality is poor and the memory space is small, the latency of reading and writing remote files on the client device will increase significantly.

[0004] Currently, mainstream distributed file systems improve the performance and reliability of client remote access by introducing client-side persistent caching. For example, the FSCache middleware that supports distributed file systems such as NFS and AFS can provide interfaces for local persistent function modules such as CacheFS and CacheFiles, helping the distributed file system to persist the obtained remote files to the local external storage. When the file needs to be accessed again, the client device can directly access the local persistent cache without going through the network, improving performance and reducing dependence on the network. Coda also adopts a similar design concept and can fully download and persist the remote file to the local external storage during the open process of the remote file.

[0005] However, the above technical solutions have certain problems in the consumer-grade scenario: The storage space and lifespan of the storage device of consumer-grade mobile devices are limited, and it is not suitable to introduce too much local persistent cache, not suitable for frequent read-write erasure, and the CPU computing power is tense. If an intermediate layer is introduced to process, it will increase the computing and calling overhead, reduce the overall system performance, and affect the user experience.

[0006] Therefore, for the consumer-grade scenario, refining the local persistent cache granularity, optimizing the local persistent cache management, and streamlining the local persistent cache call stack can improve the performance and reliability of the client device's remote access to files under limited environment and hardware conditions, thereby providing a better user experience for users. Summary of the Invention

[0007] In order to overcome the above problems and achieve refined local persistent cache granularity, optimized local persistent cache management, and streamlined local persistent cache call stack, the object of the present invention is to provide a method for optimizing the client local persistent cache based on file page swapping, so as to optimize the performance and user experience for the application of the distributed file system in the consumer-grade scenario.

[0008] The specific technical solution to achieve the object of the present invention is as follows:

[0009] A method for optimizing the client local persistent cache based on file page swapping, which is an optimization of the distributed file system in the consumer-grade scenario, includes the following steps:

[0010] 1) The application program on the client side of the distributed file system in the consumer-grade scenario makes a system call open to open a remote file;

[0011] 2) After the operating system kernel of the client receives the system call request to open the remote file, it sends a network request to the server, and the server sends back a response according to the request;

[0012] 3) When the client receives the response from the server, it determines whether there is a stale version of the file in the local memory according to the inode information of the file index node included in the response; if so, it clears the stale version and enters step 4); if not, it directly enters step 4);

[0013] 4) The client starts the first part of the process of optimizing the management of the client local persistent cache, and determines whether there is a persistent cache of the requested file in the swap partition established in the local external memory; if so, it moves the LRU linked list item corresponding to the file to the front of the linked list, indicating a cache hit; if not, it creates a new LRU linked list item and inserts it at the front of the LRU linked list; after the determination is completed, it enters step 5);

[0014] 5) The operating system kernel of the client returns the information completed by the system call open to the upper-layer application, ending the process of opening the remote file; at this time, the client application sends a system call read to read the content of the remote file;

[0015] 6) After receiving the system call request to read the remote file, the operating system kernel of the client first searches for the required file page in the page cache (page-cache) of the memory; if there is, it enters step 10), and ends the process after completion; if not, it enters step 7);

[0016] 7) The operating system kernel of the client starts to read the file page through the readpage interface provided by the virtual file system. At this time, it starts to judge whether there is a persistent cache of the requested file page in the swap partition of the local external storage; if there is, it swaps the page into the memory through the read interface (kernel_read) of the file system and enters step 10), and ends the process after completion; if not, it enters step 8);

[0017] 8) The client sends a network request to read the file page to the server, and the server sends back a response containing the file page data according to the request;

[0018] 9) After receiving the response, the client places the file page data in the response into the memory for the upper-layer application to read, then enters step 10), and simultaneously executes step 11);

[0019] 10) The operating system kernel of the client returns the information completed by the system call read to the upper-layer application;

[0020] 11) The client starts to perform persistent caching of the file data; the client writes the requested file page into the swap partition of the local external storage through the write interface (kernel_write) of the file system, and performs a record of the page persistent cache. Using the index of the page as the subscript, it finds the corresponding position in the bitmap of the LRU linked list item corresponding to the file, and sets the value of that bit to 1, indicating that the page has been persistently cached;

[0021] 12) The client starts the second part of the process of optimizing and managing the local persistent cache, updating the information about the amount of persistent cache data in the local external storage swap partition; after the requested file page is persistently cached, the amount of persistent cache data in the local external storage swap partition is increased by a PAGESIZE, that is, the size of the file page, which is 4KB, and checks whether the value of the updated amount of persistent cache data exceeds a pre-set threshold. If it exceeds, starting from the tail of the LRU linked list, the file pages of the files corresponding to the LRU linked list items are removed one by one.

[0022] The optimization of the distributed file system in the consumer - level scenario includes various distributed file systems implemented in the kernel state and application state, and is applicable as long as it involves cross - device access to remote files.

[0023] The LRU linked list in the first part of the optimization management of the client - side local persistent cache exists in the client's memory and is used to manage the persistent cache file pages in the external memory swap partition; each entry in the LRU linked list corresponds to a remote file, and the entry content includes file identification, file version information, the number of file pages that have been persistently cached, and a bitmap used to record the persistently cached file pages.

[0024] The persistent cache of the file data takes file pages as the smallest granularity. When recording page - level persistent caching, it includes using a bitmap, a linked list, or a hash table.

[0025] Each entry in the LRU linked list of the first part of the optimization management of the client - side local persistent cache can correspond to a single file page, a single file, or a set of related files; the LRU linked list can be replaced by a FIFO linked list or an LFU linked list.

[0026] In the present invention, all cache data takes file pages as the smallest persistent cache granularity and file pages as the smallest swap unit to achieve fine - grained cache optimization. The least - recently - used cache management method represented by the LRU linked list can achieve more efficient cache management in a limited space, can identify the user's file - usage behavior patterns and file access hotness to a certain extent, and thus provide a higher cache hit rate.

[0027] The beneficial effects of the present invention are manifested in that when the client is in a working state with low network bandwidth and small memory space, the present invention can reduce the latency of the client's cross - device access to remote file data, thereby providing a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the system working path involved in the present invention.

[0029] Figure 2 It is a schematic diagram of the data structure for the optimization management of the client - side local persistent cache involved in the present invention.

[0030] Figure 3 It is a timing flowchart of a specific embodiment involved in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following further describes the present invention with reference to the drawings and embodiments.

[0032] Figure 1Shown is the system working path involved in the present invention. Working path ① represents the interaction between the application program and the virtual file system through the POSIX interface of the operating system. Working path ② represents the interaction between the swap partition and the specific backing file system (such as F2FS, ExtFS, etc.). Working path ③ represents the interaction between the virtual file system and the backing file system. Working path ④ represents the interaction between the backing file system and the external storage, which is the basis for the operating system to access and store data on the external storage medium. Working path ⑤ represents the interaction between the virtual file system and the server part of the distributed file system. Based on path ④, the server can read the local file pages and send the data to the client through the network. Working path ⑥ represents the interaction between the virtual file system and the client part of the distributed file system.

[0033] The client - side local persistent caching technology based on swapping retains the obtained remote file pages by persisting the file pages in memory to the external storage. When the file page cache in the page cache of memory is reclaimed, based on path ②, the client can swap the file pages from the swap partition into memory on the premise of hitting the local persistent cache, thus avoiding network requests. Using the idea of the swap partition, the cached data is exchanged through the virtual file system and persisted to the external storage using a specific backing file system, eliminating the management of the middle layer, streamlining the call stack, and reducing the inter - layer interaction overhead.

[0034] Considering that many files (such as streaming media files) do not need to fetch the entire file in full when being accessed by users, and the data requests are not necessarily in a sequential access mode, the present invention adopts a "cache - on - demand" design. This can not only save the swap space for caching but also reduce the overhead cost of persistent caching, making the cache management more flexible. Figure 2 Shown is the specific method for optimizing the management of client - side local persistent cache data. Use a limited LRU linked list to manage the client - side local persistent cache data in units of files. Each element of this LRU linked list represents a remote file that has been accessed, which includes the file identifier, file version number, locally cached size, etc. of the file. In addition, it also includes a bitmap for recording the cached file pages. Embodiment

[0035] As Figure 3 shown, it is the timing flowchart of a specific embodiment involved in the present invention. It is divided into two sub - processes: the file opening process and the file access process.

[0036] The figure involves three objects: the client application program, the client, and the server.

[0037] The file opening process is used to update the meta - information of the client - side local persistent cache. The specific process is as follows:

[0038] 1. The client application in user mode makes a system call to open a remote file;

[0039] 2. After receiving the upper-layer request, the client's operating system kernel initiates a network request to the remote server and obtains and updates the metadata of the remote file;

[0040] 3. Based on the obtained file information, update and manage the LRU linked list of the client's local persistent cache.

[0041] The file access process is used by the operating system kernel to return the requested data to the upper layer application, which includes the client local persistent cache optimization process added in the present invention, and the process is as follows:

[0042] 1. The client application in user mode makes a system call to read a remote file;

[0043] 2. After receiving the upper-layer request, the client's operating system kernel first checks whether the relevant file page exists in the page cache of the local memory. If so, it directly returns it to the upper-layer application;

[0044] 3. If it does not exist, the file page is queried in the swap partition where the client's local persistent cache is located. If the cache hits, the relevant file page is swapped into the memory and directly returned to the upper layer, thus eliminating the need for remote network requests;

[0045] 4. If the local cache does not hit, the client starts to initiate a network request to obtain the file page data from the remote server, and asynchronously caches the obtained file page locally, swapping it from the memory into the swap partition so that it can be directly accessed the next time. After the persistence is completed, the data volume information of the persistent cache needs to be updated. If it exceeds the space limit, the persistent cache is removed from the end of the LRU linked list until the data volume of the persistent cache does not exceed the space limit.

Claims

1. A method for optimizing client - side local persistent caching based on file page swapping, which is an optimization of a distributed file system in a consumer - level scenario, is characterized in that, It includes the following steps: 1) The client-side application of the distributed file system in the consumer-grade scenario makes a system call open to open a remote file; 2) After the operating system kernel of the client receives the system call request to open the remote file, it sends a network request to the server, and the server sends back a response according to the request; 3) When the client receives the response from the server, it judges whether there is a stale version of the file in the local memory according to the inode information of the file index node included in the response; if so, it clears the stale version and enters step 4); if not, it directly enters step 4); 4) The client starts the first part of the process of optimizing the management of the client's local persistent cache, and judges whether there is a persistent cache of the requested file in the swap partition established in the local external memory; If so, it moves the LRU linked list item corresponding to the file to the front of the linked list, indicating a cache hit; if not, it creates a new LRU linked list item and inserts it at the front of the LRU linked list; after the judgment is completed, it enters step 5); 5) The operating system kernel of the client returns the information that the system call open is completed to the upper-layer application, and ends the process of opening the remote file; at this time, the client application sends a system call read to read the content of the remote file; 6) After the operating system kernel of the client receives the system call request to read the remote file, it first looks up the required file page in the page cache (page-cache) in the memory; If so, it enters step 10), and ends the process after completion; If not, it enters step 7); 7) The operating system kernel of the client starts to read the file page through the readpage interface provided by the virtual file system. At this time, it starts to judge whether there is a persistent cache of the requested file page in the swap partition of the local external memory; If so, it swaps the page into the memory through the read interface (kernel_read) of the file system and enters step 10), and ends the process after completion; If not, it enters step 8); 8) The client sends a network request for reading the file page to the server, and the server sends back a response containing the file page data according to the request; 9) After the client receives the response, it places the file page data in the response into the memory for the upper-layer application to read, then enters step 10), and simultaneously executes step 11); 10) The operating system kernel of the client returns the information that the system call read is completed to the upper-layer application; 11) The client starts to perform persistent caching of the file data; The client writes the requested file page into the swap partition of the local external memory through the write interface (kernel_write) of the file system, and performs page persistent cache recording. Using the index of the page as the subscript, it looks up the corresponding position in the bitmap of the LRU linked list item of the corresponding file, and sets the value of the bit to 1, indicating that the page has been persistently cached; 12) The client starts the second part of the process of optimizing the management of the client's local persistent cache, and updates the information about the amount of persistent cache data in the local external memory swap partition; After the requested file page is persistently cached, the amount of persistently cached data in the local swap partition of the external storage, plus a PAGESIZE, which is the size of the file page and is 4KB, is checked to see if the value of the updated persistently cached data amount exceeds a pre-set threshold. If it does, starting from the tail of the LRU linked list, the file pages of the files corresponding to the LRU linked list items are removed one by one.

2. The client - side local persistent cache optimization method based on file page swapping according to claim 1, wherein The optimization of the distributed file system in the consumer scenario includes various distributed file systems implemented in the kernel state and application state. As long as it involves cross-device access to remote files, it is applicable.

3. The client local persistent cache optimization method based on file page swapping as claimed in claim 1, wherein The LRU linked list in the first part of the client local persistent cache optimization management process exists in the client memory and is used to manage the persistent cache file pages in the external storage swap partition; each entry in the LRU linked list corresponds to a remote file, and the entry content includes file identification, file version information, the number of file pages that have been persistently cached, and a bitmap for recording the file pages that have been persistently cached.

4. The client - side local persistent cache optimization method based on file page swapping according to claim 1, wherein The persistent caching of the file data uses the file page as the smallest granularity. When recording the page persistent cache, it includes using a bitmap, a linked list, or a hash table.

5. The client - side local persistent cache optimization method based on file page swapping according to claim 3, wherein, Each entry of the LRU linked list described above can correspond to a single file page, a single file, or a series of associated file sets; the LRU linked list described above can be replaced by a FIFO linked list or an LFU linked list.

Images