A cache management method, device and related equipment
By associating the identity of the target file with the cache address, the file identifiers in the adjacent cache address are adjacent, so that the file data in the adjacent cache address is merged and written to the storage device during writing, the problem of multiple disk IO requests in the prior art is solved, more efficient storage operations are achieved, and system performance is improved.
Patent Information
- Application Number
- CN202110252260.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-08
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2041-03-08
AI Technical Summary
When the prior art writes data in the page cache to disk, the file data of adjacent pages cannot be merged and written, resulting in multiple disk IO requests, wasting bandwidth and affecting system performance.
By associating the identity of the target file with the cache address, the file identifiers in the adjacent cache address are adjacent, so that the file data in the adjacent cache address is merged and written to the storage device during writing, reducing the number of IO requests and improving bandwidth utilization.
Reduces the number of IO times of the storage device, improves the write speed, reduces the cache memory overhead, and improves system performance.
Smart Images

Figure CN115048035B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a cache management method, apparatus, and related equipment. Background Art
[0002] With the development of Internet services, electronic devices such as mobile phones, digital cameras, and car recorders are becoming more and more popular. The storage of massive data such as parallel downloading of applications / videos and multi-channel recordings has brought great challenges to the storage of operating systems. At present, most storage is to write data to the page cache first, and then write the data to the disk for permanent storage. In the pagecache scheme, user file data is copied and stored with pages as the basic memory unit. Since the page address is randomly assigned by the system, the physical addresses of the files stored in the adjacent page addresses on the disk are not continuous. Therefore, when the operating system writes the file data in the cache to the disk, the file data in several adjacent pages cannot be merged and written to the disk together. When writing multiple pages in the cache to the disk, multiple disk input / output (IO) requests are required, and the number of disk accesses is large. At the same time, a single disk IO request to write data to the disk is limited by the size of the page, and the optimal bandwidth of the disk is generally much larger than the data size of a single page, so the disk bandwidth is wasted, and the average write speed is slow, which is prone to disk IO jitter and data loss problems, affecting system performance. Summary of the invention
[0003] The present application provides a cache management method, apparatus and related equipment, which can reduce the number of storage device IO times, increase write speed, reduce cache memory overhead, and improve system performance.
[0004] In a first aspect, an embodiment of the present application provides a cache management method, the method comprising:
[0005] The electronic device receives a read request for a target file, and in response to the read request for the target file, allocates a first cache area for the target file, where the first cache area is a cache area corresponding to the read target file.
[0006] The electronic device receives a write request for a target file, and in response to the write request for the target file, allocates a second cache area for the target file and obtains an identifier of the target file; the electronic device caches the target file to a target cache address based on the target file identifier, wherein the target cache address is an address in the second cache area.
[0007] By caching the target files that need to be operated by the read request and the write request, and associating the identifier of the target file with the cache address, the file identifiers in the adjacent cache addresses in the cache are adjacent, that is, the physical addresses of the files cached in the adjacent cache addresses on the storage device are continuous. Therefore, when writing the data in the cache to the storage device, the file data in the adjacent cache addresses can be merged and written to the storage device together, reducing the number of storage device IO requests, and increasing the bandwidth of a single storage device IO request, without wasting the bandwidth of the storage device, increasing the storage device write speed, and improving the performance of the system.
[0008] The target file identifier is an identification number of the target file, and the target file identifier is used to indicate a storage address of the target file in the storage device.
[0009] In a possible implementation, within a preset time period, when the number of read requests obtained is less than the number of write requests, the allocated first cache area capacity is less than the second cache area capacity. Alternatively, within a preset time period, when the number of read requests obtained is greater than the number of write requests, the allocated first cache area capacity is greater than the second cache area capacity.
[0010] In a possible implementation, the first cache area and the second cache area both include one or more cache blocks, and the number and size of the cache blocks can be preset.
[0011] By reasonably allocating the capacity of the second cache area and the first cache area according to the number of read requests and the number of write requests, when the number of read requests is less than the number of write requests, the allocated capacity of the first cache area is smaller than the capacity of the second cache area. Therefore, the problem of allocating a large amount of cache capacity due to a small number of reads can be avoided, thereby avoiding wasting cache memory.
[0012] In a specific implementation, the electronic device caches the target file to the target cache address according to the target file identifier, specifically including: the electronic device detects the existence of the target file identifier in the second cache area, and the electronic device caches the target file to the first cache address, where the first cache address is the address corresponding to the target file identifier in the second cache area.
[0013] In a specific implementation, the electronic device caches the target file to the target cache address based on the target file identifier, specifically including: the electronic device detects that the identifier of the target file does not exist in the second cache area, and detects that the first identifier exists in the second cache area, and the first identifier is the adjacent previous identifier of the target file identifier; the electronic device caches the target file to the second cache address, and the second cache address is the adjacent next block address of the third cache address, wherein the file corresponding to the first identifier is cached in the third cache address.
[0014] In a possible implementation, before caching the target file to the second cache address, the method further includes: when the electronic device detects that the second cache address is occupied by a process, writing the file cached in the second cache address to the storage device.
[0015] In a specific implementation, the method further includes: the electronic device detects that the second cache address is not fully written by the process, and writes the file cached in the second cache address to the storage device.
[0016] In a specific implementation, the method also includes: the electronic device detects that the second cache address is filled by the process and the fourth cache address is a free address, the fourth cache address is the adjacent next block address of the second cache address, and writes the file cached in the second cache address to the storage device; the electronic device detects that the second cache address is filled by the process, and the second cache address to the fifth cache address are not free addresses, the fourth cache address is the adjacent next block address of the second cache address, the adjacent next block address of the fifth cache address is a free address, merges the files in the second cache address to the fifth cache address, and writes them to the storage device.
[0017] In a specific implementation, the files in the second cache address to the fifth cache address are merged and written into a storage device, specifically including: the second cache address to the fifth cache address include N cache blocks, and the following operations are performed on the first cache block to the Nth cache block, wherein the first cache block is cached at the second cache address, and the i-th cache block is any cache block from the first cache block to the Nth cache block; detecting whether the file identifiers in the i-th cache block and the i+1th cache block are adjacent; when it is detected that the file identifiers in the i-th cache block and the i+1th cache block are not adjacent, merging the first cache block to the i-th cache block The method comprises the following steps: first, merging the files in the first cache block to the files in the first cache block, and writing the merged file to the storage device; when it is detected that the file identifiers in the i-th cache block and the i+1-th cache block are adjacent, detecting whether the i+1-th cache block is full; when it is detected that the i+1-th cache block is not full, merging the files in the first cache block to the i+1-th cache block, and writing the merged file to the storage device; when it is detected that the i+1-th cache block is full, using the i+1-th cache block as the i-th cache block, using the i+2-th cache block as the i+1-th cache block, and detecting whether the file identifiers in the i-th cache block and the i+1-th cache block are written adjacently.
[0018] In a possible implementation, the method also includes: the electronic device detects that the identifier of the target file is smaller than the second identifier, and caches the target file to the first cache area, the second identifier being the smallest identifier among all file identifiers in the second cache area; or, the electronic device detects that the identifier of the target file is greater than the second identifier, and caches the target file to a sixth cache address, the sixth cache address being an idle address in the second cache area, the second identifier being the smallest identifier among all file identifiers in the second cache area.
[0019] In one possible implementation, the electronic device detects that the target file identifier is smaller than the second identifier, identifies the process where the target file is located as a slow write process, and the second identifier is the smallest identifier among all file identifiers in the second cache area; or, the electronic device detects that the target file identifier is larger than the second identifier, identifies the process where the target file is located as a fast write process, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
[0020] In one possible implementation, when the electronic device identifies the process where the target file is located as a slow writing process, it allocates a first cache area for the target file; or, when the electronic device identifies the process where the target file is located as a fast writing process, it allocates a second cache area for the target file.
[0021] By partitioning and caching the target file corresponding to the slow writing process and the target file corresponding to the fast writing process, it is prevented that the file corresponding to the slow writing process occupies the cache address of the file corresponding to the fast writing process, thereby improving the average writing speed of the storage device.
[0022] In a possible implementation, when the electronic device detects that the file with the target file identifier does not exist in the second cache area, the electronic device may also detect whether the second identifier is smaller than the target file identifier. When the target file identifier is detected to be smaller than the second identifier, the target file is cached in the first cache area; when the target file identifier is detected to be greater than the second identifier, the electronic device detects whether there is a file with the first identifier in the second cache area. If the file with the first identifier does not exist in the second cache area, the electronic device caches the target file to the sixth cache address; if the file with the first identifier is detected to exist in the second cache area, the electronic device writes the target file to the storage device, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
[0023] In a second aspect, an embodiment of the present application provides a cache management device, comprising various units for executing the cache management method in the first aspect or any possible implementation of the first aspect.
[0024] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory; the processor executes the code in the memory to execute the method provided in the first aspect and any possible implementation of the first aspect.
[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute a method provided in the first aspect or any possible implementation of the first aspect.
[0026] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.
[0028] Figure 1 This is an application scenario involved in the embodiments of the present application;
[0029] Figure 2 A user interface for displaying an application menu provided in an embodiment of the present application;
[0030] Figure 3 It is a recording interface provided by an embodiment of the present application;
[0031] Figure 4 It is a system architecture involved in the embodiments of the present application;
[0032] Figure 5 is a system framework diagram of a cache system provided in an embodiment of the present application;
[0033] Figure 6 This is a flow chart of a cache management method provided by an embodiment of the present application;
[0034] Figure 7 This is a schematic diagram of a cache area provided in an embodiment of the present application;
[0035] Figure 8 It is a specific flow chart provided in the embodiment of the present application;
[0036] Fig. 9 It is a schematic diagram of a cache address being occupied provided by an embodiment of the present application;
[0037] Fig.10 It is a schematic diagram of another cache address being occupied provided by an embodiment of the present application;
[0038] Fig.11 It is a schematic diagram of a merging algorithm provided in an embodiment of the present application;
[0039] Figure 12a-12e It is a schematic diagram of merging several files provided in an embodiment of the present application;
[0040] Fig.13 It is a schematic diagram of a cache address being occupied provided by an embodiment of the present application;
[0041] Fig.14 This is a flow chart of a fast and slow identification algorithm provided in an embodiment of the present application;
[0042] Fig.15 It is a schematic diagram of a cache device provided in an embodiment of the present application;
[0043] Fig.16 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present invention, but are not intended to limit the present invention.
[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0047] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0049] First, some concepts involved in the embodiments of the present application are introduced.
[0050] The virtual file system (VFS) is an interface that allows the use of different file system implementations with the operating system. VFS provides a unified interface for file and file system operations for user programs, shielding the differences and operation details of different file systems.
[0051] The file system (FS) is the method and data structure used by the operating system for files on storage devices (commonly disks, but also solid-state drives based on NAND Flash) or partitions, that is, the method of organizing files on storage devices. The software organization responsible for managing and storing file information in the operating system is called the file management system, or file system for short. The file system consists of three parts: the file system interface, the software set for object manipulation and management, and the objects and attributes. From a system perspective, the file system is a system that organizes and allocates the space of file storage devices, is responsible for file storage, and protects and retrieves stored files. Specifically, the file system is responsible for creating files for users, storing, reading, modifying, and dumping files, controlling file access, and revoking files when users no longer use them. The file allocation table (FAT) is one of the common file systems. It is a table used to record file locations. The disk space is divided into units of a certain number of sectors. Such units are called clusters, and the number of sectors contained in a cluster must be an integer power of 2. The maximum value of a cluster is 64 sectors, or 32KB. All clusters are numbered starting from 2, and each cluster has its own address number. User files and directories are stored in clusters. FAT is divided into FAT12, FAT16, and FATASK32. The difference is that the number of binary bits used by the FAT table to record any cluster link is 12, 16, and 32 respectively. When the file system allocates disk addresses to store user data, it always stores them in ascending order. For each file read and write request, there is a cluster address corresponding to it in the disk space. When reading and writing files, the cluster address where the file data is stored on the disk can be obtained according to the cluster number recorded in the FAT area of the file.
[0052] The embodiment of the present application provides a cache management method, which associates the cluster number of the target file with the cache address so that the file cluster numbers in the adjacent cache addresses in the cache are continuous, that is, the physical addresses of the files cached in the adjacent cache addresses on the storage device are continuous. Therefore, when writing the files in the cache to the storage device, the file data in the adjacent cache addresses can be merged and written to the storage device together, which reduces the number of IO requests for the storage device, and increases the bandwidth of a single IO request, without wasting the bandwidth of the storage device. At the same time, the target files that need to be operated by the read request and the write request are partitioned and cached, and the target files corresponding to the fast process and the target files corresponding to the slow process are partitioned and cached, which improves the average write speed. The size of the two cache areas can be set by yourself, avoiding wasting cache memory and improving system performance.
[0053] The electronic device involved in the embodiments of the present application can be a driving recorder, a desktop computer, a game console, a television, a projector, a personal digital assistant (PDA), a portable computer, a web tablet, a tablet computer, a wireless phone, a mobile phone, a smart phone, an e-book, a portable multimedia player (PMP), an MP3 player, a portable game console, a navigation device, a black box, a digital camera, a digital multimedia broadcasting (DMB) player, a three-dimensional (3D) television, a smart TV, a digital audio recorder, a digital audio player, a digital picture recorder, a digital picture player, a digital video recorder, and a digital video player. The present application does not make any limitation to this. The following is an example of the driving recorder to illustrate the embodiments of the present application.
[0054] The application scenarios involved in this application are described below.
[0055] Figure 1 This is an application scenario involved in the embodiments of this application. Figure 1 As shown, a user drives on a highway in a car equipped with a driving recorder 100. The driving recorder 100 mainly records the surrounding conditions during driving in real time through one or more built-in cameras, and stores one or more recorded video data in a memory card.
[0056] Figure 2 The user interface 110 for displaying the application menu on the driving recording device 100 is exemplarily shown. The user interface 110 includes: a gallery icon 111, a setting icon 112, a video icon 113, a camera icon 114, a main interface icon 115, and a time indicator 116. Among them:
[0057] The gallery icon 111 can be used to open a gallery application. For example, in response to a user operation, such as a touch operation, on the gallery icon 111, the dash cam device 100 can open the gallery application to display pictures and videos stored in the dash cam device 100. The pictures and videos stored in the dash cam device 100 include pictures and videos taken by the dash cam device 100 through a camera application.
[0058] The setting icon 112 is used to display the setting interface of the driving recorder 100;
[0059] The video icon 113 can be used for real-time recording. When a touch operation on the video icon 113 is detected, the driving recording device 100 can call the camera to record the surrounding situation in real time;
[0060] The camera icon 114 can be used to take photos. When a touch operation on the camera icon 114 is detected, the driving recording device 100 can call the camera to take photos of the surrounding situation;
[0061] A main interface icon 115 , when a touch operation acting on the main interface icon 115 is detected, the driving recording device 100 may display the main interface;
[0062] The information indication icon 116 is used to indicate at least one of time information, date information, weather information, or a text message or instant message received by a mobile phone or wearable device of a user that maintains a wireless connection with the driving recorder.
[0063] In some embodiments, Figure 2 The exemplary illustrated user interface 110 may be a main interface.
[0064] Understandably, Figure 2 The user interface on the driving recording device 100 is merely shown as an example and should not constitute a limitation on the embodiments of the present application.
[0065] like Figure 2 As shown, the user can click the video icon 113 on the interface 110 of the driving recording device 100, and the driving recording device 100 detects the touch operation on the video icon 113. In response to the operation, the driving recording device 100 can call the camera to record the surrounding situation in real time, and the driving recording device 100 can display the following Figure 3 The recording interface 210 is shown. Figure 3 An exemplary recording interface 210 on the driving recording device 100 is shown. Figure 3 As shown, the recording interface 210 includes: a video display area 211, a main interface icon 212, a return icon 213, and an information indicator 214. Among them:
[0066] The video display area 211 is used to display the picture currently captured by the camera;
[0067] The main interface icon 212 is consistent with the main interface icon 115 of the user interface 110, and will not be described in detail here;
[0068] The return icon 213 is used to return to the previous page. When a touch operation acting on the return icon 214 is detected, the driving recorder 100 can display the previous page of the current page;
[0069] The information indicator 214 is consistent with the information indicator 116 of the user interface 110 and will not be described again herein.
[0070] The user operations for starting the recording of the driving recorder 100 are not limited to those listed above. In a specific implementation, other user operations may be performed to start the recording of the driving recorder 100, and this application does not impose any limitation on this.
[0071] When the driving recorder 100 is turned on for recording, the driving recorder 100 sends the recorded video to the CPU for processing. The video data processed by the CPU is cached in the cache. When a preset condition is met, the file in the cache is written to the storage device.
[0072] In a possible implementation, the buffer in the above embodiment may be, but is not limited to, a volatile memory device such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The storage device may be, but is not limited to, a personal computer memory card international association (PCMCIA) card, a standard compact flash (CF) card, a smart media (SM) card, a memory stick, a multimedia card (MMC), RS-MMC and micro-MMC, a secure digital (SD) card, mini-SD, micro-SD and SDHC, and a universal flash storage device.
[0073] Figure 4 This is a system architecture involved in the embodiments of the present application, such as Figure 4As shown, the system includes: an application layer, a kernel layer, and a physical layer. When the driving recorder 100 is turned on, the camera 1 and / or the camera 2 in the driving recorder 100 writes the collected video or photo as a target file into the storage device. The application sends a write request for the target file to the kernel space layer. The virtual file system in the kernel space layer provides a unified file system interface. The file system detects whether the write request for the target file is a direct IO. If it is a direct IO, it is directly written to the storage device. If it is not a direct IO, the target file is cached through the cache system and then written to the storage device. Similarly, when the user uses a player to view the video or photo in the driving recorder 100, the application makes a read request for the target file. The application sends a read request for the target file to the kernel space layer. The file system detects whether the IO write request for the target file is a direct IO. If it is a direct IO, it is directly read from the storage device. If it is not a direct IO, the video or photo in the storage device is read into the cache as the target file.
[0074] The above-mentioned applications are not limited to cameras, but can also be other applications. The target files are not limited to videos or photos, but can also be other files. This application does not impose any restrictions on this.
[0075] Figure 5 The system framework diagram of the cache system is shown as an example. Figure 5 As shown, the cache system includes a read / write identification module 501, a cache management module 502, a fast / slow identification module 503, a cache merging module 504, and a cache 505. Among them:
[0076] The read / write identification module 501 can be used to identify the type of IO request for the target file, and the type of IO request includes a read request and a write request. For example, when the application performs a write operation on the target file, the write request for the target file can be identified, and when the application performs a read operation on the target file, the read request for the target file can be identified.
[0077] The cache management module 502 can be used to manage all operations in the cache system, such as read operations, write operations, allocation of cache areas, and detection of file identifiers. It is understandable that the cache management module 502 can manage the cache system in response to IO requests from the file system for target files. For example, the cache management module 502 can determine the cache address of the target file in the cache 505 in response to a write request from the file system for the target file.
[0078] The speed identification module 503 is used to identify whether the process where the target file is located is a slow writing process or a fast writing process.
[0079] The cache merging module 504 may be used to merge target file data according to instructions of the cache management module 502 .
[0080] The buffer 505 can be used to cache data of a read operation and / or a write operation. For example, when the IO request of the application to the target file is a read request, the buffer management module 502 first reads the target file from the storage device to the buffer 505, and then the buffer management module 502 reads the target file from the buffer 505. When the IO request to the target file is a write request, the buffer management module 502 first caches the target file to the buffer 505, and then writes the target file in the buffer 505 to the storage device.
[0081] Figure 6 A cache management method provided by an embodiment of the present application is shown, which is applied to Figure 5 The cache system shown in Figure 6 As shown, the cache management method includes:
[0082] S101: receiving an IO request for a target file, and determining a cache area for the target file according to the type of the IO request.
[0083] In a specific implementation, before receiving an IO request for a target file from an application, the cache management module 502 divides the cache 505 into a first cache area and a second cache area, wherein both the first cache area and the second cache area include one or more cache blocks, and the size and number of the cache blocks can be preset, for example, the size can be the FAT file system window size, such as 32K, and the number can be 3 or 5, which is not limited in this application. For example, Figure 7 As shown, the first cache area may include 7 cache blocks BLK 1 to BLK 7, and the second cache area may include 21 cache blocks BLK 1 to BLK 21. The size of the cache blocks in the first cache area may be consistent with the size of the cache blocks in the second cache area.
[0084] The cache management module 502 receives an IO request from an application to a target file, and determines a cache area for the target file according to the type of the IO request to the target file. The types of IO requests include read requests and write requests. Specifically, the read-write identification module 501 calls a system function to identify the type of IO request to the target file. If the IO request received to the target file is a read request, a first cache area is allocated to the target file. When executing the read request, the cache management module 502 reads the target file in the storage device into the first cache area. If the IO request received to the target file is a write request, a second cache area is allocated to the target file. When the system executes a write request, the cache management module 502 first caches the target file to the second cache area, and then writes the target file in the second cache area to the storage device. The system function can be the write and read functions that come with the file system, or other functions, and the embodiments of the present application do not impose any restrictions on this.
[0085] Optionally, within a preset time period, when the number of read requests obtained is less than the number of write requests, the allocated capacity of the first cache area is less than the capacity of the second cache area, that is, the number of cache blocks in the first cache area is less than the number of cache blocks in the second cache area. Alternatively, within a preset time period, when the number of read requests obtained is greater than the number of write requests, the allocated capacity of the first cache area is greater than the capacity of the second cache area.
[0086] The present application controls the size of the target file cache area required for the read request and the write request by partitioning the cache. When the application has relatively few read operations, a small amount of cache space can be allocated for the target file required for the read request to avoid wasting cache space. For example, in the driving recorder 100, generally speaking, the user mainly uses one or more built-in cameras to record the surrounding conditions during driving in real time, and stores one or more recorded video data in its storage card. It can be understood that in most cases, the video or photo is written as the target file in the driving recorder. In a few cases, the video or photo in the driving recorder will be checked. The number of write requests in the driving recorder is much greater than the number of read requests. Therefore, the first cache area can be allocated for the read request, and the second cache area can be allocated for the write request to avoid wasting cache space.
[0087] S102: When the received IO request for the target file is a write request, an identifier of the target file is obtained, and the target file is written into a target cache address according to the identifier of the target file.
[0088] Specifically, when the read-write identification module 501 identifies that the IO request of the application to the target file is a write request, the cache management module 502 obtains the identifier of the target file and records the identifier of the target file. Among them, the identifier of the target file is the identification number of the target file, and the identifier of the target file is used to indicate the storage location of the target file in the storage device. In the embodiment of the present application, the identifier of the target file can be a cluster number. It can be understood that after receiving the IO request of the application to the target file, the file system will assign an identifier, that is, a cluster number, to the target file. The cache management module 502 will record the cluster number of the file that needs to be operated by all write requests. After the cache management module 502 obtains the cluster number N of the target file, it detects the cluster numbers of all cache files in the second cache area, and checks whether there is a file with cluster number N in the second cache area. If there is a file with cluster number N, the target file is cached to the first cache address, wherein the first cache address is the cache address corresponding to the file with cluster number N in the second cache area; if there is no file with cluster number N in the second cache area, the cache management module 502 detects whether there is a file with cluster number N-1 in the second cache area.
[0089] When there is no file with cluster number N-1 in the second cache area, the cache management module 502 detects whether the cluster number N of the target file is less than the minimum cluster number in the second cache area, where cluster number N-1 is a first identifier, and the first identifier is the adjacent previous identifier of the target file identifier. When the cluster number N of the target file is less than the minimum cluster number in the second cache area, the cache management module 502 caches the target file to the first cache area; when the cluster number N of the target file is greater than the minimum cluster number in the second cache area, the cache management module 502 caches the target file to the sixth cache address and records the cluster number N of the target file in the second cache area, where the minimum cluster number is the second identifier, and the sixth cache address is a free cache address in the second cache area.
[0090] Optionally, the speed identification module 503 detects that the target file identifier is smaller than the second identifier and identifies the process where the target file is located as a slow writing process; or, the speed identification module 503 detects that the target file identifier is larger than the second identifier and identifies the process where the target file is located as a fast writing process.
[0091] Optionally, the speed identification module 503 sends the identification result of the process where the target file is located to the cache management module 502. When the cache management module 502 receives the identification result from the speed identification module 503 that the process where the target file is located is a slow writing process, the cache management module 502 allocates a first cache area for the target file; or, when the cache management module 502 receives the identification result from the speed identification module 503 that the process where the target file is located is a fast writing process, the cache management module 502 allocates a second cache area for the target file.
[0092] When there is a file with cluster number N-1 in the second cache area, the cache management module 502 searches for the cache address corresponding to the file with cluster number N-1, and detects whether the next cache address of the cache address corresponding to the file with cluster number N-1 is occupied. If the next cache address of the cache address corresponding to the file with cluster number N-1 is not occupied, the cache management module 502 caches the target file to the second cache address, and records the cluster number of the target file in the second cache area, wherein the second cache address is the next adjacent address of the third cache address, and the third cache address is the cache address of the file corresponding to the first identifier. When the next cache address of the cache address corresponding to the file with cluster number N-1 is occupied, that is, when the second cache address is occupied, the cache management module 502 writes the file in the second cache address to the storage device, releases the space of the second cache address, and the cache management module 502 caches the target file to the second cache address, and records the cluster number N of the target file in the second cache area.
[0093] By associating the cluster number of the target file with the cache address, the file cluster numbers in adjacent cache addresses in the cache are continuous, that is, the physical addresses of the files cached in adjacent cache addresses on the storage device are continuous. Therefore, when the system writes the data in the cache to the disk, the file data in the adjacent cache addresses can be merged and written to the storage device together, reducing the number of IO requests and increasing the bandwidth of a single IO request, without wasting the bandwidth of the storage device.
[0094] Next, we take the cluster number as the target file identifier as an example, combined with Figure 8 The specific process of determining the target cache address corresponding to the target file in the embodiment of the present application is introduced by way of example.
[0095] S201, detecting whether the IO request for the target file is a write request, if so, executing S202, if not, caching the target file to the first cache area.
[0096] Specifically, after the cache management module 502 receives the IO request of the application program for the target file, the read / write identification module 501 identifies whether the IO request for the target file is a write request. If the IO request for the target file is identified as a read request, the cache management module 502 allocates a first cache area for the target file. If the IO request for the target file is identified as a write request, S202 is executed.
[0097] S202: Allocate a second buffer area and obtain a cluster number N of the target file.
[0098] Specifically, when the read-write identification module 501 identifies that the IO request for the target file is a write request, the cache management module 502 allocates a second cache area for the target file, and the cache management module 502 obtains the cluster number N of the target file from the file system.
[0099] S203, detecting whether there is a file with cluster number N in the second cache area, if so, caching the target file to the first cache address, if not, executing S204.
[0100] Specifically, the cache management module 502 detects whether there is a file with cluster number N in the second cache area. If there is a file with cluster number N in the second cache area, the target file is cached to the first cache address, where the first cache address is the cache address corresponding to the cluster number N in the second cache area; if there is no file with cluster number N in the second cache area, the operation in S204 is executed.
[0101] S204, checking whether there is a file with cluster number N-1 in the second cache area, if so, cache the target file to the second cache address, if not, execute S205.
[0102] Specifically, when the cache management module 502 detects that there is no file with cluster number N in the second cache area, the cache management module 502 detects whether there is a file with cluster number N-1 in the second cache area. If it is detected that there is no file with cluster number N-1 in the second cache area, S205 is executed; if it is detected that there is a file with cluster number N-1 in the second cache area, the target file is cached to the second cache address, wherein the second cache address is the next adjacent address of the cache address corresponding to the file with cluster number N-1.
[0103] Specifically, before caching the target file to the second cache address, the cache management module 502 needs to detect whether the second cache address is occupied. When the second cache address is occupied, the cache management module 502 detects whether the second cache address is full. If it is detected that the second cache address is not full, the cache management module 502 writes the cached file in the second cache address to the storage device, releases the cache space of the second cache address, and then writes the target file to the second cache address. For example, Fig. 9 As shown, the cache address corresponding to cluster number N-1 is BLK n-1, and the next cache address of the cache address corresponding to cluster number N-1, that is, the second cache address is BLK n. It can be seen that BLK n is not full, and the cache management module 502 can write the file in BLK n to the storage device, release the space of BLK n, and then write the target file into BLK n.
[0104] If it is detected that the second cache address is full, the cache management module 502 detects whether the next address block of the second cache address is a free address. If it is detected that the next address block of the second cache address is a free address, the cache management module 502 writes the file cached in the second cache address to the storage device; if it is detected that the next address block of the second cache address is not a free address, the fifth cache address is searched, and the cache management module 502 sends the second cache address and the fifth cache address to the cache merging module 503. After receiving the second cache address and the fifth cache address sent by the cache management module 502, the cache merging module 504 merges the files cached from the second cache address to the fifth cache address to obtain a merged file, and writes this merged file to the storage device, wherein the second cache address to the fifth cache address are not free addresses, and the next cache address of the fifth cache address is a free address. For example, if Fig.10 As shown, when the second cache address BLK n is filled by the process, and the second cache address BLK n to the cache address BLK n+2 are filled by the process task3, and the next cache address (BLK n+3) of the cache address BLK n+2 is a free address, the cache address BLK n+2 is the fifth cache address.
[0105] Optionally, the second cache address to the fifth cache address include N cache blocks, wherein the first cache block is cached at the second cache address, and the i-th cache block is any cache block from the first cache block to the N-th cache block. Fig.11 As shown, the above process of merging the files cached in the second cache address to the fifth cache address may be the following steps:
[0106] S2041: Detect whether the file cluster numbers in the i-th cache block and the i+1-th cache block are adjacent, if not, execute S2042; if yes, execute S2043.
[0107] S2042: Merge the files in the first cache block and the i-th cache block, and write the merged files to the storage device.
[0108] S2043: Check whether the i+1th cache block is full, if not, execute S2045; if yes, execute S2044.
[0109] S2044: Detect whether the i+1th cache block is the Nth cache block, if not, execute S2045; if yes, execute S2046.
[0110] S2045: Merge the files in the first cache block and the (i+1)th cache block, and write the merged files to the storage device.
[0111] S2046: Use the i+1th cache block as the i-th cache block, and use the i+2th cache block as the i+1th cache block.
[0112] Specifically, when it is detected that the i+1th cache block is not fully written and it is detected that the i+1th cache block is not the Nth cache block, the i+1th cache block is updated to the ith cache block, the i+2th cache block is updated to the i+1th cache block, and the above steps S2041 to S2046 are repeated.
[0113] By merging files corresponding to adjacent cluster numbers in adjacent cache addresses and writing the merged files to the storage device, the number of storage device IOs is reduced, the bandwidth of a single storage device IO is increased, and the bandwidth of the storage device is not wasted.
[0114] For example, Figure 12a-12e As shown, it is detected that files are written to the second cache address BLK n to the cache address BLK n+2. At this time, the cache address BLK n+2 is the fifth cache address. The first cache block is cached in the second cache address, and the Nth cache block is cached in the fifth cache address. N is three, and the files in the second cache address BLK n to the cache address BLK n+2 are merged. It is detected whether the file cluster numbers in the first cache block and the second cache block are adjacent. If the file cluster numbers in the first cache block and the second cache block are not adjacent, such as Fig.12a As shown, the file in the first cache block is written into the storage device to release the space of the first cache block; if the file cluster numbers in the first cache block and the second cache block are adjacent, then check whether the second cache block is full. If it is detected that the second cache block is not full, Figure 12b As shown, merge the files in the first cache block to the second cache block, write the merged files to the storage device, and release the space from the first cache block to the second cache block; if it is detected that the second cache block is full, then detect whether the cluster numbers of the files in the second cache block and the third cache block are adjacent. If it is detected that the cluster numbers of the files in the second cache block and the third cache block are not adjacent, as shown in FIG. Fig.12c As shown, the files in the first cache block and the second cache block are merged, and the merged files are written to the storage device, and the space from the first cache block to the second cache block is released; the cluster numbers of the files in the second cache block and the third cache block are detected to be adjacent, and then the third cache block is detected to be full. If the third cache block is not full, Fig.12d As shown, merge the files in the first cache block to the third cache block, write the merged file to the storage device, and release the space from the first cache block to the third cache block; if it is detected that the third cache block is full, such as Fig.12e As shown, the files in the first cache block and the third cache block are merged, the merged files are written to the storage device, and the space from the first cache block to the third cache block is released.
[0115] S205, detecting whether the cluster number N is smaller than the minimum cluster number in the second cache area, if not, caching the target file to the sixth cache address; if yes, caching the target file to the first cache area.
[0116] Specifically, when it is detected that there is no file with cluster number N-1 in the second cache area, the cache management module 502 detects whether the cluster number N is less than the minimum cluster number in the second cache area. At this time, the fast and slow identification module 503 will obtain the target file cluster number N, detect whether the process where the target file is located is a slow process, and send the detection result to the cache management module 502. If the cluster number N is less than the minimum cluster number in the second cache area, the speed identification module 503 identifies the process where the target file is located as a slow writing process, and sends the identification result to the cache management module. After receiving the identification result of the slow process sent by the speed identification module 503, the cache management module 502 allocates the first cache area to the target file and caches the target file in the first cache area; if the cluster number N is greater than the minimum cluster number in the second cache area, the speed identification module 503 identifies the process where the target file is located as a fast writing process, and sends the identification result to the cache management module 502. After receiving the identification result of the fast process sent by the speed identification module 503, the cache management module 502 allocates the second cache area to the target file, caches the target file to the sixth cache address, and records the cluster number of the target file in the second cache area. Among them, the sixth cache address is a free cache address in the second cache area. If the cluster number N is less than the minimum cluster number in the second cache area, the cache management module 502 caches the target file in the first cache area. It is worth noting that, since the file system always allocates cluster addresses of the storage device to store file data in ascending order, when the cluster number N is smaller than the lowest cluster number in the second cache area, this situation will occur when the cache address corresponding to the target file is occupied by other processes. For example, Fig.13 A schematic diagram showing a cache address being occupied provided by an embodiment of the present application is shown. Fig.13As shown, assuming that there are 5 cache blocks BLK 1 to BLK 5 in the second cache area, in the first round of caching, the processes where the 5 file write requests with cluster numbers 1 to 5 cached in the second cache area are located are task1 to task5, among which the process task3 where the file write request with cluster number 3 is located is dormant due to system scheduling reasons, and the file corresponding to process task3 is only partially written in the cache block BLK 3, and some file data is not written. When other cache blocks are full of data, or when the free space of the cache blocks in the second cache area is less than the preset value, the cache management module 502 writes the file data in the second cache area to the storage device, or writes the file data of part of the cache blocks in the second cache area to the storage device, and releases all or part of the space in the second cache area for the caching of subsequent files. After all or part of the space in the second cache area is released, the second round of caching begins. When the second cache area starts to cache the file with cluster number 6, the cluster number recorded in the second cache area begins to be updated. When the file with cluster number 9 starts to be cached, the cluster number recorded in the second cache area is updated to 6 to 9. At this time, the cache block BLK 3 corresponding to the process task3 in the first round of caching is occupied by the process task8. If the process task3 is awakened at this time during the first round of caching, the process task3 will continue to write the file data with cluster number 3 that has not been written in the first round of caching. Then the cache management module 502 will detect whether there is a file with cluster number 3 in the second cache area. At this time, since the cluster number in the second cache area has been updated to 6 to 9, there is no file with cluster number 3 in the second cache area, and there is no file with cluster number 2 in the second cache area. Cluster number 3 is less than the minimum cluster number 6 in the second cache area. Therefore, the fast and slow identification module identifies the process task3 where the file with cluster number is located as a slow process, and caches the file with cluster number 3 to the first cache area to avoid affecting the writing speed of other processes, thereby improving the average writing speed and improving the system performance.
[0117] Fig.14 The flowchart of the speed recognition algorithm in the speed recognition module 503 is shown as an example. Fig.14 As shown, including:
[0118] S2051: Get the target file cluster number N.
[0119] S2052: Check whether the cluster number N is smaller than the minimum cluster number in the second cache area. If not, execute S2053; if so, execute S2054.
[0120] S2053: Identify that the process where the target file is located is a quick write process.
[0121] S2054: Identify that the process where the target file resides is a slow writing process.
[0122] Optionally, when the cache management module 502 detects that there is no file with cluster number N in the second cache area, the cache management module 502 can also execute S205, that is, detect whether the cluster number N is less than the minimum cluster number in the second cache area. When the cluster number N is less than the minimum cluster number in the second cache area, the cache management module 502 caches the target file to the first cache area; when the cluster number N is greater than the minimum cluster number in the second cache area, S204 is executed, that is, detect whether there is a file with cluster number N-1 in the second cache area. If it is detected that there is no file with cluster number N-1 in the second cache area, the cache management module 502 caches the target file to the sixth cache address and records the cluster number of the target file in the second cache area. If it is detected that there is a file with cluster number N-1 in the second cache area, the cache address of the file with cluster number N-1 is searched and the target file is cached to the second cache address.
[0123] The present application associates the cluster number of the target file with the cache address, so that the file cluster numbers in the adjacent cache addresses in the cache are continuous, that is, the physical addresses of the files cached in the adjacent cache addresses on the storage device are continuous. Therefore, when the system writes the data in the cache to the storage device, the file data in the adjacent cache addresses can be merged and written to the storage device together, which reduces the number of storage device IO requests, and increases the bandwidth of a single storage device IO request, without wasting the bandwidth of the storage device. At the same time, the target files that need to be operated by the read request and the write request are partitioned and cached, and the target files corresponding to the fast write process and the target files corresponding to the slow write process are partitioned and cached, which improves the average write speed. The size of the two cache areas can be set by yourself, avoiding wasting cache memory and improving system performance.
[0124] It should be noted that, for the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.
[0125] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of the present invention. Secondly, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.
[0126] Combined with the above Figures 1 to 14 The cache management method provided by the embodiment of the present application is described in detail. Figure 15 to Figure 16 , describing a cache device and an electronic device provided according to an embodiment of the present application.
[0127] Fig.151 is a schematic diagram of a cache device provided in an embodiment of the present application. The cache device 1500 includes a communication unit 1501 and a processing unit 1502, wherein:
[0128] The communication unit 1501 is used to receive a read request for a target file and receive a write request for a target file.
[0129] Processing unit 1502 is used to allocate a first cache area for the target file in response to a read request for the target file, where the first cache area is a cache area corresponding to the read target file; and allocate a second cache area for the target file in response to a write request for the target file.
[0130] The communication unit 1501 is also used to obtain the identifier of the target file.
[0131] The processing unit 1502 is further configured to write the target file into a target cache address according to the target file identifier, wherein the target cache address is an address in the second cache area.
[0132] The target file identifier is an identification number of the target file, and the target file identifier is used to indicate a storage address of the target file in the storage device.
[0133] In a possible implementation, the processing unit 1502 is further configured to allocate a first cache area capacity smaller than a second cache area capacity when the number of read requests obtained is less than the number of write requests within a preset time period. Alternatively, the processing unit 1502 is further configured to allocate a first cache area capacity larger than a second cache area capacity when the number of read requests obtained is greater than the number of write requests within a preset time period.
[0134] In a specific implementation, the processing unit 1502 is specifically used to: detect that there is an identifier of the target file in the second cache area, cache the target file to a first cache address, and the first cache address is an address corresponding to the identifier of the target file in the second cache area.
[0135] In a specific implementation, the processing unit 1502 is specifically used to detect that there is no identifier of the target file in the second cache area, and detect that there is a first identifier in the second cache area, where the first identifier is the adjacent previous identifier of the target file identifier; cache the target file to the second cache address, where the second cache address is the adjacent next block address of the third cache address, wherein the file corresponding to the first identifier is cached in the third cache address.
[0136] In a possible implementation, the processing unit 1502 is specifically configured to, before caching the target file to the second cache address, write the file cached in the second cache address to the storage device when it is detected that the second cache address is occupied by a process.
[0137] In a specific implementation, the processing unit 1502 is further used to detect that the second cache address is not fully written by the process, and write the files cached in the second cache address to the storage device; detect that the second cache address is fully written by the process and the fourth cache address is a free address, the fourth cache address is the adjacent next block address of the second cache address, and write the files cached in the second cache address to the storage device; detect that the second cache address is fully written by the process and the second cache address to the fifth cache address are not free addresses, the fourth cache address is the adjacent next block address of the second cache address, the adjacent next block address of the fifth cache address is the free address fourth cache address, merge the files from the second cache address to the fifth cache address, and write them to the storage device.
[0138] In a specific implementation, the processing unit 1502 is specifically used to include N cache blocks in the second cache address to the fifth cache address, and perform the following operations on the first cache block to the Nth cache block, wherein the first cache block is cached at the second cache address, and the i-th cache block is any cache block from the first cache block to the N-th cache block; detect whether the file identifiers in the i-th cache block and the i+1-th cache block are adjacent; when it is detected that the file identifiers in the i-th cache block and the i+1-th cache block are not adjacent, merge the files in the first cache block to the i-th cache block, and The merged file is written into the storage device; when it is detected that the file identifiers in the i-th cache block and the i+1-th cache block are adjacent, whether the i+1-th cache block is full is detected; when it is detected that the i+1-th cache block is not full, the files in the first cache block and the i+1-th cache block are merged, and the merged file is written into the storage device; when it is detected that the i+1-th cache block is full, the i+1-th cache block is used as the i-th cache block, the i+2-th cache block is used as the i+1-th cache block, and whether the file identifiers in the i-th cache block and the i+1-th cache block are written adjacently is detected.
[0139] In a specific implementation, the processing unit 1502 is specifically used to detect that the identifier of the target file is smaller than the second identifier, cache the target file to the first cache area, and the second identifier is the smallest identifier among all file identifiers in the second cache area; or, detect that the identifier of the target file is larger than the second identifier, cache the target file to a sixth cache address, the sixth cache address is a free address in the second cache area, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
[0140] In one possible implementation, the processing unit 1502 is also used to detect that the target file identifier is smaller than the second identifier, and identify the process where the target file is located as a slow write process; or, the processing unit 1502 is also used to detect that the target file identifier is larger than the second identifier, and identify the process where the target file is located as a fast write process, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
[0141] In a possible implementation, the processing unit 1502 is specifically used to allocate a first cache area for the target file when the process where the target file is located is a slow writing process; or allocate a second cache area for the target file when the process where the target file is located is a fast writing process.
[0142] It should be understood that the cache device 1500 of the embodiment of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figures 1 to 14 When the cache management method shown in , the cache device 1500 and its various modules can also be software modules.
[0143] Fig.16 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, wherein the electronic device 1600 includes: a processor 1610, a communication interface 1620, and a memory 1630, wherein the processor 1610, the communication interface 1620, and the memory 1630 are interconnected via a bus 1640, wherein the processor 1610 is used to execute instructions stored in the memory 1630. The memory 1630 stores program codes, and the processor 1610 can call the program codes stored in the memory 1630 to perform the following operations:
[0144] A read request for a target file is received, and in response to the read request for the target file, a first cache area is allocated to the target file, where the first cache area is a cache area corresponding to the read target file. A write request for a target file is received, and in response to the write request for the target file, a second cache area is allocated to the target file, and an identifier of the target file is obtained; and according to the identifier of the target file, the target file is written to a target cache address, where the target cache address is an address in the second cache area.
[0145] In the embodiment of the present application, the processor 1610 can have a variety of specific implementation forms. For example, the processor 1610 can be a combination of any one or more of the processors such as CPU, GPU, TPU or NPU. The processor 1610 can also be a single-core processor or a multi-core processor. The processor 1610 can be a combination of CPU (GPU, TPU or NPU) and hardware chips. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD complex programmable logical device (CPLD), field programmable gate array (FPGA), generic array logic (GAL) or any combination thereof. The processor 1610 can also be implemented using a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP).
[0146] The communication interface 1620 may be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface may be an Ethernet interface, a controller area network (CAN) interface, or a local interconnect network (LIN) interface, and the wireless interface may be a cellular network interface or a wireless local area network interface.
[0147] The memory 1630 may be a non-volatile memory, such as a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The memory 1630 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache.
[0148] The memory 1630 may also be used to store instructions and data, so that the processor 1610 calls the instructions stored in the memory 1630 to implement the operations performed by the processing unit 1502. In addition, the electronic device 1600 may include a Fig.15 Show more or fewer components, or have different component configurations.
[0149] The bus 1640 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0150] Optionally, the electronic device 1600 may further include an input / output interface 1650 , to which an input / output device is connected, for receiving input information and outputting operation results.
[0151] It should be understood that the electronic device 1600 of the embodiment of the present application may correspond to the cache device 1500 in the above embodiment, which will not be described in detail here.
[0152] An embodiment of the present application also provides a non-volatile computer storage medium, in which instructions are stored. When the computer storage medium is executed on a processor, the method steps in the above method embodiment can be implemented. The specific implementation of the processor of the computer storage medium in executing the above method steps can refer to the specific operations of the above method embodiment, which will not be repeated here.
[0153] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0154] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium, and the semiconductor medium can be a solid-state hard disk.
[0155] The above is only a specific implementation of the present application. Those skilled in the art may conceive of changes or substitutions based on the specific implementation provided by the present application, which should all be included in the protection scope of the present application.
Claims
1. A cache management method, characterized in that: The method comprises: receiving a read request for a target file; In response to the read request, allocating a first cache area for the target file, the first cache area being a cache area corresponding to reading the target file; Receiving a write request to the target file; In response to the write request, allocating a second buffer area for the target file and obtaining an identifier of the target file; caching the target file to a target cache address according to the identifier of the target file, wherein the target cache address is an address in the second cache area; The step of caching the target file to a target cache address according to the identifier of the target file includes: When it is detected that the identifier of the target file does not exist in the second cache area and when it is detected that the first identifier exists in the second cache area, the target file is cached to a second cache address, the first identifier is an adjacent previous identifier of the identifier of the target file, the file corresponding to the first identifier is cached in a third cache address, and the second cache address is an adjacent next block address of the third cache address.
2. The method according to claim 1, characterized in that The identifier of the target file is an identification number of the target file, and the identification number of the target file is used to indicate a storage address of the target file in a storage device.
3. The method according to claim 1, characterized in that When the number of read requests obtained within a preset time period is less than the number of write requests, the capacity of the first cache area is less than the capacity of the second cache area.
4. The method according to any one of claims 1 to 3, characterized in that The step of caching the target file to a target cache address according to the identifier of the target file further includes: It is detected that the identifier of the target file exists in the second cache area, and the target file is cached to a first cache address, where the first cache address is an address corresponding to the identifier of the target file in the second cache area.
5. The method according to claim 1, characterized in that Before caching the target file to the second cache address, the method further includes: When it is detected that the second cache address is occupied by a process, the file cached in the second cache address is written into a storage device.
6. The method according to claim 5, characterized in that The method further comprises: It is detected that the second cache address is not fully written by the process, and the file cached in the second cache address is written to the storage device; it is detected that the second cache address is fully written by the process and the fourth cache address is a free address, and the fourth cache address is the next adjacent address of the second cache address, and the file cached in the second cache address is written to the storage device; It is detected that the second cache address is filled by the process, and none of the second cache address to the fifth cache address is a free address, the fourth cache address is the adjacent next block address of the second cache address, and the adjacent next block address of the fifth cache address is a free address, and the files from the second cache address to the fifth cache address are merged and written to the storage device.
7. The method according to claim 6, characterized in that The step of merging the files from the second cache address to the fifth cache address and writing the files into the storage device comprises: The second cache address to the fifth cache address include N cache blocks, and the following operation is performed on the first cache block to the Nth cache block, wherein the first cache block is cached at the second cache address, and the i-th cache block is any cache block from the first cache block to the Nth cache block; Detecting whether the file identifiers in the i-th cache block and the i+1-th cache block are adjacent; When it is detected that the file identifiers in the i-th cache block and the i+1-th cache block are not adjacent, merging the first cache block with the file in the i-th cache block, and writing the merged file to the storage device; When it is detected that the file identifiers in the i-th cache block and the i+1-th cache block are adjacent, detecting whether the i+1-th cache block is full; When it is detected that the i+1th cache block is not fully written, merging the first cache block with the file in the i+1th cache block, and writing the merged file into the storage device; When it is detected that the i+1th cache block is full, the i+1th cache block is used as the i-th cache block, the i+2th cache block is used as the i+1th cache block, and it is detected whether the file identifiers in the i-th cache block and the i+1th cache block are adjacent.
8. The method according to claim 1, characterized in that The method further comprises: It is detected that the identifier of the target file is smaller than the second identifier, and the target file is cached in the first cache area, and the second identifier is the smallest identifier among all file identifiers in the second cache area; or, It is detected that the identifier of the target file is greater than the second identifier, and the target file is cached to a sixth cache address, where the sixth cache address is a free address in the second cache area, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: When it is detected that the identifier of the target file is smaller than the second identifier, the process where the target file is located is identified as a slow-writing process, and the second identifier is the smallest identifier among all file identifiers in the second cache area; or, when it is detected that the identifier of the target file is larger than the second identifier, the process where the target file is located is identified as a fast-writing process, and the second identifier is the smallest identifier among all file identifiers in the second cache area.
10. The method according to claim 9, characterized in that The method comprises: When the process where the target file is located is the slow writing process, a first buffer area is allocated to the target file; or, when the process where the target file is located is the fast writing process, a second buffer area is allocated to the target file.
11. An electronic device, characterized in that: include: A processor and a memory, wherein the processor executes the code in the memory to perform the method according to any one of claims 1 to 10.
12. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Method and device for memory system management
CN107506314A
Systems and methods for a file-level cache
US20120210066A1