Cache control method and system based on file data volume, terminal and medium

Abstract

The invention discloses a cache control method and system based on file data volume, a terminal and a medium, the method comprises the following steps: determining the file data volume of written files, classifying the files based on the file data volume to obtain a first type of files and a second type of files, the file data volume of the first type of files being smaller than the file data volume of the second type of files; starting a first data caching program corresponding to the first type of files, and caching the first type of files to a local storage device based on the first data caching program; target storage equipment corresponding to local storage equipment in the distributed file system is obtained, the first class of files are synchronized to the target storage equipment, and both the local storage equipment and the target storage equipment interact with the Ceph cluster. According to the method and the device, the files with small file data volume can be firstly cached to the local storage device and then run to the corresponding target storage device, so that the data reading and writing efficiency can be improved, and the problem of cache single-point fault is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed file cache technology, and in particular to a cache control method, system, terminal and medium based on file data volume. Background Art

[0002] When processing file uploads, the client usually passes the original data information of the file to the OSD (storage component) in the Ceph cluster through the network, rather than directly to the MDS (MetadataSeve, a component used to store file system metadata) of other clients. For distributed file systems, when a user copies a file from a platform such as the Windows operating system, the file is stored directly on the local hard disk. Then, the client of the distributed file system needs to send the data of the entire file to the OSD in the Ceph cluster through the network, and the OSD is responsible for storing the data to each node of the Ceph cluster (that is, each client in the Ceph cluster). Therefore, this process reduces the efficiency of file transfer, especially when transferring some files with relatively small data volumes, this process greatly wastes the time of file transfer.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a cache control method, system, terminal and medium based on file data volume are provided, aiming to solve the problem in the prior art that when transmitting some files with relatively small data volume, a lot of file transmission time is wasted.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a cache control method based on file data volume, wherein the method comprises: Determine the file data volume of the plurality of files written, and classify the plurality of files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; Starting a first data cache program corresponding to the first type of files, and caching the first type of files to a local storage device based on the first data cache program; Obtain a target storage device corresponding to the local storage device in a distributed file storage system, and synchronize the first type of files to the target storage device, wherein both the local storage device and the target storage device interact with a Ceph cluster.

[0006] In one implementation, determining the file data volume of the plurality of files written includes: After a file write event is obtained, all files corresponding to the file write event are obtained, and file information of all files is determined; Based on the file information, the file data volume of each file is determined, wherein the file data volume reflects the amount of space occupied by the file on the storage device.

[0007] In one implementation, the classifying the plurality of files based on the file data volume to obtain first-category files and second-category files includes: Sort all files from small to large according to the file data volume to obtain a file sequence; Obtaining the median of the file data volume of all files, and determining the file corresponding to the median; Determine a classification node based on the file corresponding to the median; A plurality of files are classified based on the classification node to obtain the first category files and the second category files.

[0008] In one implementation, determining a classification node based on the file corresponding to the median includes: Obtain the first file data volume of the file corresponding to the median; Respectively obtain the second file data amounts of the files adjacent to the left and to the right of the file corresponding to the median in the file sequence; Calculating a difference between the first file data volume and the second file data volume; If there is a difference greater than the difference threshold, the file corresponding to the median is used as a classification node.

[0009] In one implementation, determining a classification node based on the file corresponding to the median includes: If the differences are all smaller than the difference threshold, determining the files adjacent to the left and the right of the file corresponding to the median in the file sequence; The classification node is re-determined based on the files adjacent to the left and the right of the file corresponding to the median.

[0010] In one implementation, obtaining the target storage device corresponding to the local storage device in the distributed file storage system includes: Obtaining a preset mapping file, wherein the mapping file reflects the corresponding relationship between a plurality of storage devices; The local storage device is matched with the mapping file to determine a target storage device corresponding to the local storage device.

[0011] In one implementation, the method further includes: Starting a second data cache program corresponding to the second type of files; Based on the second data caching program, the second type of files are saved in the Ceph cluster.

[0012] In a second aspect, an embodiment of the present invention further provides a cache control system based on file data volume, wherein the system is used to implement the steps of the cache control method based on file data volume in the above scheme, and the system includes: A file classification module, used to determine the file data volume of the plurality of files written, and classify the plurality of files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; A local cache module, used to start a first data cache program corresponding to the first type of files, and cache the first type of files to a local storage device based on the first data cache program; A synchronization cache module is used to obtain a target storage device corresponding to the local storage device in a distributed file storage system, and synchronize the first type of files to the target storage device, wherein both the local storage device and the target storage device interact with a Ceph cluster.

[0013] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and a cache control program based on file data volume stored in the memory and executable on the processor, and when the processor executes the cache control program based on file data volume, the steps of the cache control method based on file data volume of any one of the above-mentioned schemes are implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a cache control program based on file data volume is stored on the computer-readable storage medium, and when the cache control program based on file data volume is executed by a processor, the steps of the cache control method based on file data volume described in any one of the above-mentioned schemes are implemented.

[0015] Beneficial effects: Compared with the prior art, the present invention provides a cache control method based on file data volume. The present invention first determines the file data volume of several files written, and classifies several files based on the file data volume to obtain first-class files and second-class files, wherein the file data volume of the first-class files is less than the file data volume of the second-class files. Then, the first data cache program corresponding to the first-class files is started, and the first-class files are cached to the local storage device based on the first data cache program. Finally, the target storage device corresponding to the local storage device in the distributed file storage system is obtained, and the first-class files are synchronized to the target storage device, wherein the local storage device and the target storage device both interact with the Ceph cluster. The present invention can first cache files with small file data volume to the local storage device, and then synchronize them to the corresponding target storage device, thereby realizing the synchronization of the first-class files to different nodes in the Ceph cluster, which not only improves the efficiency of data reading and writing, but also solves the problem of single point failure of the cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention provides a flowchart of a preferred embodiment of a cache control method based on file data volume.

[0017] Figure 2 A schematic diagram of an application of a cache control method based on file data volume provided by an embodiment of the present invention.

[0018] Figure 3 A schematic diagram of the architecture of a cache control system based on file data volume provided by an embodiment of the present invention.

[0019] Figure 4 A functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations or steps, nor must they be executed in the order described. For example, some operations or steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0022] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms. It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish between identical or similar items with substantially identical functions and effects. For example, the first control information and the second control information are only used to distinguish different control information, and their order is not limited. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences. It should be further understood that the term “and / or” used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0023] In order to solve the problems of the prior art, this embodiment provides a cache control method based on file data volume. Based on the method of this embodiment, files with small file data volume can be first cached in a local storage device, and then synchronized to the corresponding target storage device, which not only improves the efficiency of data reading and writing, but also solves the problem of cache single point failure. In specific application, this embodiment first determines the file data volume of several files written, classifies several files based on the file data volume, and obtains first-class files and second-class files, wherein the file data volume of the first-class files is less than the file data volume of the second-class files. Then, the first data cache program corresponding to the first-class files is started, and the first-class files are cached to the local storage device based on the first data cache program. Finally, the target storage device corresponding to the local storage device in the distributed file storage system is obtained, and the first-class files are synchronized to the target storage device, wherein both the local storage device and the target storage device interact with the Ceph cluster.

[0024] The cache control method based on file data volume of this embodiment can be applied to a terminal, which can be a computer, a mobile phone, a smart TV and other intelligent terminal products. Figure 1 As shown in , the cache control method based on file data volume of this embodiment includes the following steps: Step S100: determine the file data volume of several files to be written, and classify the several files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is smaller than the file data volume of the second-category files.

[0025] In this embodiment, if Figure 2 As shown in , the cache control method based on file data volume of this embodiment can be applied to a distributed file system, which includes a Ceph cluster and several fuse clients interacting with the Ceph cluster. The fuse client is the terminal of this embodiment. When a file is written to the terminal, such as by Figure 2 When the NFS (Network File System) in writes files to the fuse client, the terminal will determine the file data volume of the several files written. The file data volume can be the file size, that is, the amount of space occupied by the file on the storage device. After determining the file data volume of the file, the several files can be classified by the file data volume to obtain the first category of files and the second category of files. In this embodiment, the file data volume of the first category of files is less than the file data volume of the second category of files. The purpose of classifying files in this embodiment is to facilitate cache control for smaller files in subsequent steps and improve cache efficiency.

[0026] Specifically, after obtaining a file write event, the terminal of this embodiment can obtain the file information of all files corresponding to the file write event. The file information of this embodiment includes the file name of each file and the attribute information of each file, and the attribute information reflects the file data volume of the file, that is, the file size. Therefore, this embodiment can determine the file data volume of each file based on the file information. In one implementation, when this embodiment classifies a number of files based on the file data volume, it includes the following steps: Step S101, sort all files from small to large according to the file data volume to obtain a file sequence; Step S102: Obtain the median of the file data volume of all files, and determine the file corresponding to the median; Step S103, determining a classification node based on the file corresponding to the median; Step S104: classify a plurality of files based on the classification node to obtain the first category files and the second category files.

[0027] Specifically, the present embodiment can first sort all the files according to the file data volume from small to large to obtain a file sequence. Then, the median of the file data volume is determined from the file data volume of all the files, and the median is the middle value of all the file data volumes. Then the file corresponding to the median is determined. Then, based on the file corresponding to the median, a classification node is determined, and the classification node can be used to divide the file sequence to obtain a first type of file and a second type of file. In specific applications, the present embodiment can obtain the first file data volume of the file corresponding to the median and the second file data volume of the files adjacent to the left and right of the file corresponding to the median in the file sequence. The second file data volume refers to the file data volume of the left adjacent file and the file data volume of the right adjacent file. When calculating the difference, the present embodiment can respectively calculate the difference between the first file data volume of the file corresponding to the median and the second file data volume of the left adjacent file, and calculate the difference between the first file data volume of the file corresponding to the median and the second file data volume of the right adjacent file. Next, the calculated difference is compared with the preset difference threshold. If the difference is greater than the difference threshold (i.e., any one of the two calculated differences is greater than the difference threshold), it means that the file corresponding to the median has a large difference in file data volume from the file adjacent to the left and the file adjacent to the right. Therefore, the file corresponding to the median can be used as a classification node. Specifically, this embodiment uses the file corresponding to the median as a classification node, and the file to the left of the file corresponding to the median in the file sequence is used as the first category of files, and the file to the right of the file corresponding to the median is used as the second category of files, so as to achieve the classification of several files. The file data volume of each file in the first category of files obtained by classification is smaller than the file data volume of each file in the second category of files. Therefore, the first category of files is a small file.

[0028] If the two calculated differences are compared with the preset difference threshold, and the differences are both less than the difference threshold, it means that the file corresponding to the median has a small difference in file data volume between the file adjacent to the left and the file adjacent to the right, so it cannot be used as a classification node. At this time, the file adjacent to the left and the file adjacent to the right of the file corresponding to the median in the file sequence can be obtained respectively, and then the determined left adjacent file and right adjacent file are re-analyzed respectively to determine the classification node. The analysis process is the same as the above analysis method, which is to find the adjacent files, and then calculate the difference between the file data volume, and compare the difference with the difference threshold. Only when the difference is greater than the difference threshold, the classification node can be determined, so as to obtain the first category file and the second category file based on the classification node. This embodiment determines the classification node based on the file of the median, which can not only more reasonably divide the file sequence into two categories of files with different file data volumes, but also quickly determine the classification node, so as to better realize file classification.

[0029] In another implementation, the present embodiment may also pre-set a file threshold (such as 1024kb), and then compare the file data volume of all files with the file threshold respectively, so as to screen out files whose file data volume is less than the file threshold, and classify the files whose file data volume is less than the file threshold as first-category files, and classify the files whose file data volume is greater than the file threshold as second-category files, thereby realizing file classification.

[0030] Step S200: start a first data cache program corresponding to the first type of files, and cache the first type of files to a local storage device based on the first data cache program.

[0031] After classifying all the files, the first category files and the second category files are obtained. Since the file data volume of each file in the first category files is smaller than the file data volume of each file in the second category files, the first category files are small files. In this embodiment, for small files, a preset first data cache program can be started. The first data cache program can cache the first category files to a local storage device first, such as caching to Figure 2 In the NVMe (solid state drive) in the present embodiment, the first type of files are first cached to the local storage device so that the first type of files can be synchronized from the local storage device to the Ceph cluster later.

[0032] Step S300: obtain a target storage device corresponding to the local storage device in the distributed file system, and synchronize the first type of files to the target storage device, wherein both the local storage device and the target storage device interact with the Ceph cluster.

[0033] After caching the first type of files to the local storage device, this embodiment can determine the target storage device corresponding to the local storage device from the distributed file system, and then store the first type of files cached in the local storage device to the target storage device. The local storage device and the target storage device of this embodiment both interact with the Ceph cluster. Therefore, this embodiment saves the first type of files to the target storage device, thereby caching the first type of files to the nodes of the Ceph cluster.

[0034] In one implementation, when saving the first type of files to the target storage device, this embodiment includes the following steps: Step S301: obtaining a preset mapping file, wherein the mapping file reflects the corresponding relationship between a plurality of storage devices; Step S302: Match the local storage device with the mapping file to determine a target storage device corresponding to the local storage device.

[0035] The present embodiment pre-sets a mapping file, which reflects the corresponding relationship between several storage devices, and the corresponding relationship can be a one-to-one corresponding relationship or a one-to-many corresponding relationship. Therefore, the present embodiment matches the local storage device with the mapping file, and can find the target storage device corresponding to the local storage device, and then stores the first type of files stored in the local storage device to the target storage device, thereby realizing the synchronous cache of the first type of files. In the present embodiment, the target storage device and the local storage device of the present embodiment can be devices of the same type or different types of devices. In addition, the local storage device and the target storage device are respectively located in different terminals in the Ceph cluster, so that in the process of synchronizing the first type of files from the local storage device to the target storage device, the purpose of transmitting the first type of files to the nodes of the Ceph cluster is realized. When the local storage device corresponds to multiple target storage devices, the first type of files can be synchronized to multiple target storage devices, that is, the first type of files are synchronized to multiple different terminals, so that the first type of files are synchronized to different nodes in the Ceph cluster, and because the file data volume of the first type of files is small, the whole process is easy to realize.

[0036] In addition, since the first type of files are cached in the local storage device and the target storage device, when the first type of files are subsequently transferred to other nodes in the Ceph cluster, the first type of files can be transferred from the local storage device or the target storage device to other nodes in the Ceph cluster. In this way, even if a terminal fails, the first type of files can be transferred to the Ceph cluster from another terminal, thereby solving the problem of single point failure of the cache and ensuring that data will not be lost.

[0037] In actual applications, each terminal is equipped with a storage device. This embodiment can map all storage devices that interact with the Ceph cluster based on the RDMA over Converged Ethernet (RoCE) technology to form a corresponding relationship, so as to achieve global mapping of all terminals in the Ceph cluster. Figure 2 As shown in the figure, the fuse client on the left and the fuse client on the right are both provided with NVMe (solid state drive), and the NVMe (solid state drive) in the fuse client on the left is mapped with the NVMe (solid state drive) in the fuse client on the right, so that when the fuse client on the left receives the first type of files and caches the first type of files to its own NVMe (solid state drive), the first type of files can be synchronized to the NVMe (solid state drive) in the fuse client on the right, thereby realizing synchronous caching of the first type of files.

[0038] This embodiment can first cache files with small file data volumes into a local storage device, and then synchronize them to the corresponding target storage device, thereby synchronizing the first type of files to different nodes in the Ceph cluster, without the need to go through the process of first sending the data of the entire file to the OSD in the Ceph cluster, and then the OSD is responsible for storing the data in each node of the Ceph cluster as in the prior art. Therefore, it can not only improve the efficiency of data reading and writing, but also solve the problem of cache single point failure.

[0039] In another implementation, for the second type of files, this embodiment can directly store the second type of files in the Ceph cluster. Since the second type of files have a large amount of file data, directly storing them in each node of the Ceph cluster will save time and improve the efficiency of data reading and writing.

[0040] Based on the above embodiments, the present invention also provides a cache control system based on file data volume, such as Figure 3 As shown in , the system includes: a file classification module 10, a local cache module 20 and a synchronization cache module 30. Specifically, the file classification module 10 is used to determine the file data volume of several files written, classify several files based on the file data volume, and obtain first-category files and second-category files, and the file data volume of the first-category files is less than the file data volume of the second-category files. The local cache module 20 is used to start a first data cache program corresponding to the first-category files, and cache the first-category files to a local storage device based on the first data cache program. The synchronization cache module 30 is used to obtain a target storage device corresponding to the local storage device in a distributed file system, and synchronize the first-category files to the target storage device, wherein both the local storage device and the target storage device interact with a Ceph cluster.

[0041] In one implementation, the file classification module 10 includes: A file acquisition unit, configured to acquire all files corresponding to a file writing event and determine file information of all files after acquiring the file writing event; The data volume analysis unit is used to determine the file data volume of each file based on the file information, wherein the file data volume reflects the amount of space occupied by the file on the storage device.

[0042] In one implementation, the file classification module 10 further includes: A file sequence determination unit, used to sort all files in ascending order according to the file data volume to obtain a file sequence; A median analysis unit, used to obtain the median of the file data volume of all files and determine the file corresponding to the median; A classification node analysis unit, used to determine a classification node based on the file corresponding to the median; The classification implementation unit is used to classify a plurality of files based on the classification node to obtain the first category files and the second category files.

[0043] In one implementation, the classification node analysis unit includes: A first file data volume analysis subunit, configured to obtain a first file data volume of the file corresponding to the median; A second file data volume analysis subunit is used to respectively obtain second file data volumes of files adjacent to the left and to the right of the file corresponding to the median in the file sequence; a difference calculation subunit, configured to calculate a difference between the first file data volume and the second file data volume; The classification node determination subunit is used to take the file corresponding to the median as a classification node if the difference is greater than the difference threshold.

[0044] In one implementation, the classification node analysis unit further includes: an adjacent file determination subunit, configured to determine, if the differences are all less than the difference threshold, files adjacent to the left and to the right of the file corresponding to the median in the file sequence; The classification node redetermining subunit is used to redetermine the classification node based on the files adjacent to the left and to the right of the file corresponding to the median.

[0045] In one implementation, the synchronization cache module 30 includes: A mapping file acquisition unit, used to acquire a preset mapping file, wherein the mapping file reflects the corresponding relationship between a plurality of storage devices; The target storage device determining unit is used to match the local storage device with the mapping file to determine the target storage device corresponding to the local storage device.

[0046] In one implementation, the system further includes: A second cache program starting unit, used to start a second data cache program corresponding to the second type of file; The second type of file saving unit is used to save the second type of files in the Ceph cluster based on the second data caching program.

[0047] The working principles of each module in the cache control system based on file data volume in this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0048] Each module in the cache control system based on file data volume can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the terminal in the form of hardware, or can be stored in the memory in the terminal in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0049] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 4 The terminal may include one or more processors 100 ( Figure 4 Only one is shown in the figure), a memory 101 and a computer program 102 stored in the memory 101 and executable on one or more processors 100, for example, a sleep analysis program based on multi-sensor data. When one or more processors 100 execute the computer program 102, the various steps in the embodiment of the sleep analysis method based on multi-sensor data can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, the functions of various modules / units in the embodiment of the sleep analysis system based on multi-sensor data can be implemented, which is not limited here.

[0050] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0051] In one embodiment, the memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0052] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0053] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operating database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double operational data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cache control method based on file data volume, characterized in that: The method comprises: Determine the file data volume of the plurality of files written, and classify the plurality of files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; Starting a first data cache program corresponding to the first type of files, and caching the first type of files to a local storage device based on the first data cache program; Obtain a target storage device corresponding to the local storage device in a distributed file system, and synchronize the first type of files to the target storage device, wherein both the local storage device and the target storage device interact with a Ceph cluster.

2. The cache control method based on file data volume according to claim 1, characterized in that: The determining the file data volume of the plurality of files written includes: After a file write event is obtained, all files corresponding to the file write event are obtained, and file information of all files is determined; Based on the file information, the file data volume of each file is determined, wherein the file data volume reflects the amount of space occupied by the file on the storage device.

3. The cache control method based on file data volume according to claim 1, characterized in that: The classifying the plurality of files based on the file data volume to obtain first-category files and second-category files includes: Sort all files from small to large according to the file data volume to obtain a file sequence; Obtaining the median of the file data volume of all files, and determining the file corresponding to the median; Determine a classification node based on the file corresponding to the median; A plurality of files are classified based on the classification node to obtain the first category files and the second category files.

4. The cache control method based on file data volume according to claim 3, characterized in that: The determining of the classification node based on the file corresponding to the median includes: Obtain the first file data volume of the file corresponding to the median; Respectively obtain the second file data amounts of the files adjacent to the left and to the right of the file corresponding to the median in the file sequence; Calculating a difference between the first file data volume and the second file data volume; If there is a difference greater than the difference threshold, the file corresponding to the median is used as a classification node.

5. The cache control method based on file data volume according to claim 4, characterized in that: The determining of the classification node based on the file corresponding to the median includes: If the differences are all smaller than the difference threshold, determining the files adjacent to the left and the right of the file corresponding to the median in the file sequence; The classification node is re-determined based on the files adjacent to the left and the right of the file corresponding to the median.

6. The cache control method based on file data volume according to claim 1, characterized in that: The obtaining of the target storage device corresponding to the local storage device in the distributed file system includes: Obtaining a preset mapping file, wherein the mapping file reflects the corresponding relationship between a plurality of storage devices; The local storage device is matched with the mapping file to determine a target storage device corresponding to the local storage device.

7. The cache control method based on file data volume according to claim 1, characterized in that: The method further comprises: Starting a second data cache program corresponding to the second type of files; Based on the second data caching program, the second type of files are saved in the Ceph cluster.

8. A cache control system based on file data volume, characterized in that: The system is used to implement the steps of the cache control method based on file data volume according to any one of claims 1 to 7, and the system includes: A file classification module, used to determine the file data volume of the plurality of files written, and classify the plurality of files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; A local cache module, used to start a first data cache program corresponding to the first type of files, and cache the first type of files to a local storage device based on the first data cache program; A synchronization cache module is used to obtain a target storage device corresponding to the local storage device in a distributed file system, and synchronize the first type of files to the target storage device, wherein both the local storage device and the target storage device interact with a Ceph cluster.

9. A terminal, characterized in that: The terminal includes a memory, a processor, and a cache control program based on file data volume stored in the memory and executable on the processor. When the processor executes the cache control program based on file data volume, the steps of the cache control method based on file data volume as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a cache control program based on file data volume. When the cache control program based on file data volume is executed by a processor, the steps of the cache control method based on file data volume as described in any one of claims 1 to 7 are implemented.

Images