Global cache control method and system of distributed file system, terminal and medium

Abstract

The invention discloses a global cache control method and system for a distributed file system, a terminal and a medium, and the method comprises the steps: obtaining all clients interacting with a Ceph cluster, determining a first client and a second client based on all clients, each client in the Ceph cluster having at least one client corresponding to the client, and determining the first client and the second client according to the first client and the second client, only one first client is provided, and a plurality of second clients are provided; controlling the file to be written into the first client, and caching the written file based on the first client; and based on the first client, synchronously caching the file cached in the first client to the second client so as to realize global caching of the file by all clients. According to the invention, after the file is written into the first client, the first client can synchronously cache the file to the remaining clients, so that global caching of the file by all the clients is realized, and the efficiency and stability of data caching are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed file cache, and in particular to a global cache control method, system, terminal and medium of a distributed file system. Background Art

[0002] In the process of processing file upload, for a distributed file system, the data of the entire file needs to be sent to each node in the Ceph cluster through the network. In the prior art, the client of the distributed file system needs to send the data of the entire file to the OSD in the Ceph cluster through the network, and then the OSD is responsible for storing the data in each node of the Ceph cluster, rather than directly passing it to the MDS (MetadataSeve, a component for storing file system metadata) of other clients. This process reduces the efficiency of file transfer and greatly wastes the time of file transfer.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a global cache control method, system, terminal and medium of a distributed file system are provided, aiming to solve the problems that the prior art reduces the file transfer efficiency and greatly wastes the file transfer time when processing file uploads.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a global cache control method for a distributed file system, wherein the method comprises: Obtain all clients interacting with the Ceph cluster, and determine a first client and a second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients; Controlling the writing of files to the first client, and caching the written files based on the first client; Based on the first client, the file cached in the first client is synchronously cached in the second client to achieve global caching of files by all clients.

[0006] In one implementation, determining the first client and the second client based on all clients includes: Obtaining the corresponding relationships between all clients, and forming a relationship table based on the corresponding relationships, wherein the relationship table reflects the number of clients that have corresponding relationships with each client; Based on the relationship table, a client with the largest number of clients having corresponding relationships is determined, and the client with the largest number of clients having corresponding relationships is used as the first client, and the remaining clients after removing the first client from all clients are used as the second clients.

[0007] In one implementation, determining the first client and the second client based on all clients includes: Sort all clients in ascending order of storage space to obtain a client sequence; Determine a candidate client from the client sequence, wherein the candidate client is a client having a storage space greater than a preset space threshold; Any one of the candidate clients is used as the first client, and the remaining clients after removing the first client from all the clients are used as the second clients.

[0008] In one implementation, controlling the writing of a file to the first client, and caching the written file based on the first client, includes: After the files are written to the first client, determining the file data volume of the written files, and classifying the files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; caching the first type of files in a local storage device in the first client; The second type of files are saved in the Ceph cluster.

[0009] In one implementation, a plurality of files are classified based on the file data volume to obtain first-category files and second-category files, including: Get the preset file threshold; Comparing the file data volumes of all files with the file thresholds respectively; Files whose file data volume is less than the file threshold are regarded as first-category files, and files whose file data volume is greater than the file threshold are regarded as second-category files.

[0010] In one implementation, based on the first client, synchronously caching the file cached in the first client to the second client to implement global caching of the file by all clients includes: Taking the first client as a root node and the second client as a child node of the root node, and constructing a tree diagram based on the root node and the child nodes; Based on the node relationship in the tree diagram, starting from the first client, the first type of files cached in the first client are synchronously cached in the second client.

[0011] In one implementation, taking the first client as a root node and the second client as a child node of the root node includes: Acquire, from the second clients, clients that have a corresponding relationship with the first client to obtain associated clients; Taking the first client as a root node and taking the associated client as a first child node of the root node; Acquire a client that has a corresponding relationship with the associated client, and use the client that has a corresponding relationship with the associated client as a second child node; And so on, until the last client in the second client is traversed and all child nodes are obtained.

[0012] In a second aspect, an embodiment of the present invention further provides a global cache control system of a distributed file system, the system is used to implement the steps of the global cache control method of the distributed file system described in the above scheme, and the system includes: A client partitioning module is used to obtain all clients interacting with the Ceph cluster, and determine a first client and a second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is only one first client, and there are multiple second clients; A cache processing module, used for controlling the writing of files to the first client, and performing cache processing on the written files based on the first client; The file synchronization module is used to synchronously cache the files cached in the first client to the second client based on the first client, so as to realize global caching of files by all clients.

[0013] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and a global cache control program of a distributed file system stored in the memory and executable on the processor, and when the processor executes the global cache control program of the distributed file system, the steps of the global cache control method of the distributed file system of any one of the above-mentioned schemes are implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a global cache control program of a distributed file system is stored on the computer-readable storage medium, and when the global cache control program of the distributed file system is executed by a processor, the steps of the global cache control method of the distributed file system described in any one of the above-mentioned schemes are implemented.

[0015] Beneficial effect: Compared with the prior art, the present invention provides a global cache control method for a distributed file system. The present invention first obtains all clients interacting with the Ceph cluster, and determines the first client and the second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients. Then, the control file is written to the first client, and the written file is cached based on the first client. Finally, based on the first client, the file cached in the first client is synchronously cached to the second client to achieve global caching of files by all clients. The present invention can determine the first client from all clients interacting with the Ceph cluster, and the first client can serve as a leader. After the file is written to the first client, the first client can synchronously cache the file to the remaining clients, thereby achieving global caching of files by all clients and improving the efficiency and stability of data caching. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention provides a flowchart of a preferred embodiment of a global cache control method for a distributed file system.

[0017] Figure 2 A schematic diagram of an application of a global cache control method for a distributed file system provided by an embodiment of the present invention.

[0018] Figure 3 A schematic diagram of the architecture of a global cache control system of a distributed file system provided by an embodiment of the present invention.

[0019] Figure 4 A functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations or steps, nor must they be executed in the order described. For example, some operations or steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0022] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms. It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish between identical or similar items with substantially identical functions and effects. For example, the first control information and the second control information are only used to distinguish different control information, and their order is not limited. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences. It should be further understood that the term “and / or” used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0023] In order to solve the problems of the prior art, this embodiment provides a global cache control method for a distributed file system. Based on the method of this embodiment, global caching of files by all clients can be realized, thereby improving the efficiency and stability of data caching. In specific applications, this embodiment first obtains all clients interacting with the Ceph cluster, and determines the first client and the second client based on all clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients. Then, the control file is written to the first client, and the written file is cached based on the first client. Finally, based on the first client, the file cached in the first client is synchronously cached in the second client to realize global caching of files by all clients. This embodiment can determine the first client from all clients interacting with the Ceph cluster, and the first client can be used as a leader. After the file is written to the first client, the first client can synchronously cache the file to the remaining clients.

[0024] The global cache control method of the distributed file system of this embodiment can be applied to a terminal, which can be a mobile phone, a computer, a smart TV or other intelligent product terminal. Figure 1 As shown in , the global cache control method of the distributed file system includes the following steps: Step S100, obtain all clients interacting with the Ceph cluster, and determine the first client and the second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients.

[0025] like Figure 2 As shown in , the cache control method based on file data volume of this embodiment can be applied to a distributed file system, which includes a Ceph cluster and a plurality of fuse clients interacting with the Ceph cluster. Among all the clients interacting with the Ceph cluster, each client has at least one corresponding client. In other words, each client has a corresponding relationship with at least one other client. Then, this embodiment can divide all the clients, and determine the first client and the second client therefrom, and the determined first client has one and only one, and the second client has multiple.

[0026] In one implementation, when dividing all clients, this embodiment includes: Step S101: Obtain the corresponding relationships between all clients, and form a relationship table based on the corresponding relationships, wherein the relationship table reflects the number of clients that have corresponding relationships with each client; Step S102: based on the relationship table, determine the client with the largest number of clients having corresponding relationships, and use the client with the largest number of clients having corresponding relationships as the first client, and use the remaining clients after removing the first client from all clients as the second client.

[0027] Specifically, since there is a correspondence between all clients, and each client corresponds to at least one other client, this embodiment can summarize the correspondence between all clients to form a relationship table, which reflects the number of other clients corresponding to each client, that is, the number of clients with which each client has a corresponding relationship. Each row in the relationship table is a client and the number of clients corresponding to the client. In actual application, this embodiment numbers all clients, and each client has a unique number, which is convenient for summarizing the correspondence between clients. The left column of the relationship table is the client number, and the right column is the corresponding number of clients. Based on the relationship table, it can be determined which client corresponds to the largest number of clients, that is, which client has the largest number of correspondences. In this way, the client with the largest number of clients with corresponding relationships can be used as the first client, and the remaining clients after removing the first client from all clients are used as the second client, thereby realizing the division of clients, as shown in Table 1 below.

[0028] Table 1

[0029] As can be seen from Table 1, each row on the left corresponds to a client, and each row on the right is the number of clients corresponding to each client. As can be seen from Table 1, the number of clients corresponding to client 2 is the largest, that is, the number of other clients corresponding to client 2 is the largest, so client 2 is used as the first client, and the remaining clients 1, 3, 4, and 5 are all used as second clients. In this embodiment, the client with the largest number of clients corresponding to each other is used as the first client, so that in the subsequent file caching process, the first client can be used as a leader to synchronize files to other clients, which is more efficient.

[0030] In another implementation, when dividing all clients, this embodiment may further include: Step S11, sort all clients in ascending order of storage space to obtain a client sequence; Step S12: determining a candidate client from the client sequence, wherein the candidate client is a client whose storage space is greater than a preset space threshold; Step S13: taking any one of the candidate clients as the first client, and taking all the clients except the first client as the second clients.

[0031] Specifically, this embodiment can sort all clients according to the storage space from small to large to obtain a client sequence, and the storage space is the size of the client's storage capacity. After obtaining the client sequence, this embodiment can determine candidate clients from the client sequence, for example, select clients whose storage space is greater than a preset space threshold from the client sequence as candidate clients. Of course, in another implementation, this embodiment can also select the client with the storage space at the front in the client sequence as candidate clients, such as the top three as candidate clients. The storage space of these candidate clients is relatively large, and it is convenient to implement file storage. Therefore, this embodiment can use any one of the candidate clients as the first client, and the remaining clients after removing the first client from all the clients as the second client. The first client determined in this embodiment is a client with a larger storage space, which is convenient for caching the written files in the subsequent file caching process.

[0032] Step S200: Control the writing of files to the first client, and perform cache processing on the written files based on the first client.

[0033] After determining the first client and the second client, this embodiment can write the file to the first client first. Since the number of clients corresponding to the first client is the largest, or the first client is a client with a larger storage space, it is convenient for the first client to cache the written file and can also quickly synchronize the file to the second client.

[0034] In one implementation, when the first client of this embodiment performs cache processing on the written file, the process includes the following steps: Step S201: after writing files to the first client, determine the file data volume of the written files, classify the files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; Step S202: caching the first type of files in a local storage device in the first client; Step S203: Save the second type of files in the Ceph cluster.

[0035] Specifically, when a file is written to the first client, such as by Figure 2 When the NFS (Network File System) in the storage medium writes files to the fuse client, this embodiment will determine the file data volume of several files written. The file data volume can be the file size, that is, the amount of space occupied by the file on the storage device. After determining the file data volume of the file, the several files can be classified by the file data volume to obtain the first category of files and the second category of files. In this embodiment, the file data volume of the first category of files is less than the file data volume of the second category of files. The purpose of classifying files in this embodiment is to facilitate different caching methods for files according to different file data volumes, thereby improving caching efficiency.

[0036] In specific applications, after the first client of this embodiment obtains a file write event, it can obtain the file information of all files corresponding to the file write event. The file information of this embodiment includes the file name of each file and the attribute information of each file, and the attribute information reflects the file data volume of the file, that is, the file size. Therefore, this embodiment can determine the file data volume of each file based on the file information. This embodiment pre-sets a file threshold (such as 1024kb), and compares the file data volume of all files with the file threshold respectively, so as to filter out files with a file data volume less than the file threshold, and treats files with a file data volume less than the file threshold as first-class files, and treats files with a file data volume greater than the file threshold as second-class files, thereby realizing file classification.

[0037] In another implementation, when classifying the written files, the present embodiment can also classify the files by sorting all the files to obtain a file sequence, and then implement the file classification for the file sequence. Specifically, the present embodiment can first sort all the files according to the file data volume from small to large to obtain a file sequence. Then, the median of the file data volume is determined from the file data volume of all the files, and the median is the middle value of all the file data volumes. Then the file corresponding to the median is determined. Then, based on the file corresponding to the median, a classification node is determined, and the classification node can be used to divide the file sequence to obtain a first type of file and a second type of file. In specific applications, the present embodiment can obtain the first file data volume of the file corresponding to the median and the second file data volume of the files adjacent to the left and right of the file corresponding to the median in the file sequence. The second file data volume refers to the file data volume of the left adjacent file and the file data volume of the right adjacent file. When calculating the difference, the present embodiment can respectively calculate the difference between the first file data volume of the file corresponding to the median and the second file data volume of the left adjacent file, and calculate the difference between the first file data volume of the file corresponding to the median and the second file data volume of the right adjacent file. Next, the calculated difference is compared with the preset difference threshold. If the difference is greater than the difference threshold (i.e., any one of the two calculated differences is greater than the difference threshold), it means that the file corresponding to the median has a large difference in file data volume from the file adjacent to the left and the file adjacent to the right. Therefore, the file corresponding to the median can be used as a classification node. Specifically, in this embodiment, the file corresponding to the median is used as a classification node, and the file to the left of the file corresponding to the median in the file sequence is used as the first category of files, and the file to the right of the file corresponding to the median is used as the second category of files, so as to achieve file classification. The file data volume of each file in the first category of files obtained by classification is smaller than the file data volume of each file in the second category of files. Therefore, the first category of files is a small file.

[0038] If the two calculated differences are compared with the preset difference threshold, and the differences are both less than the difference threshold, it means that the file corresponding to the median has a small difference in file data volume between the file adjacent to the left and the file adjacent to the right, so it cannot be used as a classification node. At this time, the file adjacent to the left and the file adjacent to the right of the file corresponding to the median in the file sequence can be obtained respectively, and then the determined left adjacent file and right adjacent file are re-analyzed respectively to determine the classification node. The analysis process is the same as the above analysis method, which is to find the adjacent files, and then calculate the difference between the file data volume, and compare the difference with the difference threshold. Only when the difference is greater than the difference threshold, the classification node can be determined, so as to obtain the first category file and the second category file based on the classification node. This embodiment determines the classification node based on the file of the median, which can not only more reasonably divide the file sequence into two categories of files with different file data volumes, but also quickly determine the classification node, so as to better realize file classification.

[0039] Step S300: Based on the first client, the file cached in the first client is synchronously cached in the second client to achieve global caching of files by all clients.

[0040] After caching the first type of files in the local storage device of the first client, this embodiment can synchronously cache the first type of files to the second client, thereby realizing global caching of the first type of files by all clients, so that the first type of files can reach each node in the Ceph cluster, that is, reach each client in the Ceph cluster.

[0041] In one implementation, this embodiment includes the following steps when implementing global caching: Step S301: taking the first client as a root node and the second client as a child node of the root node, and constructing a tree diagram based on the root node and the child nodes; Step S302: Based on the node relationship in the tree diagram, starting from the first client, the first type of files cached in the first client are synchronously cached in the second client.

[0042] Since some clients in the first client and the second client store direct correspondence, and other clients that do not correspond to the first client also have corresponding clients, a tree diagram can be constructed based on the correspondence of all clients, and the tree diagram reflects the correspondence between all clients. Specifically, this embodiment obtains clients that have a corresponding relationship with the first client from the second client to obtain associated clients. Then the first client is taken as the root node, and the associated client is taken as the first child node of the root node. Next, the client that has a corresponding relationship with the associated client is obtained, and the client that has a corresponding relationship with the associated client is taken as the second child node. And so on, repeat the above operation of obtaining clients with corresponding relationships until the last client in the second client is traversed, and all child nodes can be obtained. Finally, a tree diagram is constructed based on the root node and all the child nodes. From the tree diagram, it can be seen that the first child node is the child node of the root node, and the second child node is the child node of the first child node. Therefore, based on the node relationship of the tree diagram, the first type of file can be cached from the first client to all the second clients in sequence, that is, the first type of file is cached from the root node to the first node, the second node, and finally the Nth node. In actual applications, the corresponding relationship between the first client and the second client is reflected in the corresponding relationship between the storage devices, for example, Figure 2 As shown in , the left fuse client and the right fuse client are both provided with NVMe (solid state drive), and the NVMe (solid state drive) in the left fuse client corresponds to the NVMe (solid state drive) in the right fuse client, thus realizing the correspondence between the two clients. Based on this, this embodiment can synchronously cache the first type of files stored in the first client to the second client according to the node relationship of the above tree diagram, so that global caching can be realized, and the synchronization of the first type of files to different nodes in the Ceph cluster can be realized, and since the file data volume of the first type of files is small, the whole process is easy to implement.

[0043] In addition, since the first type of files are cached in both the first client and the second client, when the first type of files are subsequently transferred to other nodes in the Ceph cluster, the first type of files can be transferred to other nodes in the Ceph cluster from the first client or the second client. In this way, even if a terminal fails, the first type of files can be transferred to the Ceph cluster from another terminal, thereby solving the problem of single point failure of the cache and ensuring that data will not be lost.

[0044] In summary, this embodiment can determine the first client from all clients interacting with the Ceph cluster. The first client can serve as a leader. After the file is written to the first client, the first client can synchronously cache the file to the remaining clients, thereby realizing global caching of the file by all clients and improving the efficiency and stability of data caching.

[0045] Based on the above embodiment, the present invention also provides a global cache control system for a distributed file system, the system is used to implement the steps of the global cache control method for a distributed file system described in the above embodiment, such as Figure 3 As shown in, the system includes: a client division module 10, a cache processing module 20 and a file synchronization module 30. Specifically, the client division module 10 is used to obtain all clients interacting with the Ceph cluster, and determine the first client and the second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients. The cache processing module 20 is used to control the writing of files to the first client, and cache the written files based on the first client. The file synchronization module 30 is used to cache the files cached in the first client to the second client based on the first client, so as to realize global caching of files by all clients.

[0046] In one implementation, the client partitioning module 10 includes: A relationship table determination unit, used to obtain the corresponding relationship between all clients, and form a relationship table based on the corresponding relationship, wherein the relationship table reflects the number of clients that have a corresponding relationship with each client; The first client determination unit is used to determine the client with the largest number of clients having corresponding relationships based on the relationship table, and use the client with the largest number of clients having corresponding relationships as the first client, and use the remaining clients after removing the first client from all clients as the second client.

[0047] In one implementation, the client partitioning module 10 further includes: A client sequence determination unit, used to sort all clients in ascending order of storage space to obtain a client sequence; A candidate client determination unit, configured to determine a candidate client from a client sequence, wherein the candidate client is a client having a storage space greater than a preset space threshold; The second client determination unit is configured to select any one of the candidate clients as the first client and the remaining clients from all the clients excluding the first client as the second clients.

[0048] In one implementation, the cache processing module 20 includes: A file classification unit, configured to determine, after the files are written to the first client, file data volumes of the written files, and classify the files based on the file data volumes to obtain first-category files and second-category files, wherein the file data volumes of the first-category files are smaller than the file data volumes of the second-category files; A first-category file caching unit, configured to cache the first-category files in a local storage device in the first client; The second type file saving unit is used to save the second type files in the Ceph cluster.

[0049] In one implementation, the file classification unit includes: A file threshold acquisition subunit, used to acquire a preset file threshold; A file comparison subunit, used to compare the file data volumes of all files with the file threshold respectively; The file division subunit is used to classify files with a file data volume less than a file threshold as first-category files, and to classify files with a file data volume greater than the file threshold as second-category files.

[0050] In one implementation, the file synchronization module 30 includes: A tree diagram construction unit, configured to use the first client as a root node, the second client as a sub-node of the root node in sequence, and to construct a tree diagram based on the root node and the sub-stages; The file caching unit is used to synchronously cache the first type of files cached in the first client to the second client starting from the first client based on the node relationship in the tree diagram.

[0051] In one implementation, the tree diagram construction unit includes: an associated client determination subunit, configured to obtain, from the second clients, clients that have a corresponding relationship with the first client, to obtain associated clients; A first subnode determining subunit, configured to use the first client as a root node and an associated client as a first subnode of the root node; A second sub-node determining sub-unit, configured to obtain a client corresponding to the associated client, and use the client corresponding to the associated client as a second sub-node; All child nodes determine the child units, and so on, until the last client in the second client is traversed to obtain all child nodes.

[0052] The working principles of each module in the global cache control system of the distributed file system of this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0053] Each module in the global cache control system of the above distributed file system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the terminal in the form of hardware, or can be stored in the memory in the terminal in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0054] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 4 The terminal may include one or more processors 100 ( Figure 4 Only one is shown in the figure), a memory 101 and a computer program 102 stored in the memory 101 and executable on one or more processors 100, for example, a sleep analysis program based on multi-sensor data. When one or more processors 100 execute the computer program 102, the various steps in the embodiment of the sleep analysis method based on multi-sensor data can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, the functions of various modules / units in the embodiment of the sleep analysis system based on multi-sensor data can be implemented, which is not limited here.

[0055] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0056] In one embodiment, the memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0057] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0058] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operating database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double operational data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A global cache control method for a distributed file system, characterized in that: The method comprises: Obtain all clients interacting with the Ceph cluster, and determine a first client and a second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is one and only one first client, and there are multiple second clients; Controlling the writing of files to the first client, and caching the written files based on the first client; Based on the first client, the file cached in the first client is synchronously cached in the second client to achieve global caching of files by all clients.

2. The global cache control method of a distributed file system according to claim 1, characterized in that: Determining a first client and a second client based on all clients includes: Obtaining the corresponding relationships between all clients, and forming a relationship table based on the corresponding relationships, wherein the relationship table reflects the number of clients that have corresponding relationships with each client; Based on the relationship table, a client with the largest number of clients having corresponding relationships is determined, and the client with the largest number of clients having corresponding relationships is used as the first client, and the remaining clients after removing the first client from all clients are used as the second clients.

3. The global cache control method of a distributed file system according to claim 1, characterized in that: Determining a first client and a second client based on all clients includes: Sort all clients in ascending order of storage space to obtain a client sequence; Determine a candidate client from the client sequence, wherein the candidate client is a client having a storage space greater than a preset space threshold; Any one of the candidate clients is used as the first client, and the remaining clients after removing the first client from all the clients are used as the second clients.

4. The global cache control method of a distributed file system according to claim 1, characterized in that: Controlling the writing of files to the first client, and caching the written files based on the first client, including: After the files are written to the first client, determining the file data volume of the written files, and classifying the files based on the file data volume to obtain first-category files and second-category files, wherein the file data volume of the first-category files is less than the file data volume of the second-category files; caching the first type of files in a local storage device in the first client; The second type of files are saved in the Ceph cluster.

5. The global cache control method of a distributed file system according to claim 4, characterized in that: Classifying a number of files based on the file data volume to obtain first-category files and second-category files includes: Get the preset file threshold; Comparing the file data volumes of all files with the file thresholds respectively; Files whose file data volume is less than the file threshold are regarded as first-category files, and files whose file data volume is greater than the file threshold are regarded as second-category files.

6. The global cache control method of a distributed file system according to claim 5, characterized in that: Based on the first client, synchronously caching the file cached in the first client to the second client to implement global caching of the file by all clients, including: Taking the first client as a root node and the second client as a child node of the root node, and constructing a tree diagram based on the root node and the child nodes; Based on the node relationship in the tree diagram, starting from the first client, the first type of files cached in the first client are synchronously cached in the second client.

7. The global cache control method of a distributed file system according to claim 6, characterized in that: Taking the first client as a root node and the second client as a child node of the root node includes: Acquire, from the second clients, clients that have a corresponding relationship with the first client to obtain associated clients; Taking the first client as a root node and taking the associated client as a first child node of the root node; Acquire a client that has a corresponding relationship with the associated client, and use the client that has a corresponding relationship with the associated client as a second child node; And so on, until the last client in the second client is traversed and all child nodes are obtained.

8. A global cache control system for a distributed file system, characterized in that: The system is used to implement the steps of the global cache control method of the distributed file system according to any one of claims 1 to 7, and the system includes: A client partitioning module is used to obtain all clients interacting with the Ceph cluster, and determine a first client and a second client based on all the clients, wherein each client in the Ceph cluster has at least one corresponding client, there is only one first client, and there are multiple second clients; A cache processing module, used for controlling the writing of files to the first client, and performing cache processing on the written files based on the first client; The file synchronization module is used to synchronously cache the files cached in the first client to the second client based on the first client, so as to realize global caching of files by all clients.

9. A terminal, characterized in that: The terminal includes a memory, a processor, and a global cache control program of a distributed file system stored in the memory and executable on the processor. When the processor executes the global cache control program of the distributed file system, the steps of the global cache control method of the distributed file system as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a global cache control program for a distributed file system. When the global cache control program for a distributed file system is executed by a processor, the steps of the global cache control method for a distributed file system as described in any one of claims 1 to 7 are implemented.

Images