File Reading Method and Device

Abstract

The present application provides a file reading method and apparatus. The method is applied to a read cache control module which is within a file storage system. The file storage system further includes an MDS node and a DS node. The method includes: receiving a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located; determining whether the file size meets a preset file attribute policy; if it meets, determining whether the directory information meets a preset spatial locality policy; if it meets, obtaining the read requests received within a historical time and determining whether the read requests received within the historical time meet a preset temporal locality policy; if it meets, reading the file to be read and the associated files of the file to be read from a storage medium according to a preset read cache policy, and storing the file to be read and the associated files in a cache.

Description

Technical Field

[0001] This application relates to the field of computer storage, and particularly to a file reading method and apparatus. Background Art

[0002] Currently, some applications need to read the required files in as little time as possible. At this time, the file storage system needs a high number of file operations per second (abbreviation: OPS) and low latency.

[0003] To achieve the goal of reading multiple files as soon as possible, the application can choose to adopt a concurrent reading strategy, that is, send multiple file reading commands to the file storage system simultaneously, and improve the reading efficiency through concurrent access. However, there are also some applications that access the file storage system through a single thread to read files. At this time, in order to improve the file reading speed, the file storage system needs to have a lower latency. The general process of an application reading a file from the file storage system is as Figure 1 shown. Figure 1 It is a flowchart of an application reading a file from the file storage system.

[0004] In Figure 1 , the application (Application) sends a file metadata read request to the metadata server (abbreviation: MDS) node (step ①), and the MDS node returns the file metadata to the App (step ②). The App sends a file data read request to the data server (abbreviation: DS) node based on the file metadata (step ③), and the DS node returns the file data to the App (step ④).

[0005] For an application that accesses the file storage system through a single thread, reading each file requires repeating the above process, and after one file is read, the next file can be read. For example, an application wants to read 100 files of the same size from a certain file directory. For each file, if step ① takes 1 ms, step ② takes 3 ms, step ③ takes 1 ms, and step ④ takes 5 ms, then the total time for the application to read the file is 10 ms. From this calculation, it takes 1 s for the application to read 100 files, that is, its read OPS is 100.

[0006] In the case of single-threaded access, in order to improve the reading performance, it is necessary to reduce the access latency of the file. In the above steps, the latency of steps ① and ③ is relatively fixed, mainly network latency. The latency of steps ② and ④ depends on the situation. Whether it is the MDS or the DS node, if the content to be accessed hits the corresponding cache, the response latency can be greatly reduced. Therefore, reducing the latency of steps ② and ④ is the key to improving the single-threaded reading performance.

[0007] In the existing solution, if the file being read is a large file, such as an MB-level or even GB-level file, the latency in step ④ occupies the main part. For this reason, the Linux operating system has a built-in prefetching function for sequential reading of large files. When the operating system determines that the file should be in the prefetch mode, it pre-reads the file into the cache in advance.

[0008] However, the existing solution only solves the problem of improving the reading efficiency of large files, and does not improve the reading efficiency of small files. For example, as Figure 2 shown, Figure 2 is a schematic diagram of the structure of the files stored in the file storage system.

[0009] In Figure 2 , if the application needs to read all the files (File1 to File99) under "subdirectory 13" in a single thread, the sizes of these files are all small (tens of KB to hundreds of KB). According to the foregoing method, the operating system cannot prefetch the file metadata information and file data, and it cannot improve the reading efficiency. Summary of the Invention

[0010] In view of this, the present application provides a file reading method and device to solve the problem that in the existing file reading method, the file metadata information and file data of small files cannot be prefetched, and the reading efficiency cannot be improved.

[0011] In a first aspect, the present application provides a file reading method, which is applied to a read cache control module. The read cache control module is in a file storage system, and the file storage system further includes an MDS node and a DS node. The method includes:

[0012] Receiving a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located;

[0013] Judging whether the file size meets a preset file attribute policy;

[0014] If it meets, then judging whether the directory information meets a preset spatial locality policy;

[0015] If it meets, then obtaining the read requests received within a historical time, and judging whether the read requests received within the historical time meet a preset temporal locality policy;

[0016] If it meets, then reading the file to be read and the associated files of the file to be read from the storage medium according to a preset read cache policy, and storing the file to be read and the associated files in the cache.

[0017] In a second aspect, the present application provides a file reading device, which is applied to a read cache control module. The read cache control module is within a file storage system, and the file storage system further includes an MDS node and a DS node. The device includes:

[0018] a receiving unit, configured to receive a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and directory information of the directory where the file to be read is located;

[0019] a first determination unit, configured to determine whether the file size meets a preset file attribute policy;

[0020] a second determination unit, configured to, if it is satisfied, determine whether the directory information meets a preset spatial locality policy;

[0021] a third determination unit, configured to, if it is satisfied, obtain the read requests received within a historical time, and determine whether the read requests received within the historical time meet a preset temporal locality policy;

[0022] a read cache unit, configured to, if it is satisfied, according to a preset read cache policy, read the file to be read and the associated file of the file to be read from a storage medium, and store the file to be read and the associated file into a cache.

[0023] In a third aspect, the present application provides a network device, including a processor and a machine-readable storage medium. The machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is caused by the machine-executable instructions to execute the method provided in the first aspect of the present application.

[0024] Therefore, by applying the file reading method and device provided by the present application, the read cache control module receives a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and directory information of the directory where the file to be read is located; the read cache control module determines whether the file size meets a preset file attribute policy; if it is satisfied, the read cache control module determines whether the directory information meets a preset spatial locality policy; if it is satisfied, the read cache control module obtains the read requests received within a historical time, and determines whether the read requests received within the historical time meet a preset temporal locality policy; if it is satisfied, according to a preset read cache policy, the read cache control module reads the file to be read and the associated file of the file to be read from a storage medium, and stores the file to be read and the associated file into a cache.

[0025] In this way, the problem in the existing file reading method that the file metadata information and file data of small files cannot be prefetched and the reading efficiency cannot be improved is solved. When a single thread of an application centrally reads small files in a certain directory or certain directories, based on a preset caching policy, the reading efficiency can be greatly improved. In a hybrid flash configuration (a combination of HDD and SSD), the number of small files read per second can be increased by 2-3 times. Description of the Drawings

[0026] Figure 1 It is a network diagram for an application to read files from a file storage system;

[0027] Figure 2 It is a structural diagram of files stored in a file storage system;

[0028] Figure 3 It is a flowchart of the file reading method provided by an embodiment of the present application;

[0029] Figure 4 It is a network diagram of the application file reading method provided by an embodiment of the present application;

[0030] Figure 5 It is an internal structure diagram of the read cache control module provided by an embodiment of the present application;

[0031] Figure 6 It is a structural diagram of the file reading device provided by an embodiment of the present application;

[0032] Figure 7 It is a hardware structure diagram of a network device provided by an embodiment of the present application. Detailed Embodiments

[0033] Here, exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0034] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the" and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the corresponding listed items.

[0035] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0036] The file reading method provided by the embodiments of this application will be described in detail below. Refer to Figure 3 , Figure 3 which is a flowchart of the file reading method provided by the embodiments of this application. This method is applied to a read cache control module, and the read cache control module is within a file storage system. The file reading method provided by the embodiments of this application may include the following steps.

[0037] Step 310: Receive a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located.

[0038] Specifically, as Figure 4 shown, Figure 4 is a schematic networking diagram of the application file reading method provided by the embodiments of this application. This networking includes a host and a file storage system. An application (Application) has been set up in the host. The file storage system includes an MDS node, a DS node, and a read cache control module.

[0039] When the application accesses the file storage system based on a single thread, the application sends a read request to the MDS node or the DS node. The read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located.

[0040] After receiving the read request, the MDS node or the DS node transmits the read request to the read cache control module. After receiving the read request, the read cache control module obtains the file size of the file to be read and the directory information of the directory where the file to be read is located from the read request.

[0041] It can be understood that the application sends a file metadata read request to the MDS node, and the application sends a file data read request to the DS node. In the embodiments of this application, the above two types of requests are collectively referred to as read requests. This read request is the same as the read request generated and sent by existing applications, and other content included therein will not be described herein.

[0042] After the MDS node or the DS node receives a read request, it can search in the cache in the existing manner to check if there is a matching file metadata or file data. If it exists, it sends the file metadata or file data to the application and transmits the read request to the read cache control module. If it does not exist, it transmits the read request to the read cache control module.

[0043] Step 320: Determine whether the file size meets the preset file attribute policy.

[0044] Specifically, according to the description of step 310, after the read cache control module obtains the file size of the file to be read and the directory information of the directory where the file to be read is located, it determines whether the file size meets the file attribute policy. If it meets, it executes step 330; if it does not meet, it ends this processing.

[0045] Furthermore, the specific process for the read cache control module to determine whether the file size meets the file attribute policy is as follows:

[0046] The read cache control module determines whether the file size is not greater than the preset file size threshold; if it is not greater, it determines that the file size meets the file attribute policy; if it is greater, it determines that the file size does not meet the file attribute policy.

[0047] Even further, the preset file size threshold can be specifically 500KB. That is, files with a file size less than or equal to 500KB are determined to meet the file attribute policy, and such files can be called small files.

[0048] Step 330: If it meets, determine whether the directory information meets the preset spatial locality policy.

[0049] Specifically, according to the judgment in step 320, if the file size meets the file attribute policy, the read cache control module determines whether the directory information meets the spatial locality policy. If it meets, it executes step 340; if it does not meet, it ends this processing.

[0050] Furthermore, the specific process for the read cache control module to determine whether the directory information meets the spatial locality policy is as follows:

[0051] The read cache control module determines whether the directory information is a directory specified by the user (this directory includes the parent directory and the sub - directory); if it is, it determines that the directory information meets the spatial locality policy; if it is not, it determines that the directory information does not meet the spatial locality policy.

[0052] Even further, the directory specified by the user is a directory that the user predicted in advance that the application will read files from multiple times in the future, and when configuring the spatial locality policy, this directory is configured as the spatial locality policy.

[0053] Step 340: If the condition is met, obtain the read requests received within the historical time, and determine whether the read requests received within the historical time meet the preset temporal locality policy.

[0054] Specifically, according to the judgment in Step 340, if the directory information meets the spatial locality policy, the read cache control module obtains the read requests received within the historical time.

[0055] Among them, the historical time is a previous period of time. For example, the read requests received within the past 1 second. The read request is a request to access files under the same directory (the directory specified by the user described in Step 330).

[0056] The read cache control module determines whether the read requests received within the historical time meet the preset temporal locality policy. If they meet, execute Step 350; if not, end this processing.

[0057] Furthermore, in one implementation, the specific process for the read cache control module to determine whether the read requests received within the historical time meet the temporal locality policy is as follows:

[0058] The read cache control module determines whether the number of read requests received within the historical time is not less than the preset number threshold; if not less, it is determined that the read requests received within the historical time meet the event locality policy; if less, it is determined that the read requests received within the historical time do not meet the temporal locality policy.

[0059] Even further, the preset number threshold can be specifically 5. That is, if the number of read requests to access the same directory received within the past 1 second is not less than 5, the read cache control module determines that the read requests received within the historical time meet the temporal locality policy.

[0060] In the embodiments of the present application, the directory can be implemented by a directory tree. For example, a B-tree or a B+-tree. Each read request's reception time is marked at the directory to indicate that the directory has been accessed by the read request. The read cache control module can use the time sliding window method to obtain the number of read requests within the historical time.

[0061] In another implementation, the specific process for the read cache control module to determine whether the read requests received within the historical time meet the temporal locality policy is as follows:

[0062] The read cache control module determines whether all the read requests received within the historical time are sent by the same user terminal; if so, it is determined that the read requests received within the historical time meet the event locality policy; if not, it is determined that the read requests received within the historical time do not meet the temporal locality policy.

[0063] Furthermore, in the embodiments of the present application, the user terminal information for sending each read request is marked at the directory. The user terminal information may specifically be the IP address of the terminal or the user domain. The read cache control module may adopt the time sliding window method to obtain read requests within the historical time, and determine whether the obtained read requests are sent by the same user terminal based on the user terminal information.

[0064] Step 350: If satisfied, according to the preset read cache policy, read the file to be read and the associated files of the file to be read from the storage medium, and store the file to be read and the associated files in the cache.

[0065] Specifically, according to the judgment in step 340, if the read requests received within the historical time satisfy the temporal locality policy, the read cache control module reads the file to be read and the associated files of the file to be read from the storage medium according to the preset read cache policy, and stores the file to be read and the associated files in the cache.

[0066] Furthermore, the specific process of the read cache control module reading the file to be read and the associated files of the file to be read from the storage medium according to the preset read cache policy and storing the file to be read and the associated files in the cache is as follows:

[0067] The read cache control module calls a callback function. Based on the callback function, the file to be read and the associated files are read from the storage medium. The read cache control module determines whether the total file size of all the read files, including the file to be read and the associated files, is not greater than the file size threshold (for example, 500 KB). If not greater, the read cache control module stores all the read files in the cache; if greater, this processing ends.

[0068] Among them, the associated files may specifically be files in the same directory as the file to be read. For example, as Figure 2 shown, if the file to be read is File1 under subdirectory 13, the related file can be any one of File2 - File99 under subdirectory 12.

[0069] The read cache policy can be specifically configured to read all files or partial files in the directory and subdirectories. For example, if the file to be read is File1 under subdirectory 13, according to the read cache policy, File1 and File2 - File50 can be read; or, File1 and File2 - File99, that is, subdirectory 13, can be read.

[0070] The above read cache control module determines whether the total file size of all the files read is not greater than the file size threshold. Specifically: the read cache control module determines whether the file sizes of "File1 and File2 - File50" are not greater than 500KB. If not greater, the read cache control module stores "File1 and File2 - File50" in the cache; or, the read cache control module determines whether the file size of "subdirectory 13" is not greater than 500KB. If not greater, the read cache control module stores "subdirectory 13" in the cache.

[0071] Therefore, by applying the file reading method provided in this application, the read cache control module receives a read request sent by the MDS node or the DS node. The read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located; the read cache control module determines whether the file size meets the preset file attribute policy; if it meets, the read cache control module determines whether the directory information meets the preset spatial locality policy; if it meets, the read cache control module obtains the read requests received within the historical time and determines whether the read requests received within the historical time meet the preset temporal locality policy; if it meets, according to the preset read cache policy, the read cache control module reads the file to be read and the associated files of the file to be read from the storage medium, and stores the file to be read and the associated files in the cache.

[0072] In this way, it solves the problem in the existing file reading method that the file metadata information and file data of small files cannot be prefetched, and the reading efficiency cannot be improved. When a single thread is used to centrally read small files in a certain directory or some directories, based on the preset cache policy, the reading efficiency can be greatly improved. In a hybrid flash configuration (a combination of HDD and SSD), the number of small files read per second can be increased by 2 - 3 times.

[0073] Optionally, in the embodiment of this application, the read cache control module also marks the reception time of the read request at the directory where the file to be read is located; or, the read cache control module marks the address information and domain information included in the read request at the directory where the file to be read is located.

[0074] In this way, when the read cache control module executes step 340, it obtains the reception time of each read request or the user terminal information at the directory.

[0075] The following provides a detailed description of the read cache control module provided in the embodiment of this application. Refer to Figure 5 , Figure 5 which is the internal structure diagram of the read cache control module provided in the embodiment of this application.

[0076] As Figure 5As shown, the read cache control module includes a read request classification sub-module, a policy control sub-module, and a read cache processing sub-module. The basic functions of each sub-module are described as follows.

[0077] Read request classification sub-module: Labels and classifies the read requests sent by the MDS node or the DS node according to the policies defined by the "policy control sub-module". When the read requests meet various control policies, it calls the callback functions registered by the "read cache processing sub-module" to process the files to be read.

[0078] Policy control sub-module: Defines various control policies, and determines that the read requests meet the policies only when the read requests meet all control policies at the same time. Among them, various control policies are described as follows.

[0079] File attribute policy: Describes information such as the file size (for example, sets the file size threshold to 500KB);

[0080] Spatial locality policy: Describes information such as the directories and subdirectories specified by the user;

[0081] Temporal locality policy: Defines the historical time length (for example, the number of read requests accessing the same directory received per second, and whether the read requests accessing the same directory received per second are sent by the same user terminal);

[0082] Read cache processing sub-module: Defines the read cache policy (for example, reads the data or metadata in all files or some files under the directory and subdirectories and stores them in the cache; when the size of all files or some files exceeds the file size threshold set in the file attribute policy, no cache processing is performed). The read cache processing sub-module also implements the cache function by registering callback functions to the read request classification sub-module.

[0083] It can be understood that the above-mentioned steps in the foregoing embodiments are jointly completed by the read request classification sub-module, the policy control sub-module, and the read cache processing sub-module.

[0084] Based on the same inventive concept, the embodiments of the present application also provide a file reading device corresponding to the file reading method. See Figure 6 , Figure 6 This is the structural diagram of the file reading device provided by the embodiments of the present application. The device is applied to the read cache control module, the read cache control module is in the file storage system, and the file storage system also includes an MDS node and a DS node. The device includes:

[0085] A receiving unit 610, configured to receive the read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located;

[0086] The first judgment unit 620 is configured to judge whether the file size meets a preset file attribute policy;

[0087] The second judgment unit 630 is configured to, if it is satisfied, judge whether the directory information meets a preset spatial locality policy;

[0088] The third judgment unit 640 is configured to, if it is satisfied, obtain read requests received within a historical time, and judge whether the read requests received within the historical time meet a preset temporal locality policy;

[0089] The read cache unit 650 is configured to, if it is satisfied, read the file to be read and the associated file of the file to be read from a storage medium according to a preset read cache policy, and store the file to be read and the associated file in a cache.

[0090] Optionally, the first judgment unit 620 is specifically configured to judge whether the file size is not greater than a preset file size threshold;

[0091] If it is not greater than, it is determined that the file size meets the file attribute policy

[0092] Optionally, the second judgment unit 630 is specifically configured to judge whether the directory information is a directory specified by a user;

[0093] If so, it is determined that the directory information meets the spatial locality policy.

[0094] Optionally, the third judgment unit 640 is specifically configured to judge whether the number of read requests received within the historical time is not less than a preset number threshold;

[0095] If it is not less than, it is determined that the read requests received within the historical time meet the temporal locality policy.

[0096] Optionally, the third judgment unit 640 is specifically configured to judge whether the read requests received within the historical time are all sent by the same user terminal;

[0097] If so, it is determined that the read requests received within the historical time meet the temporal locality policy.

[0098] Optionally, the device further includes:

[0099] A marking unit (not shown in the figure) is configured to mark the time when the read request is received at the directory where the file to be read is located;

[0100] Alternatively, mark the address information and domain information included in the read request at the directory where the file to be read is located.

[0101] Optionally, the read cache unit 650 is specifically configured to call a callback function;

[0102] Based on the callback function, read the file to be read and the associated file from the storage medium;

[0103] Determine whether the total file size of all the read files is not greater than the file size threshold, where the all files include the file to be read and the associated file;

[0104] If not greater than, store all the read files in the cache.

[0105] Therefore, by applying the file reading device provided in this application, the read cache control module receives a read request sent by an MDS node or a DS node, and the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located; the read cache control module determines whether the file size meets the preset file attribute policy; if it meets, the read cache control module determines whether the directory information meets the preset spatial locality policy; if it meets, the read cache control module obtains the read requests received within the historical time and determines whether the read requests received within the historical time meet the preset temporal locality policy; if it meets, according to the preset read cache policy, the read cache control module reads the file to be read and the associated file of the file to be read from the storage medium, and stores the file to be read and the associated file in the cache.

[0106] In this way, the problem that in the existing file reading method, the file metadata information and file data of small files cannot be prefetched and the reading efficiency cannot be improved is solved. When a single thread is used to centrally read small files in a certain directory or some directories, based on the preset cache policy, the reading efficiency can be greatly improved. In a hybrid flash configuration (a combination of HDD and SSD), the number of small files read per second can be increased by 2-3 times.

[0107] Based on the same inventive concept, an embodiment of this application also provides a network device, as Figure 7 shown, including a processor 710, a transceiver 720, and a machine-readable storage medium 730. The machine-readable storage medium 730 stores machine-executable instructions that can be executed by the processor 710. The processor 710 is prompted by the machine-executable instructions to execute the file reading method provided in the embodiment of this application. The foregoing Figure 6 shown communication device can be implemented by using the hardware structure of the network device as Figure 7 shown.

[0108] The above computer-readable storage medium 730 may include a Random Access Memory (RAM), or may also include a non-volatile memory (NVM), such as at least one disk memory. Optionally, the computer-readable storage medium 730 may also be at least one storage device located far from the aforementioned processor 710.

[0109] The above processor 710 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0110] In the embodiment of the present application, the processor 710 is caused by the machine-executable instructions read from the machine-readable storage medium 730 to be capable of implementing itself and invoking the transceiver 720 to execute the file reading method described in the foregoing embodiment of the present application.

[0111] In addition, the embodiment of the present application provides a machine-readable storage medium 730, and the machine-readable storage medium 730 stores machine-executable instructions. When being called and executed by the processor 710, the machine-executable instructions cause the processor 710 itself and the transceiver 720 to be invoked to execute the file reading method described in the foregoing embodiment of the present application.

[0112] For the specific implementation process of the functions and roles of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and will not be elaborated here.

[0113] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0114] For the embodiments of the file reading device and the machine-readable storage medium, since the method content involved is basically similar to the foregoing method embodiments, the description is relatively simple, and the relevant parts can be referred to the descriptions of the method embodiments.

[0115] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A file reading method, characterized in that, The method is applied to a read cache control module, which is within a file storage system. The file storage system further includes an MDS node and a DS node. The method includes: Receiving a read request sent by the MDS node or the DS node, where the read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located; Judging whether the file size meets a preset file attribute policy; If the file size meets the file attribute policy, then judging whether the directory information meets a preset spatial locality policy; If the directory information meets the spatial locality policy, then obtaining the read requests received within a historical time and judging whether the read requests received within the historical time meet a preset temporal locality policy; If the read request meets the temporal locality policy, then according to a preset read cache policy, reading the file to be read and the associated files of the file to be read from a storage medium and storing the file to be read and the associated files in a cache; Wherein, the spatial locality policy is to judge whether the directory information is a directory specified by a user, and the temporal locality policy is to judge whether the number of read requests is not less than a preset number threshold, or the temporal locality policy is to judge whether the read requests are all sent by the same user terminal.

2. The method according to claim 1, characterized in that, The judging whether the file size meets a preset file attribute policy specifically includes: Judging whether the file size is not greater than a preset file size threshold; If it is not greater, then determining that the file size meets the file attribute policy.

3. The method according to claim 1, characterized in that, The judging whether the directory information meets a preset spatial locality policy specifically includes: Judging whether the directory information is a directory specified by a user; If so, then determining that the directory information meets the spatial locality policy.

4. The method according to claim 1, characterized in that, The judging whether the read requests received within the historical time meet a preset temporal locality policy specifically includes: Judging whether the number of read requests received within the historical time is not less than a preset number threshold; If it is not less, then determining that the read requests received within the historical time meet the temporal locality policy.

5. The method according to claim 1, wherein The judging whether the read requests received within the historical time meet a preset temporal locality policy specifically includes: Judging whether the read requests received within the historical time are all sent by the same user terminal; If so, then determining that the read requests received within the historical time meet the temporal locality policy.

6. The method according to any one of claims 4 or 5, characterized in that, The method further includes: Marking the time when the read request is received at the directory where the file to be read is located; Or, marking the address information and domain information included in the read request at the directory where the file to be read is located.

7. The method according to claim 2, wherein The reading the file to be read and the associated files of the file to be read from a storage medium and storing the file to be read and the associated files in a cache according to a preset read cache policy specifically includes: Invoking a callback function; Based on the callback function, reading the file to be read and the associated files from the storage medium; Determine whether the total file size of all the read files is not greater than the file size threshold, where the all files include the file to be read and the associated file; If it is not greater than, store all the read files in the cache.

8. A file reading device, characterized in that, The device is applied to a read cache control module, and the read cache control module is within a file storage system. The file storage system further includes an MDS node and a DS node. The device includes: A receiving unit, configured to receive a read request sent by the MDS node or the DS node. The read request includes the file size of the file to be read and the directory information of the directory where the file to be read is located; A first determination unit, configured to determine whether the file size meets a preset file attribute policy; A second determination unit, configured to, if the file size meets the file attribute policy, determine whether the directory information meets a preset spatial locality policy; A third determination unit, configured to, if the directory information meets the spatial locality policy, obtain the read requests received within a historical time and determine whether the read requests received within the historical time meet a preset temporal locality policy; A read cache unit, configured to, if the read request meets the temporal locality policy, read the file to be read and the associated file of the file to be read from a storage medium according to a preset read cache policy, and store the file to be read and the associated file in the cache; Wherein, the spatial locality policy is to determine whether the directory information is a directory specified by a user, and the temporal locality policy is to determine whether the number of read requests is not less than a preset number threshold, or the temporal locality policy is to determine whether the read requests are all sent by the same user terminal.

9. The device according to claim 8, characterized in that, The first determination unit is specifically configured to determine whether the file size is not greater than a preset file size threshold; If it is not greater than, determine that the file size meets the file attribute policy.

10. The device according to claim 8, characterized in that, The second determination unit is specifically configured to determine whether the directory information is a directory specified by a user; If so, determine that the directory information meets the spatial locality policy.

11. The device according to claim 8, characterized in that, The third determination unit is specifically configured to determine whether the number of read requests received within the historical time is not less than a preset number threshold; If it is not less than, determine that the read requests received within the historical time meet the temporal locality policy.

12. The device according to claim 8, characterized in that, The third determination unit is specifically configured to determine whether the read requests received within the historical time are all sent by the same user terminal; If so, determine that the read requests received within the historical time meet the temporal locality policy.

13. The device according to any one of claims 11 or 12, characterized in that, The device further includes: A marking unit, configured to mark the time when the read request is received at the directory where the file to be read is located; Or mark the address information and domain information included in the read request at the directory where the file to be read is located.

14. The device according to claim 9, wherein The read cache unit is specifically configured to call a callback function; Based on the callback function, read the file to be read and the associated file from the storage medium; Determine whether the total file size of all the read files is not greater than the file size threshold, where the all files include the file to be read and the associated file; If it is not greater than, store all the read files in the cache.

Images