File access method and device, computer equipment, storage medium and program product
By dynamically selecting virtual or non-virtual directory access strategies, the performance bottleneck in directory operations is resolved, achieving efficient file access.
Patent Information
- Application Number
- CN202511779937.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-03-03
AI Technical Summary
Existing technologies have performance bottlenecks in directory-based operation scenarios, especially when using the `ls` command to access directories, which takes longer. Virtual directory solutions increase the number of operations, leading to performance degradation.
Based on the complex parameters of the target real directory structure, the access strategy is dynamically selected. Simple directories are accessed directly, while complex directories are accessed through virtual directory caching mapping relationships to avoid repeated path resolution.
It improves the access efficiency of the file system, ensuring high performance at both the directory level and the file level, thus meeting the performance requirements of different scenarios.
Smart Images
Figure CN121597642A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a file access method, apparatus, computer equipment, storage medium, and program product. Background Technology
[0002] Compared to block storage, file storage requires the preservation of metadata information. This metadata management is typically handled by a Meta Data Storage (MDS) program, making MDS a key factor affecting file storage performance. Generally, a directory is bound to a single MDS process. When reading a large number of small files within a directory, the MDS experiences a performance bottleneck, resulting in poor random read performance for small files. To address this issue, a virtual directory approach is introduced. This involves implementing the following functionality in file storage: when a client needs to randomly read a large number of small files within a single directory, this request is transformed into multiple virtual directories. These virtual directories are then distributed across multiple MDS processes, allowing for concurrent reading of small files within a single directory, thereby improving read performance for large numbers of small files in a single directory.
[0003] The current solution has performance issues in directory-related operation scenarios, such as ls (list), which involves multiple opendir / readdir operations. The virtual directory solution will result in more directory operations, which will increase the time taken by ls. Summary of the Invention
[0004] Therefore, it is necessary to provide a file access method, apparatus, computer device, storage medium, and program product that can quickly and efficiently access files in the file system, addressing the aforementioned technical problems.
[0005] Firstly, this application provides a file access method, including:
[0006] Upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined; wherein, the structural complexity parameter characterizes the directory structure complexity of the target real directory;
[0007] Based on the structural complexity parameters, the access strategy for the file access operation corresponding to the file access request is determined; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory;
[0008] The file access operation is performed according to the access policy.
[0009] In one embodiment, determining the access strategy for the file access operation based on the structural complexity parameters includes:
[0010] When the structural complexity parameter is the first complexity parameter, the access strategy of the file access operation is determined according to the operation type of the file access operation corresponding to the file access request;
[0011] When the structural complexity parameter is the second complexity parameter, the access strategy for the file access operation is determined to be a virtual directory access strategy.
[0012] When the structural complexity parameter is the third complexity parameter, the access strategy for the file access operation is determined to be a non-virtual directory access strategy; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; and the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
[0013] In one embodiment, when there are multiple file access operations, determining the access strategy for the file access operation based on the operation type of the file access operation corresponding to the file access request includes:
[0014] Determine the first number of file access operations whose operation type is file-level operation, and determine the second number of file access operations whose operation type is directory-level operation;
[0015] If the first number of operations is greater than or equal to the second number of operations, the access strategy for the file access operation is determined to be a virtual directory access strategy.
[0016] If the number of the first operation is less than the number of the second operation, the access strategy for the file access operation is determined to be a non-virtual directory access strategy.
[0017] In one embodiment, when the access policy is a virtual directory access policy, performing the file access operation according to the access policy includes:
[0018] Based on directory relationship mapping information, candidate virtual directories corresponding to the target real directory are determined; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system;
[0019] Based on the file attribute information of the file to which the file access operation is applied, select the target virtual directory from the candidate virtual directories;
[0020] The metadata storage (MDS) process corresponding to the target virtual directory is invoked to perform the file access operation.
[0021] In one embodiment, selecting a target virtual directory from the candidate virtual directories based on the file attribute information of the file to which the file access operation is performed includes:
[0022] The file hash value is determined based on the file attribute information of the file to which the file access operation is performed;
[0023] Select the target virtual directory from the candidate virtual directories based on the file hash value.
[0024] In one embodiment, when the access policy is a non-virtual directory access policy, performing the file access operation according to the access policy includes:
[0025] The MDS process corresponding to the real directory is invoked to perform the file access operation.
[0026] Secondly, this application also provides a file access device, comprising:
[0027] The parameter determination module is used to determine the structural complexity parameters of the target real directory when a file access request for the target real directory in the file system is received; wherein, the structural complexity parameters characterize the directory structure complexity of the target real directory;
[0028] The strategy determination module is used to determine the access strategy of the file access operation corresponding to the file access request based on the structural complexity parameters; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory;
[0029] The file access module is used to perform the file access operation according to the access policy.
[0030] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0031] Upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined; wherein, the structural complexity parameter characterizes the directory structure complexity of the target real directory;
[0032] Based on the structural complexity parameters, the access strategy for the file access operation corresponding to the file access request is determined; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory;
[0033] The file access operation is performed according to the access policy.
[0034] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0035] Upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined; wherein, the structural complexity parameter characterizes the directory structure complexity of the target real directory;
[0036] Based on the structural complexity parameters, the access strategy for the file access operation corresponding to the file access request is determined; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory;
[0037] The file access operation is performed according to the access policy.
[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0039] Upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined; wherein, the structural complexity parameter characterizes the directory structure complexity of the target real directory;
[0040] Based on the structural complexity parameters, the access strategy for the file access operation corresponding to the file access request is determined; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory;
[0041] The file access operation is performed according to the access policy.
[0042] The aforementioned file access method, apparatus, computer device, storage medium, and program product, upon receiving a file access request for a target real directory in a file system, determine the structural complexity parameters of the target real directory; wherein, the structural complexity parameters characterize the complexity of the directory structure of the target real directory; based on the structural complexity parameters, determine the access strategy for the file access operation corresponding to the file access request; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; a virtual directory access strategy is a strategy to access the target real directory through a target virtual directory corresponding to the target real directory; a non-virtual directory access strategy is a strategy to directly access the target real directory; and the file access operation is executed according to the access strategy. This scheme dynamically selects a strategy based on directory complexity. Simple directories are accessed directly to reduce redundant overhead, while complex directories avoid repeated parsing of complex paths by caching mapping relationships through virtual directories. This ensures high performance at both the directory level and file level, meeting the performance requirements of different scenarios, satisfying the effective use of applications, and improving access efficiency. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0044] Figure 1 This is a diagram illustrating the application environment of a file access method in one embodiment;
[0045] Figure 2 This is a flowchart illustrating a file access method in one embodiment;
[0046] Figure 3 This is a schematic diagram of a virtual directory structure in one embodiment;
[0047] Figure 4 This is a flowchart illustrating the process of determining the access strategy for a file access operation in one embodiment.
[0048] Figure 5 This is a flowchart illustrating the execution of file access operations in one embodiment;
[0049] Figure 6 This is a schematic diagram of directory relationship mapping information in one embodiment;
[0050] Figure 7 This is a flowchart illustrating a file access method in another embodiment;
[0051] Figure 8This is a structural block diagram of a file access device in one embodiment;
[0052] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0054] The file access method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the file system 101 is the core component responsible for managing and storing file data. In this embodiment, the file system 101 is the system being accessed; the client 102 is the application terminal facing the user. The file access method provided in this embodiment can be executed by the backend server of the file system 101.
[0055] In one exemplary embodiment, such as Figure 2 As shown, a file access method is provided, which can be applied to... Figure 1 Taking the backend server of file system 101 as an example, the following steps are included:
[0056] S201, upon receiving a file access request for a target real directory in the file system, determine the structural complexity parameters of the target real directory.
[0057] In this context, the real directory is the entity directory directly created and managed within the file system; the target real directory is the directory accessed by the file access request. The structural complexity parameter characterizes the complexity of the target real directory's directory structure. This parameter is dynamically calculated based on the directory hierarchy of the target real directory.
[0058] Optionally, the complex structural parameters of the real directory are pre-stored in the file system and can be associated with the directory identifier of the real directory. Therefore, the complex structural parameters of the target real directory can be retrieved from the file system based on the directory identifier of the target real directory.
[0059] S202, Based on the structural complexity parameters, determine the access strategy for the file access operation corresponding to the file access request.
[0060] The access strategy can be either a virtual directory access strategy or a non-virtual directory access strategy. The virtual directory access strategy is a strategy that accesses the target real directory through the target virtual directory corresponding to the target real directory. The non-virtual directory access strategy is a strategy that directly accesses the target real directory.
[0061] It should be noted that, in this embodiment, when a client needs to randomly read a large number of small files in a single directory, this request is converted into multiple virtual directories. These virtual directories can then be distributed across multiple MDS processes, allowing for concurrent reading of small files within a single directory, thereby improving the reading performance of a large number of small files in a single directory. For example, reading a large number of small files in the directory / mnt / testdir1 is performed by splitting them into four virtual directories: / mnt / testdir1-p1, / mnt / testdir1-p2, / mnt / testdir1-p3, and / mnt / testdir1-p4, and then reading the corresponding files in each of these four virtual directories in parallel. Figure 3 The diagram shown is a schematic of the virtual directory structure provided in an embodiment of this application.
[0062] Optionally, the complexity of the hierarchical structure of the real directory can be divided into multiple levels based on structural complexity parameters, and different access strategies can be assigned to different levels. For example, the structural complexity parameters may include a first complexity parameter, a second complexity parameter, and a third complexity parameter, where the directory structure complexity represented by the first complexity parameter is lower than that represented by the second complexity parameter; and the directory structure complexity represented by the second complexity parameter is lower than that represented by the third complexity parameter. The method for determining the access strategy for file access operations corresponding to file access requests for different complexity scenarios is as follows.
[0063] Optionally, when the structural complexity parameter is the first complexity parameter, the access strategy for the file access operation is determined based on the operation type of the file access operation corresponding to the file access request. If the structural complexity parameter is the first complexity parameter, it indicates that the target real directory has low complexity; for example, the target real directory may only include the parent directory, or only the parent directory and its subdirectories. In this case, the hierarchical structure of the target real directory is simple, and an access strategy can be assigned to the file access operation based on the operation type of the file access operation corresponding to the file access request. In this embodiment, the operation types include, but are not limited to, directory-level operations and file-level operations. Directory-level operations are operations on the directory itself or on the directory structure; file-level operations are operations on the file entity, directly affecting the file's content or its metadata. Directory-level operations include ls, directory data query, directory creation, directory deletion, and directory movement; file-level operations include file reading, file writing, and file metadata operations. Different operation types have different dependencies on the directory structure and performance requirements, necessitating matching with corresponding access strategies.
[0064] Specifically, the specific operation type can be identified through the API interface, command parameters, or operation representation of the file access request. Furthermore, for directory-level operations, the access strategy of the file access operation can be directly determined to be a non-virtual directory access strategy; for file-level operations, the access strategy of the file access operation can be directly determined to be a virtual directory access strategy.
[0065] Optionally, when the structural complexity parameter is the second complexity parameter, the access strategy for file access operations is determined to be a virtual directory access strategy. For example, if the parent directory contains many subdirectories, but a few of these subdirectories contain complex sub-subdirectories, and most of the subdirectories are already last-level directories, then the structural complexity parameter of the target real directory can be determined to be the second complexity parameter. In this case, the access strategy for file access operations can be directly determined to be a virtual directory access strategy.
[0066] Optionally, when the structural complexity parameter is the third complexity parameter, the access strategy for file access operations is determined to be a non-virtual directory access strategy. For example, if the parent directory contains many subdirectories, most of which contain complex sub-subdirectories, and only a few subdirectories are the last-level directories, then the structural complexity parameter of the target real directory can be determined to be the third complexity parameter. In this case, the access strategy for file access operations can be directly determined to be a non-virtual directory access strategy.
[0067] Understandably, the access policy decision-making mechanism, which features complex parameter hierarchy and dynamic adaptation to operation type, solves the problem of the security and efficiency imbalance of the traditional one-size-fits-all strategy (fixed virtual directory / direct access).
[0068] S203, Perform file access operations according to the access policy.
[0069] Optionally, file access operations can be performed according to a fixed access flow based on the access policy.
[0070] In the above file access method, upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined. This structural complexity parameter characterizes the complexity of the target real directory's directory structure. Based on the structural complexity parameter, an access strategy for the file access operation corresponding to the file access request is determined. This access strategy can be a virtual directory access strategy or a non-virtual directory access strategy. The virtual directory access strategy accesses the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy directly accesses the target real directory. Based on the access strategy, the file access operation is executed. This scheme dynamically selects a strategy based on directory complexity. Simple directories are accessed directly to reduce redundant overhead, while complex directories use virtual directory caching mapping relationships to avoid repeatedly parsing complex paths. This ensures high performance at both the directory and file levels, meeting the performance requirements of different scenarios, satisfying the effective use of applications, and improving access efficiency.
[0071] Optionally, in an exemplary embodiment, such as Figure 4 As shown, when there are multiple file access operations, a method for determining the access strategy for file access operations is provided, which specifically includes the following steps:
[0072] S401, determine the first number of file access operations of type file level and the second number of file access operations of type directory level.
[0073] Optionally, when dealing with a large number of operations on the same directory, it is necessary to determine the access strategy for file access operations based on the access type of all file access operations. Therefore, it is first necessary to determine the number of operations for each file access operation. Specifically, the operation object and behavior of each file access operation can be parsed, and based on the parsing results, the first number of operations for file access operations of the file-level operation type and the second number of operations for file access operations of the directory-level operation type can be determined.
[0074] S402, if the number of first operations is greater than or equal to the number of second operations, determine that the access strategy for the file access operation is the virtual directory access strategy.
[0075] Optionally, if the number of first operations is greater than or equal to the number of second operations, that is, if most file access operations are file-level operations, the access strategy for file access operations can be directly determined as a virtual directory access strategy.
[0076] S403, if the number of first operations is less than the number of second operations, determine that the access strategy for the file access operation is a non-virtual directory access strategy.
[0077] Optionally, if the number of first operations is less than the number of second operations, that is, if most file access operations are directory-level operations, the access strategy for file access operations can be directly determined to be a non-virtual directory access strategy.
[0078] In this embodiment, access strategies are dynamically selected based on actual operation behavior data to avoid resource waste or low access efficiency caused by a one-size-fits-all configuration. Virtual directory access strategies can reduce storage redundancy through file index aggregation management, while non-virtual directory strategies can reduce the performance overhead caused by directory nesting and realize on-demand resource allocation. In file operation-intensive scenarios, virtual directory strategies support fast file location and batch operations; in directory operation-intensive scenarios, non-virtual directory strategies simplify directory structure management and reduce user operation complexity.
[0079] Optionally, in one embodiment, such as Figure 5 As shown, when the access policy is a virtual directory access policy, a method for executing file access operations is provided, specifically including the following steps:
[0080] S501, Based on the directory relationship mapping information, determine the candidate virtual directory corresponding to the target real directory.
[0081] The directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system. This directory relationship mapping information can be presented in tabular form, such as... Figure 6 The diagram shown is a schematic representation of directory relationship mapping information provided in an embodiment of this application.
[0082] Optionally, the candidate virtual directory corresponding to the target real directory can be found directly from the directory relationship mapping information based on the directory identifier of the target real directory.
[0083] S502, select the target virtual directory from the candidate virtual directories based on the file attribute information of the file to which the file access operation is applied.
[0084] Among them, file attribute information refers to the inherent or dynamic attributes of a file, including but not limited to file size, file type, file name, and file identifier.
[0085] It should be noted that since a real directory is decomposed into multiple virtual directories, and a virtual directory contains multiple files, when constructing directory relationship mapping information, the matching conditions between virtual directories and file attributes can be preset. During the filtering process, the candidate virtual directory set is verified one by one, and the only virtual directory that meets the conditions is selected as the target virtual directory.
[0086] In one embodiment, the name of the virtual directory can be generated using a hash algorithm. Specifically, the name of the virtual directory is generated based on the file attribute information of the files contained within the virtual directory. In this case, the file hash value can be determined based on the file attribute information of the files affected by the file access operation; the target virtual directory is then selected from the candidate virtual directories based on the file hash value. Specifically, the file attribute dimensions involved in the hash calculation need to be clearly defined, including but not limited to file name and file identifier. A standardized hash algorithm is used to perform a digest calculation on the selected file attribute information, generating a fixed-length hash value as the file hash value. Furthermore, a pre-defined binding relationship between the hash value and the virtual directory is established, and the target directory is obtained by matching candidate virtual directories based on the characteristics of the hash value. It is understood that a hash value generated based on the core file attributes can uniquely identify a file, avoiding virtual directory matching errors caused by path changes or renaming.
[0087] S503 invokes the metadata storage (MDS) process corresponding to the target virtual directory to perform file access operations.
[0088] The MDS process is responsible for managing the mapping metadata between virtual directories and real directories, file attribute indexes, and access permissions. It is the core execution carrier of virtual directory access policies. Each virtual directory can be bound to an independent MDS process (or an instance in a shared process pool) to achieve metadata isolation and parallel processing.
[0089] Optionally, obtain the MDS process identifier pre-bound to the target virtual directory, send a file access operation instruction to the MDS process, carrying the file path, operation type (read / write / delete, etc.) and necessary attribute information; the MDS process converts the virtual directory path into a real directory path based on the directory relationship mapping information, performs the underlying file operation, and returns the result (such as operation success flag, file data).
[0090] In this embodiment, users access files through a virtual directory without needing to be aware of the actual storage path. It is compatible with complex architectures such as distributed storage and cloud storage. Based on the binding relationship between file attributes and virtual directories, it enables more granular access permission management.
[0091] In one embodiment, when the access policy is a non-virtual directory access policy, the MDS process corresponding to the real directory can be directly invoked to perform file access operations.
[0092] Figure 7 This is a flowchart illustrating a file access method in another embodiment. Based on the above embodiments, this embodiment provides an optional example of a file access method. (Combined with...) Figure 7 The specific implementation process is as follows:
[0093] S701, upon receiving a file access request for a target real directory in the file system, determines the structural complexity parameters of the target real directory.
[0094] Among them, the structural complexity parameter characterizes the complexity of the directory structure of the target real directory.
[0095] S702, if the structural complexity parameter is the first complex parameter, execute S703; if the structural complexity parameter is the second complex parameter, execute S706; if the structural complexity parameter is the third complex parameter, execute S705.
[0096] S703, determine the first number of file access operations whose operation type is file-level operation, and determine the second number of file access operations whose operation type is directory-level operation.
[0097] S704, determine whether the first operation quantity is less than the second operation quantity; if yes, then execute S705; if no, then execute S706.
[0098] S705, determine that the access policy for file access operations is a non-virtual directory access policy, and execute S710.
[0099] S706, determine that the access policy for file access operations is the virtual directory access policy, and execute S707.
[0100] S707, Based on the directory relationship mapping information, determine the candidate virtual directory corresponding to the target real directory.
[0101] The directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system.
[0102] S708: Select the target virtual directory from the candidate virtual directories based on the file attribute information of the file to which the file access operation is performed.
[0103] Optionally, the file hash value can be determined based on the file attribute information of the file to which the file access operation is performed; and the target virtual directory can be selected from the candidate virtual directories based on the file hash value.
[0104] S709 invokes the MDS process corresponding to the target virtual directory to perform file access operations.
[0105] S710 calls the MDS process corresponding to the real directory to perform file access operations.
[0106] The specific processes of S701-S710 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.
[0107] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0108] Based on the same inventive concept, this application also provides a file access apparatus for implementing the file access method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more file access apparatus embodiments provided below can be found in the limitations of the file access method described above, and will not be repeated here.
[0109] In one exemplary embodiment, such as Figure 8 As shown, a file access device 800 is provided, including: a parameter determination module 810, a policy determination module 820, and a file access module 830, wherein:
[0110] The parameter determination module 810 is used to determine the structural complexity parameters of the target real directory when a file access request for the target real directory in the file system is received; wherein, the structural complexity parameters characterize the complexity of the directory structure of the target real directory.
[0111] The strategy determination module 820 is used to determine the access strategy of the file access operation corresponding to the file access request based on the structurally complex parameters; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory.
[0112] The file access module 830 is used to perform file access operations according to the access policy.
[0113] The aforementioned file access device, upon receiving a file access request for a target real directory in the file system, determines the structural complexity parameter of the target real directory; whereby the structural complexity parameter characterizes the complexity of the directory structure of the target real directory; based on the structural complexity parameter, it determines the access strategy for the file access operation corresponding to the file access request; wherein the access strategy is either a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory; and the file access operation is executed according to the access strategy. This scheme dynamically selects a strategy based on directory complexity. Simple directories are accessed directly to reduce redundant overhead, while complex directories avoid repeated parsing of complex paths by caching mapping relationships through virtual directories. This ensures high performance at both the directory level and file level, meeting the performance requirements of different scenarios, satisfying the effective use of applications, and improving access efficiency.
[0114] In one embodiment, the policy determination module 820 includes:
[0115] The first determining unit is used to determine the access strategy of the file access operation based on the operation type of the file access operation corresponding to the file access request, when the structural complexity parameter is the first complexity parameter.
[0116] The second determining unit is used to determine the access strategy of the file access operation as a virtual directory access strategy when the structural complexity parameter is the second complex parameter.
[0117] The third determining unit is used to determine that the access strategy for file access operations is a non-virtual directory access strategy when the structural complexity parameter is the third complexity parameter; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; and the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
[0118] In one embodiment, when there are multiple file access operations, the first determining unit is specifically used for:
[0119] Determine the first number of file access operations that are file-level operations and the second number of file access operations that are directory-level operations; if the first number of operations is greater than or equal to the second number of operations, determine that the access strategy for the file access operation is a virtual directory access strategy; if the first number of operations is less than the second number of operations, determine that the access strategy for the file access operation is a non-virtual directory access strategy.
[0120] In one embodiment, when the access policy is a virtual directory access policy, the file access module 830 includes:
[0121] The directory determination unit is used to determine the candidate virtual directories corresponding to the target real directory based on the directory relationship mapping information; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system.
[0122] The directory selection unit is used to select the target virtual directory from the candidate virtual directories based on the file attribute information of the file to which the file access operation is applied.
[0123] The file access unit is used to invoke the metadata storage (MDS) process corresponding to the target virtual directory to perform file access operations.
[0124] In one embodiment, the directory selection unit is specifically used for:
[0125] Determine the file hash value based on the file attribute information of the file to which the file access operation is performed; select the target virtual directory from the candidate virtual directories based on the file hash value.
[0126] In one embodiment, when the access policy is a non-virtual directory access policy, the file access module 830 is further configured to:
[0127] Invoke the MDS process corresponding to the real directory to perform file access operations.
[0128] Each module in the aforementioned file access device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0129] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a file access method.
[0130] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0131] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0132] Upon receiving a file access request for a target real directory in the file system, determine the structural complexity parameter of the target real directory; where the structural complexity parameter characterizes the complexity of the directory structure of the target real directory.
[0133] Based on the structural complexity parameters, determine the access strategy for the file access operation corresponding to the file access request; wherein, the access strategy is either a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy to access the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy to directly access the target real directory.
[0134] Perform file access operations according to the access policy.
[0135] In one embodiment, when the processor executes a computer program to determine the access strategy for file access operations based on structurally complex parameters, it also performs the following steps:
[0136] When the structural complexity parameter is the first complexity parameter, the access strategy for the file access operation is determined based on the operation type of the file access operation corresponding to the file access request; when the structural complexity parameter is the second complexity parameter, the access strategy for the file access operation is determined to be a virtual directory access strategy; when the structural complexity parameter is the third complexity parameter, the access strategy for the file access operation is determined to be a non-virtual directory access strategy; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
[0137] In one embodiment, when there are multiple file access operations, when the processor executes a computer program to determine the access strategy for a file access operation based on the operation type of the file access operation corresponding to the file access request, it also performs the following steps:
[0138] Determine the first number of file access operations that are file-level operations and the second number of file access operations that are directory-level operations; if the first number of operations is greater than or equal to the second number of operations, determine that the access strategy for the file access operation is a virtual directory access strategy; if the first number of operations is less than the second number of operations, determine that the access strategy for the file access operation is a non-virtual directory access strategy.
[0139] In one embodiment, when the access policy is a virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0140] Based on the directory relationship mapping information, candidate virtual directories corresponding to the target real directory are determined; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system; based on the file attribute information of the file to which the file access operation is applied, the target virtual directory is selected from the candidate virtual directories; the metadata storage MDS process corresponding to the target virtual directory is invoked to perform the file access operation.
[0141] In one embodiment, when the processor executes a computer program to select a target virtual directory from candidate virtual directories based on the file attribute information of the file to which the file access operation is performed, it also performs the following steps:
[0142] Determine the file hash value based on the file attribute information of the file to which the file access operation is performed; select the target virtual directory from the candidate virtual directories based on the file hash value.
[0143] In one embodiment, when the access policy is a non-virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0144] Invoke the MDS process corresponding to the real directory to perform file access operations.
[0145] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0146] Upon receiving a file access request for a target real directory in the file system, determine the structural complexity parameter of the target real directory; where the structural complexity parameter characterizes the complexity of the directory structure of the target real directory.
[0147] Based on the structural complexity parameters, determine the access strategy for the file access operation corresponding to the file access request; wherein, the access strategy is either a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy to access the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy to directly access the target real directory.
[0148] Perform file access operations according to the access policy.
[0149] In one embodiment, when the processor executes a computer program to determine the access strategy for file access operations based on structurally complex parameters, it also performs the following steps:
[0150] When the structural complexity parameter is the first complexity parameter, the access strategy for the file access operation is determined based on the operation type of the file access operation corresponding to the file access request; when the structural complexity parameter is the second complexity parameter, the access strategy for the file access operation is determined to be a virtual directory access strategy; when the structural complexity parameter is the third complexity parameter, the access strategy for the file access operation is determined to be a non-virtual directory access strategy; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
[0151] In one embodiment, when there are multiple file access operations, when the processor executes a computer program to determine the access strategy for a file access operation based on the operation type of the file access operation corresponding to the file access request, it also performs the following steps:
[0152] Determine the first number of file access operations that are file-level operations and the second number of file access operations that are directory-level operations; if the first number of operations is greater than or equal to the second number of operations, determine that the access strategy for the file access operation is a virtual directory access strategy; if the first number of operations is less than the second number of operations, determine that the access strategy for the file access operation is a non-virtual directory access strategy.
[0153] In one embodiment, when the access policy is a virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0154] Based on the directory relationship mapping information, candidate virtual directories corresponding to the target real directory are determined; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system; based on the file attribute information of the file to which the file access operation is applied, the target virtual directory is selected from the candidate virtual directories; the metadata storage MDS process corresponding to the target virtual directory is invoked to perform the file access operation.
[0155] In one embodiment, when the processor executes a computer program to select a target virtual directory from candidate virtual directories based on the file attribute information of the file to which the file access operation is performed, it also performs the following steps:
[0156] Determine the file hash value based on the file attribute information of the file to which the file access operation is performed; select the target virtual directory from the candidate virtual directories based on the file hash value.
[0157] In one embodiment, when the access policy is a non-virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0158] Invoke the MDS process corresponding to the real directory to perform file access operations.
[0159] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0160] Upon receiving a file access request for a target real directory in the file system, determine the structural complexity parameter of the target real directory; where the structural complexity parameter characterizes the complexity of the directory structure of the target real directory.
[0161] Based on the structural complexity parameters, determine the access strategy for the file access operation corresponding to the file access request; wherein, the access strategy is either a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy to access the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy to directly access the target real directory.
[0162] Perform file access operations according to the access policy.
[0163] In one embodiment, when the processor executes a computer program to determine the access strategy for file access operations based on structurally complex parameters, it also performs the following steps:
[0164] When the structural complexity parameter is the first complexity parameter, the access strategy for the file access operation is determined based on the operation type of the file access operation corresponding to the file access request; when the structural complexity parameter is the second complexity parameter, the access strategy for the file access operation is determined to be a virtual directory access strategy; when the structural complexity parameter is the third complexity parameter, the access strategy for the file access operation is determined to be a non-virtual directory access strategy; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
[0165] In one embodiment, when there are multiple file access operations, when the processor executes a computer program to determine the access strategy for a file access operation based on the operation type of the file access operation corresponding to the file access request, it also performs the following steps:
[0166] Determine the first number of file access operations that are file-level operations and the second number of file access operations that are directory-level operations; if the first number of operations is greater than or equal to the second number of operations, determine that the access strategy for the file access operation is a virtual directory access strategy; if the first number of operations is less than the second number of operations, determine that the access strategy for the file access operation is a non-virtual directory access strategy.
[0167] In one embodiment, when the access policy is a virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0168] Based on the directory relationship mapping information, candidate virtual directories corresponding to the target real directory are determined; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system; based on the file attribute information of the file to which the file access operation is applied, the target virtual directory is selected from the candidate virtual directories; the metadata storage MDS process corresponding to the target virtual directory is invoked to perform the file access operation.
[0169] In one embodiment, when the processor executes a computer program to select a target virtual directory from candidate virtual directories based on the file attribute information of the file to which the file access operation is performed, it also performs the following steps:
[0170] Determine the file hash value based on the file attribute information of the file to which the file access operation is performed; select the target virtual directory from the candidate virtual directories based on the file hash value.
[0171] In one embodiment, when the access policy is a non-virtual directory access policy, when the processor executes a computer program to perform file access operations according to the access policy, it also implements the following steps:
[0172] Invoke the MDS process corresponding to the real directory to perform file access operations.
[0173] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0174] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A file access method, characterized in that, The method includes: Upon receiving a file access request for a target real directory in the file system, the structural complexity parameter of the target real directory is determined; wherein, the structural complexity parameter characterizes the directory structure complexity of the target real directory; Based on the structural complexity parameters, the access strategy for the file access operation corresponding to the file access request is determined; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory; The file access operation is performed according to the access policy.
2. The method according to claim 1, characterized in that, The step of determining the access strategy for the file access operation based on the structural complexity parameters includes: When the structural complexity parameter is the first complexity parameter, the access strategy of the file access operation is determined according to the operation type of the file access operation corresponding to the file access request; When the structural complexity parameter is the second complexity parameter, the access strategy for the file access operation is determined to be a virtual directory access strategy. When the structural complexity parameter is the third complexity parameter, the access strategy for the file access operation is determined to be a non-virtual directory access strategy; wherein, the directory structure complexity represented by the first complexity parameter is lower than the directory structure complexity represented by the second complexity parameter; and the directory structure complexity represented by the second complexity parameter is lower than the directory structure complexity represented by the third complexity parameter.
3. The method according to claim 2, characterized in that, When there are multiple file access operations, determining the access strategy for each file access operation based on its operation type includes: Determine the first number of file access operations whose operation type is file-level operation, and determine the second number of file access operations whose operation type is directory-level operation; If the first number of operations is greater than or equal to the second number of operations, the access strategy for the file access operation is determined to be a virtual directory access strategy. If the number of the first operation is less than the number of the second operation, the access strategy for the file access operation is determined to be a non-virtual directory access strategy.
4. The method according to any one of claims 1-3, characterized in that, When the access policy is a virtual directory access policy, performing the file access operation according to the access policy includes: Based on directory relationship mapping information, candidate virtual directories corresponding to the target real directory are determined; wherein, the directory relationship mapping information includes the correspondence between different real directories and different virtual directories in the file system; Based on the file attribute information of the file to which the file access operation is applied, select the target virtual directory from the candidate virtual directories; The metadata storage (MDS) process corresponding to the target virtual directory is invoked to perform the file access operation.
5. The method according to claim 4, characterized in that, The step of selecting a target virtual directory from the candidate virtual directories based on the file attribute information of the file to which the file access operation is performed includes: The file hash value is determined based on the file attribute information of the file to which the file access operation is performed; Select the target virtual directory from the candidate virtual directories based on the file hash value.
6. The method according to any one of claims 1-3, characterized in that, When the access policy is a non-virtual directory access policy, performing the file access operation according to the access policy includes: The MDS process corresponding to the real directory is invoked to perform the file access operation.
7. A file access device, characterized in that, The device includes: The parameter determination module is used to determine the structural complexity parameters of the target real directory when a file access request for the target real directory in the file system is received; wherein, the structural complexity parameters characterize the directory structure complexity of the target real directory; The strategy determination module is used to determine the access strategy of the file access operation corresponding to the file access request based on the structural complexity parameters; wherein, the access strategy is a virtual directory access strategy or a non-virtual directory access strategy; the virtual directory access strategy is a strategy of accessing the target real directory through the target virtual directory corresponding to the target real directory; the non-virtual directory access strategy is a strategy of directly accessing the target real directory; The file access module is used to perform the file access operation according to the access policy.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.