Method and device for operating distributed object storage file by user mode file system

Abstract

The invention discloses a method and a device for operating a distributed object storage file by a user mode file system. The method comprises the following steps of: initializing a process of the user mode file system; reading a preset cache directory in the process of the user-mode file system, and mapping a first index node of a mounting directory of the user-mode file system to a second index node of the preset cache directory; the mounted directory comprises a user access path of the target file in the distributed object storage; performing recursive reading in sequence according to the user access path, and judging whether a preset cache directory contains all levels of directories of the user access path or not; if not, creating a corresponding directory according to the user access path under the preset cache directory, determining a second index node in the corresponding directory, and reading all levels of directories under the preset cache directory by the user state file system process according to a mapping relationship between the first index node and the second index node, and accessing the target file stored by the distributed object according to the directory at each level.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and in particular, to a method and device for a user-mode file system to operate on distributed object storage files. Background Art

[0002] With the rapid development of computer processing capabilities, data processing has involved a large amount of unstructured data. A single-machine file system can no longer meet the data storage requirements. Distributed object storage systems are chosen by more and more users due to their high availability and scalability. The S3 storage protocol (Simple Storage Service protocol) of distributed object storage provides a reliable, highly scalable, and low-cost object storage service. Its flexibility and flat architecture are more convenient for storage, with a capacity of more than EB, realizing theoretically unlimited expansion of object storage capacity and the number of objects. However, the access interface protocol of the S3 storage protocol of distributed object storage is single, and the data access performance is poor, so its application scope is limited. File storage can take into account multiple applications and multiple user accesses, and is more convenient for file sharing. When users store data, such as transcoding and storing audio and video files using audio and video transcoding software, they more expect to access files in distributed object storage through the file system protocol, so as to access the distributed object storage system as if accessing local files, without being restricted by the permission management of object storage, which is convenient for users to access. Summary of the Invention

[0003] In view of the above problems, embodiments of the present application are proposed to provide a method and device for a user-mode file system to operate on distributed object storage files that overcome the above problems or at least partially solve the above problems.

[0004] According to a first aspect of the embodiments of the present application, a method for a user-mode file system to operate on distributed object storage files is provided, which includes:

[0005] Initializing a user-mode file system process;

[0006] Reading a preset cache directory within the user-mode file system process, and mapping a first inode of the mount directory of the user-mode file system to a second inode of the preset cache directory; the mount directory includes the user access path of the target file in the distributed object storage;

[0007] Recursively reading in sequence according to the user access path, and determining whether the preset cache directory contains each level of directory of the user access path;

[0008] If not, creating a corresponding directory according to the user access path under the preset cache directory, and determining a second inode in the corresponding directory;

[0009] According to the mapping relationship between the first inode and the second inode, the user-space file system process reads the directories at all levels under the preset cache directory to access the target file in the distributed object storage according to the directories at all levels; the second inode stores metadata, and the metadata includes file type and file operation permissions.

[0010] Optionally, the user access path includes an object storage type, a bucket name, and a target object name.

[0011] Optionally, initializing the user-space file system process further includes:

[0012] Set the data structure of the first inode, and the data structure of the first inode is the same as that of the second inode;

[0013] Mapping the first inode of the mount directory of the user-space file system to the second inode of the preset cache directory specifically means:

[0014] Replace the first inode with the second inode of the preset cache directory.

[0015] Optionally, creating corresponding directories under the preset cache directory according to the user access path, and setting file operation permissions on the second inodes of the corresponding directories further includes:

[0016] Construct directories at all levels under the preset cache directory according to the user access path; the preset cache directory has the same level as the mount directory;

[0017] Set the file type and file operation permissions of the directories at all levels under the preset cache directory; the file operation permissions include read operation, write operation, and / or execute operation.

[0018] Optionally, if it is determined that the preset cache directory contains directories at all levels of the user access path, the method further includes:

[0019] According to the mapping relationship between the first inode and the second inode, the user-space file system process reads the existing directories at all levels under the preset cache directory to access the target file in the distributed object storage according to the directories at all levels.

[0020] Optionally, according to the mapping relationship between the first inode and the second inode, the user-space file system process reads the directories at all levels under the preset cache directory to access the target file in the distributed object storage further includes:

[0021] According to the mapping relationship between the first inode and the second inode, the user-space file system process sequentially obtains the file type and file operation permissions included in the second inodes of the directories at all levels under the preset cache directory;

[0022] Determine the object storage type, bucket name, and target object name of the distributed object storage according to the file types of each level of directories;

[0023] Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and operate on the target file according to the file operation permissions.

[0024] Optionally, obtaining the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and operating on the target file according to the file operation permissions includes:

[0025] Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and read the target file according to the read operation;

[0026] And / or,

[0027] Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, determine whether there is a write operation on the target file, and if so, write the target file according to the object storage type, bucket name, and target object name.

[0028] Optionally, further reading the preset cache directory within the user-mode file system process includes:

[0029] Determine whether there is a preset cache directory;

[0030] If not, construct the preset cache directory and read the preset cache directory within the user-mode file system process.

[0031] Optionally, further reading the preset cache directory within the user-mode file system process includes:

[0032] Determine whether there is a preset cache directory and whether there are operation permissions for the preset cache directory;

[0033] If not, the user-mode file system process fails to start.

[0034] According to the second aspect of the embodiments of the present application, there is provided a user-mode file system device for operating on distributed object storage files, which includes:

[0035] An initialization module, adapted to initialize the user-mode file system process;

[0036] A mapping module, adapted to read a preset cache directory within the user-mode file system process, and map a first inode of the mount directory of the user-mode file system to a second inode of the preset cache directory; the mount directory includes the user access path of the target file in the distributed object storage;

[0037] A directory reading module, adapted to recursively read in sequence according to a user access path, and determine whether a preset cache directory contains each level directory of the user access path;

[0038] A directory creating module, adapted to, if it is determined that the preset cache directory does not contain each level directory of the user access path, create corresponding directories under the preset cache directory according to the user access path, and determine a second inode in the corresponding directories;

[0039] A file operation module, adapted to, according to the mapping relationship between a first inode and a second inode, read each level directory under the preset cache directory by a user-mode file system process, so as to access a target file in a distributed object storage according to each level directory; the second inode stores metadata, and the metadata includes a file type and file operation permissions.

[0040] According to a third aspect of the embodiments of the present application, there is provided a computing device, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0041] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned user-mode file system's method for operating distributed object storage files.

[0042] According to a fourth aspect of the embodiments of the present application, there is provided a computer storage medium, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to execute the operations corresponding to the above-mentioned user-mode file system's method for operating distributed object storage files.

[0043] According to a fifth aspect of the embodiments of the present application, there is provided a computer program product, including at least one executable instruction, and the executable instruction causes a processor to execute the operations corresponding to the above-mentioned user-mode file system's method for operating distributed object storage files.

[0044] According to the method and device for operating distributed object storage files by the user-mode file system provided by the present application, a preset cache directory is set in an operating system, and a user-mode file system process reads the preset cache directory, maps a first inode of a mounting directory of the user-mode file system to a second inode of the preset cache directory, and creates corresponding directories under the preset cache directory in sequence according to the user access path of the mounting directory. The preset cache directory has the same level as the mounting directory, which facilitates the first inode to directly reuse the second inode of the preset cache directory. The second inode can indirectly manage the first inode, and the second inode can be persistently stored, will not be lost due to the restart of the user-mode file system process, and does not need to borrow a third-party persistent component, without increasing costs.

[0045] The above description is only an overview of the technical solution of this application. In order to understand the technical means of this application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specific embodiments of this application are specifically given. Description of the Drawings

[0046] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of this application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0047] Figure 1 Shows a flowchart of a method for operating a distributed object storage file by a user-mode file system according to an embodiment of this application;

[0048] Figure 2 Shows a schematic diagram of a user-mode file system accessing a distributed object storage file;

[0049] Figure 3 Shows a flowchart of a method for operating a distributed object storage file by a user-mode file system according to another embodiment of this application;

[0050] Figure 4 Shows a schematic structural diagram of an apparatus for operating a distributed object storage file by a user-mode file system according to an embodiment of this application;

[0051] Figure 5 Shows a schematic structural diagram of a computing device according to an embodiment of this application. Detailed Embodiments

[0052] The exemplary embodiments of this application will be described in more detail below with reference to the drawings. Although the exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that this application can be understood more thoroughly and the scope of this application can be fully conveyed to those skilled in the art.

[0053] First, the noun terms involved in one or more embodiments of this application are explained.

[0054] Unix: A multi-user, multi-process computer operating system, originating in the 1970s, originally designed for network operating systems but can also be used as a stand-alone operating system.

[0055] Linux: An open-source, Unix-like operating system.

[0056] Inode: Index Node, an index node, a core concept in the file systems of Unix and Unix-like operating systems (such as Linux). It is a data structure that stores file metadata in the file system.

[0057] fuse: Filesystem in Userspace, an interface that allows users to create custom file systems in user space. fuse.ko is the kernel module for the user-space file system, and this module is generally built into the Linux system.

[0058] libfuse: A user-space library that provides a function library for communicating with the fuse.ko kernel module.

[0059] VFS: Virtual File System, an abstracted virtual file system that abstracts many different file system types and implements a common access method for all file system types.

[0060] Figure 1 The flowchart of the method for operating on distributed object storage files by the user-space file system according to an embodiment of the present application is shown, as Figure 1 shown, and the method includes the following steps:

[0061] Step S101, initialize the user-space file system process.

[0062] When operating on distributed object storage files based on the user-space file system, since distributed object storage files consider data security issues and set permission control policies in distributed object storage, such as setting file access permissions for storage buckets and target objects, etc., after building a user-space file system using a library such as libfuse, when accessing distributed object storage files, it is necessary to add more parameter information for permission authentication, etc. In addition to the target file, a lot of redundant information is added, which makes the operation of the target file complex for users. Moreover, when the user-space file system accesses distributed object storage, it needs to use the index node Inode. However, if the Inode information is stored in the memory of the user-space file system process, when the user-space file system process restarts, the Inode information will be lost and cannot be persistently stored. If the Inode information is stored in a third-party persistent component, etc., it will increase complexity and operation and maintenance costs.

[0063] Considering the above problems, in this embodiment, the inodes used by the user-space file system process can reuse the inodes of the operating system. By maintaining the mapping relationship between the inodes of the operating system and the inodes in the user-space file system process, the inode information of the user-space file system process can be persistently saved, which is convenient for subsequent reuse and will not be lost due to the shutdown and restart of the user-space file system process. When using the user-space file system, first initialize the user-space file system process. During initialization, the user-space file system is based on the FUSE (Filesystem in Userspace) framework and is implemented by, for example, setting the FUSE library, registering callback functions, processing mount parameters, starting the main loop to listen for and process file system requests, etc. When processing the mount parameters, set the corresponding mount directory. The mount directory can use, for example, "mount" as the root directory, and the subordinate directories include the user access path of the target file in the distributed object storage. Here, the user access path corresponds to the location where the target file is stored in the distributed object storage, rather than the directory of the file system in the operating system. The user access path includes the object storage type, bucket name, and target object name, etc. For example, the object storage type is oss, the bucket name is bucket, and the target object name is object. Through the user access path, it can be determined that the target file is stored at the object location in the bucket of the oss cloud server. According to the user access path, the mount directory is determined to be " / mount / oss / bucket / object". After initializing the user-space file system process, the mount directory is obtained. The mount directory here is obtained based on the user access path. In this embodiment, the mount directory is correspondingly converted into a preset cache directory, and a mapping relationship is established between the first inode of each subdirectory of the mount directory and the second inode of the preset cache directory. The second inode of the preset cache directory is reused, and the second inode records the metadata corresponding to the user access path of the distributed object storage, without storing it in the user-space file system process.

[0064] Step S102, read the preset cache directory within the user-space file system process, and map the first inode of the mount directory of the user-space file system to the second inode of the preset cache directory.

[0065] The file system of the operating system can use, for example, the Linux file system, and the second inode uses the data structure of the Inode index node of the Linux file system. The first inode of the mount directory of the user-space file system also uses the same data structure as the second inode of the file system in the operating system, which is convenient for establishing a mapping relationship between the first inode and the second inode, so that the file system of the operating system can use the second inode to manage the first inode within the user-space file system process.

[0066] Specifically, after the user-mode file system process is initialized, the preset cache directory of the operating system is read within the user-mode file system process. The preset cache directory is set in the file system of the operating system and can be persistently stored. The root directory of the preset cache directory can be preset, such as / cache_path. Based on the root directory, various subdirectories can be constructed below it corresponding to the user access paths of the distributed object storage. The metadata (inode information) corresponding to the user access paths can be persistently stored in the file system of the operating system, avoiding loss due to the restart of the user-mode file system process and facilitating subsequent direct access to the distributed object storage based on the persistent inode information to operate on the target file. The metadata of the file can be stored in the second inode of the preset cache directory. The metadata includes the file type and file operation permissions. Each subdirectory in the preset cache directory has its own second inode information corresponding to the file type and file operation permissions of each subdirectory. Here, to facilitate the user to directly operate when accessing the target file of the distributed object storage, the file operation permissions can be set to include all permissions such as read operation, write operation, and execution operation, without the need to consider permission authentication and other issues when accessing the target file of the distributed object storage.

[0067] After the user-mode file system process reads the preset cache directory of the operating system, the first inode of the mount directory of the user-mode file system is mapped to the second inode of the preset cache directory, establishing a mapping relationship between the first inode and the second inode. Subsequently, the information of the first inodes of various subdirectories constructed in the preset cache directory can be synchronized with the second inode, so that the user-mode file system process can reuse the information of the first inode.

[0068] Step S103, recursively read in sequence according to the user access path, and determine whether the preset cache directory contains each level of directory of the user access path.

[0069] Recursively read in sequence according to the object storage type, bucket name, and target object name included in the user access path in the mount directory, that is, read the object storage type, bucket name, and target object name in sequence. According to the read object storage type, bucket name, and target object name, determine whether the corresponding levels of directories are included in the preset cache directory. If not, execute step S104 to create each level of directory in sequence. If included, there is no need to create directories again. The levels of directories can be determined according to the preset cache directory. According to the metadata included in the second inode information of each level of directory, access the target file of the distributed object storage correspondingly.

[0070] For the preset cache directory, it is possible to determine whether the preset cache directory contains the directories at all levels corresponding to the user access path through methods such as directory lookup and directory testing. When making the determination, the user access path is judged level by level to ensure the consistency of the directories at all levels under the preset cache directory with the user access path, and to ensure that the subsequent user-mode file system process can correctly determine the storage location of the distributed object storage based on the preset cache directory and operate on the target file.

[0071] Step S104: Create a corresponding directory under the preset cache directory according to the user access path, and determine the second inode in the corresponding directory.

[0072] When the preset cache directory does not contain the directories at all levels corresponding to the user access path, create first-level subdirectories, second-level subdirectories, etc. under the preset cache directory in sequence according to the user access path. When creating the subdirectories at all levels under the preset cache directory, the second inode information corresponding to various subdirectories can be recorded, including file type, file operation permissions, etc. The file type is set to different file types according to the object storage type, bucket name, and target object name included in the user access path. The file operation permissions can be set to different permissions according to different requirements, such as read operation, write operation, execution operation, etc., which are specifically set according to the implementation situation and are not limited here.

[0073] After constructing the directories corresponding to the user access path under the preset cache directory, the second inodes of the directories at all levels store the metadata of the directories at all levels. The metadata includes the file type and the file operation permissions. Based on the second inodes of the directories at all levels, the location of the target file in the distributed object storage, the file operation permissions, etc. can be obtained.

[0074] Step S105: According to the mapping relationship between the first inode and the second inode, the user-mode file system process reads the directories at all levels under the preset cache directory to access the target file of the distributed object storage according to the directories at all levels.

[0075] According to the mapping relationship between the first inode and the second inode, that is, the first inode can be replaced by the second inode. Thus, the user-mode file system process can directly obtain the information of the second inode based on the first inode, and then read the directories at all levels under the preset cache directory. The user-mode file system process converts the hierarchical path of the directories at all levels under the preset cache directory obtained into the path of the flat structure of the distributed object storage, and by calling the corresponding operation interface, obtains the target file of the distributed object storage and operates on it, which will not be elaborated here.

[0076] In an optional embodiment, such as Figure 2As shown, taking the Linux operating system as an example, the target file of the distributed object storage is located in storage terminals such as XX cloud storage and private cloud storage. When a user accesses the target file, a mount directory is specified, such as / mount / oss / bucket / object, that is, the cloud storage of the target file is in the object of bucket in the oss cloud storage. According to the mount directory, the user-space file system communicates with fuse (Filesystem in Userspace) based on libfuse (user-space library), and interacts with the VFS (Virtual File Systems), XFS (Extended Filesystem), ext4 (Fourth extended filesystem), etc. of the Linux file system (the interaction process is not shown in this figure). The first inode of the user-space file system reuses the second inode of the Linux file system. The cloud storage terminal, bucket name, object name, etc. are determined according to the file type. The object file is obtained from the cloud storage according to the file operation permission. The second inode can be persistently stored in the Linux file system, and there is no need to reset the file type and file operation permission every time the target file is obtained.

[0077] According to the method for operating on distributed object storage files by the user-space file system provided by this application, a preset cache directory is set in the operating system. The user-space file system process reads the preset cache directory, maps the first inode of the mount directory of the user-space file system to the second inode of the preset cache directory, and creates corresponding directories under the preset cache directory in sequence according to the user access path of the mount directory. The preset cache directory has the same level as the mount directory, which is convenient for the first inode to directly reuse the second inode of the preset cache directory. The first inode can be indirectly managed through the second inode, and the second inode can be persistently stored, will not be lost due to the restart of the user-space file system process, and there is no need to borrow a third-party persistent component, without increasing costs.

[0078] Figure 3 The flowchart of the method for operating on distributed object storage files by the user-space file system according to an embodiment of this application is shown, as Figure 3 shown, the method includes the following steps:

[0079] Step S301, initialize the user-space file system process.

[0080] The user-space file system process is initialized, and the user specifies a mount directory so that the user-space file system can access the mount directory. Set the data structure of the first inode of the user-space file system process, where the data structure of the first inode is the same as that of the second inode used by the operating system's file system, so that the first inode can be replaced with the second inode later. The data structure of the first inode is as follows:

[0081] struct inode{

[0082] unsigned int i_mode; / / File type and permissions

[0083] unsigned int i_nlink; / / Hard link count

[0084] unsigned int i_size; / / File size (bytes)

[0085] unsigned long i_atime; / / Last access time

[0086] unsigned long i_mtime; Last modification time

[0087] unsigned long i_ctime; / / inode modification time

[0088] unsigned int i_ino; / / inode number

[0089] unsigned int i_blocks; / / Number of disk blocks occupied

[0090] struct super_block *i_sb; / / Pointer to the file system super block

[0091] / / Other file system specific fields:

[0092] }

[0093] The fields of the above data structure of the first inode are for illustrative purposes, and are specifically set according to the data structure of the second inode. Among them, the file type and permission fields are used to determine the location of the target file in the distributed object storage later, and the permissions for the user to operate on the target file.

[0094] Step S302, read the preset cache directory within the user-space file system process, determine the data structure of the first inode of the user-space file system according to the data structure of the second inode, and replace the first inode of the mount directory of the user-space file system with the second inode of the preset cache directory.

[0095] The preset cache directory is set within the file system of the operating system and can be set in advance, such as set to / cache_path. The user-space file system process can read the preset cache directory. When reading, it can first determine whether the preset cache directory exists. If it does not exist, the preset cache directory can be created first, and then the preset cache directory can be read within the user-space file system process. Or, if it is determined whether the preset cache directory exists and has the operation permission for the preset cache directory to increase permission control for the user. If not, it means the user does not have the permission to access the preset cache directory, and the user-space file system process fails to start. It can be set according to the actual implementation situation, and the judgment of the preset cache directory can be set according to requirements, which is not limited here.

[0096] Read the preset cache directory within the user-space file system process. The second inode of the preset cache directory stores the metadata of the file, and the metadata includes the file type and file operation permission. Replace the first inode of the mount directory of the user-space file system with the second inode of the preset cache directory. The first inode can be managed through the second inode, that is, after setting the metadata of the second inodes of each subdirectory under the preset cache directory, the user-space file system process can reuse the metadata of the second inode through the mapping relationship between the first inode and the second inode, so as to realize the management of the first inode by borrowing the second inode, which is convenient for the user-space file system process to use the metadata of the first inode.

[0097] Step S303: Recursively read in sequence according to the user access path, and determine whether the preset cache directory contains each level of directory in the user access path.

[0098] The mount directory contains the user access path of the target file in the distributed object storage. The user access path includes the object storage type, bucket name, and target object name. For example, the mount directory is / mount / oss / bucket / object, where the user access path includes / oss / bucket / object, oss is cloud storage, bucket corresponds to the bucket name, and object corresponds to the target object name. According to the user access path, recursively read in sequence, that is, read the next-level subdirectory, second-level subdirectory, and third-level subdirectory under the user access path in sequence, and determine whether the corresponding first-level subdirectory, second-level subdirectory, and third-level subdirectory are included under the preset cache directory. If the corresponding directory is not included, execute step S304. If all recursive judgments include the directory, execute step S305.

[0099] Step S304: Construct each level of directory under the preset cache directory according to the user access path, and set the file type and file operation permission of each level of directory under the preset cache directory.

[0100] When it is determined that the preset cache directory does not contain the directories at all levels of the user access path, corresponding directories can be created under the preset cache directory. For example, if the preset cache directory is / cache_path, and it is determined that it does not contain the first-level subdirectory of the user access path, then a first-level subdirectory is created for it under the preset cache directory, and the preset cache directory becomes / cache_path / oss. Set the metadata of the second inode of the first-level subdirectory, and set the file type to, for example, the object storage type. The user-mode file system process can determine the corresponding storage terminal according to the file type, and set the file operation permissions to, for example, read operation, write operation, and execute operation. Here, all file operation permissions are set. The user-mode file system process can read the next-level directory according to the file operation permissions until the target file is read and operate on the target file. Here, the file type of the second inode of the first-level subdirectory is set to the object storage type, the file type of the second inode of the second-level subdirectory is set to the bucket name, and the file type of the second inode of the third-level subdirectory is set to the target object name. According to the file types of the first-level subdirectory, the second-level subdirectory, and the third-level subdirectory, the flat structure of the distributed object storage can be determined. The path structure of the distributed object storage mainly consists of the storage cloud (object storage type), buckets, and objects. Each bucket can store thousands of objects, and each object has a unique identifier. In this embodiment, the corresponding path structure of the distributed object storage obtained is the object storage type / bucket name / target object name, and the target file can be determined accordingly.

[0101] After constructing the corresponding directories, if the directory is not the last level of the user access path, then step S303 is continued to be executed, and according to the user access path, the corresponding directories at all levels are constructed under the preset cache directory until the user access path under the mount directory is converted into the directories at all levels under the preset cache directory, that is, the mount directory / mount / oss / bucket / object is correspondingly converted into / cache_path / oss / bucket / object, and the final preset cache directory has the same level as the mount directory. The metadata of the second inode of the directories at all levels under the preset cache directory, its file type is set to, for example, the object storage type, bucket name, target object name according to the corresponding levels of the directories. The above is for illustrative purposes, and it is specifically set according to the flat structure of the distributed object storage, and no limitation is made here. The file operation permissions are set to, for example, read operation, write operation, and execute operation to facilitate corresponding operations on the target file after it is read.

[0102] Step S305: According to the mapping relationship between the first index node and the second index node, the user-mode file system process sequentially obtains the file types and file operation permissions included in the first index node of each level of directory under the preset cache directory. According to the file types of each level of directory, determine the object storage type, bucket name, and target object name of the distributed object storage. Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and operate on the target file according to the file operation permissions.

[0103] When each level of directory containing the user access path is under the preset cache directory, according to the mapping relationship between the first index node and the second index node, the metadata of the first index node reuses the metadata of the second index node, and the user-mode file system process can directly obtain each location information of the distributed object storage based on the first index node.

[0104] Specifically, when the user-mode file system process reads the preset cache directory, it sequentially obtains the file types and file operation permissions included in the second index node of each level of directory under the preset cache directory. According to the file types of each level of directory, the object storage type, bucket name, and target object name of the distributed object storage can be determined. According to the object storage type, determine the cloud storage terminal for storing the target file, the bucket name determines the corresponding bucket, and the target object name determines the target object of the bucket. The identifier of the target file in the distributed object storage is obtained according to the object storage type, bucket name, and target object name. The user-mode file system process can call the corresponding interface of the distributed object storage according to the identifier to obtain the target file of the distributed object storage, and operate on the target file according to the file operation permissions.

[0105] The operations on the target file can include read operations, write operations, execution operations, etc. For read operations and execution operations, after obtaining the target file of the distributed object storage according to the object storage type, bucket name, and target object name, according to the existing file operation permissions, such as a read operation can read the target file correspondingly for display, or according to the execution operation, obtain the target file for execution, etc. For write operations, after obtaining the target file of the distributed object storage according to the object storage type, bucket name, and target object name, after modifying the target file, it can first determine whether there is write operation permission for the target file. If so, determine the target file location according to the object storage type, bucket name, and target object name, and write the target file to complete the modification and update of the target file. The above is for illustrative purposes, and it is specifically set according to the actual situation and is not limited here.

[0106] According to the method for operating distributed object storage files by the user-mode file system provided in this application, using a preset cache directory, the mounting directory of the user-mode file system is converted into each level of directory under the preset cache directory. The corresponding file type and file operation permissions are set in the second inode of the preset cache directory. The user-mode file system process sequentially reads each level of directory of the preset cache directory, obtains the path structure of the distributed object storage according to the file type, and can conveniently access the target object in the distributed object storage according to the file operation permissions without adding too many access parameters. The second inode can be persistently stored in the file system of the operating system. When the user accesses the target file again based on the user-mode file system process later, the converted preset cache directory can be directly found through the mounting directory, and the target file can be accessed based on the metadata of the second inode stored in the preset cache directory without specifically storing the corresponding inode, reducing complexity and cost.

[0107] Figure 4 FIG. 4 shows a schematic structural diagram of an apparatus for operating distributed object storage files by the user-mode file system provided in an embodiment of this application. As Figure 4 shown, the apparatus includes:

[0108] An initialization module 410, adapted to initialize the user-mode file system process;

[0109] A mapping module 420, adapted to read the preset cache directory within the user-mode file system process and map the first inode of the mounting directory of the user-mode file system to the second inode of the preset cache directory; the mounting directory includes the user access path of the target file in the distributed object storage;

[0110] A directory reading module 430, adapted to recursively read sequentially according to the user access path and determine whether the preset cache directory contains each level of directory of the user access path;

[0111] A directory creation module 440, adapted to, if it is determined that the preset cache directory does not contain each level of directory of the user access path, create a corresponding directory under the preset cache directory according to the user access path and determine the second inode in the corresponding directory;

[0112] A file operation module 450, adapted to, according to the mapping relationship between the first inode and the second inode, read each level of directory under the preset cache directory by the user-mode file system process to access the target file in the distributed object storage according to each level of directory; the second inode stores metadata, and the metadata includes the file type and the file operation permissions.

[0113] Optionally, the user access path includes the object storage type, the bucket name, and the target object name.

[0114] Optionally, the initialization module 410 is further adapted to:

[0115] Set the data structure of the first index node, which is the same as that of the second index node;

[0116] The mapping module 420 is further adapted to:

[0117] Replace the first index node with the second index node of the preset cache directory.

[0118] Optionally, the file operation module 450 is further adapted to:

[0119] Construct the directories at all levels under the preset cache directory according to the user access path; the preset cache directory has the same level as the mount directory;

[0120] Set the file types and file operation permissions of the directories at all levels under the preset cache directory; the file operation permissions include read operation, write operation, and / or execution operation.

[0121] Optionally, if it is determined that the preset cache directory contains the directories at all levels of the user access path, the file operation module 450 is further adapted to:

[0122] According to the mapping relationship between the first index node and the second index node, the user-mode file system process reads the existing directories at all levels under the preset cache directory to access the target file in the distributed object storage according to the directories at all levels.

[0123] Optionally, the file operation module 450 is further adapted to:

[0124] According to the mapping relationship between the first index node and the second index node, the user-mode file system process sequentially obtains the file types and file operation permissions included in the second index node of the directories at all levels under the preset cache directory;

[0125] Determine the object storage type, bucket name, and target object name of the distributed object storage according to the file types of the directories at all levels;

[0126] Obtain the target file in the distributed object storage according to the object storage type, bucket name, and target object name, and operate on the target file according to the file operation permissions.

[0127] Optionally, the file operation module 450 is further adapted to:

[0128] Obtain the target file in the distributed object storage according to the object storage type, bucket name, and target object name, and read the target file according to the read operation;

[0129] And / or,

[0130] Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and determine whether there is a write operation on the target file. If so, write the target file according to the object storage type, bucket name, and target object name.

[0131] Optionally, the mapping module 420 is further adapted to:

[0132] Determine whether there is a preset cache directory;

[0133] If not, construct a preset cache directory and read the preset cache directory within the user-space file system process.

[0134] Optionally, the mapping module 420 is further adapted to:

[0135] Determine whether there is a preset cache directory and whether there is an operation permission for the preset cache directory;

[0136] If not, the user-space file system process fails to start.

[0137] The descriptions of the above modules refer to the corresponding descriptions in the method embodiments and will not be elaborated here.

[0138] According to the user-space file system for distributed object storage file operation device provided by the present application, a preset cache directory is set in the operating system, the user-space file system process reads the preset cache directory, maps the first inode of the mount directory of the user-space file system to the second inode of the preset cache directory, and creates corresponding directories under the preset cache directory in sequence according to the user access path of the mount directory. The preset cache directory has the same level as the mount directory, which facilitates the first inode to directly reuse the second inode of the preset cache directory. Through the second inode, the first inode can be indirectly managed, and the second inode can be persistently stored, will not be lost due to the restart of the user-space file system process, and there is no need to borrow a third-party persistent component, without increasing costs.

[0139] The present application also provides a non-volatile computer storage medium, and the computer storage medium stores at least one executable instruction, and the executable instruction can execute the operations corresponding to the user-space file system for distributed object storage file operation method in any of the above method embodiments.

[0140] The present application also provides a computer program product, and the computer program product includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to execute the operations corresponding to the user-space file system for distributed object storage file operation method in any of the above method embodiments.

[0141] Figure 5The structural schematic diagram of a computing device according to an embodiment of the present application is shown. The specific implementation of the present application does not limit the specific implementation of the computing device.

[0142] As Figure 5 shown, the computing device may include: a processor 502, a communications interface 504, a memory 506, and a communication bus 508.

[0143] Among them:

[0144] The processor 502, the communications interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0145] The communications interface 504 is used to communicate with network elements of other devices such as clients or other servers.

[0146] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned embodiments of the method for operating distributed object storage files by the user-mode file system.

[0147] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0148] The processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the present application. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0149] The memory 506 is used to store the program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0150] The program 510 is specifically used to enable the processor 502 to execute the method for operating distributed object storage files by the user-mode file system in any of the above method embodiments. For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the corresponding units in the above-mentioned embodiments of the method for operating distributed object storage files by the user-mode file system, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated here.

[0151] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. A variety of general-purpose systems may also be used in conjunction with the teachings presented herein. The structure required to construct such systems will be apparent from the above description. Additionally, this application is not directed to any particular programming language. It should be appreciated that the teachings of this application can be implemented in a variety of programming languages, and the description of a particular language above is for the purpose of disclosing the preferred embodiments of this application.

[0152] In the specification provided herein, numerous specific details are set forth. However, it can be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0153] Similarly, it should be understood that in order to streamline this application and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of this application, the various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed subject matter of this application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all of the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

[0154] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0155] In addition, those skilled in the art can understand that although some embodiments herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of this application and forms different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0156] Each component embodiment of this application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components according to this application. This application can also be implemented as a device or apparatus program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0157] It should be noted that the above embodiments illustrate this application rather than limit this application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for a user-space file system to operate on files in a distributed object storage, which includes: Initializing a user-space file system process; Reading a preset cache directory within the user-space file system process, and mapping a first inode of a mount directory of the user-space file system to a second inode of the preset cache directory; The mount directory contains a user access path of a target file in the distributed object storage; Recursively reading in sequence according to the user access path, and determining whether the preset cache directory contains each level of directory of the user access path; If not, creating corresponding directories under the preset cache directory according to the user access path, and determining a second inode in the corresponding directories; According to the mapping relationship between the first inode and the second inode, the user-space file system process reads each level of directory under the preset cache directory to access the target file in the distributed object storage according to each level of directory; The second inode stores metadata, and the metadata includes a file type and file operation permissions.

2. The method according to claim 1, wherein The user access path includes an object storage type, a bucket name, and a target object name.

3. The method according to claim 1 or 2, wherein, The initializing the user-space file system process further includes: Setting a data structure of the first inode, and the data structure of the first inode is the same as that of the second inode; The mapping the first inode of the mount directory of the user-space file system to the second inode of the preset cache directory specifically is: Replacing the first inode with the second inode of the preset cache directory.

4. The method according to any one of claims 1 to 3, wherein, The creating corresponding directories under the preset cache directory according to the user access path, and setting file operation permissions for the second inode in the corresponding directories further includes: Constructing each level of directory under the preset cache directory according to the user access path; the preset cache directory has the same level as the mount directory; Setting the file type and file operation permissions for each level of directory under the preset cache directory; the file operation permissions include read operation, write operation, and / or execute operation.

5. The method according to any one of claims 1-4, wherein, If it is determined that the preset cache directory contains each level of directory of the user access path, the method further includes: According to the mapping relationship between the first inode and the second inode, the user-space file system process reads the existing each level of directory under the preset cache directory to access the target file in the distributed object storage according to each level of directory.

6. The method according to any one of claims 1-5, wherein, The according to the mapping relationship between the first inode and the second inode, the user-space file system process reads each level of directory under the preset cache directory to access the target file in the distributed object storage according to each level of directory further includes: According to the mapping relationship between the first inode and the second inode, the user-space file system process sequentially obtains the file type and file operation permissions included in the second inode of each level of directory under the preset cache directory; Determining the object storage type, bucket name, and target object name of the distributed object storage according to the file type of each level of directory; Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and operate on the target file according to the file operation permission.

7. The method according to any one of claims 1-6, wherein The step of obtaining the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and operating on the target file according to the file operation permission further includes: Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, and read the target file according to the read operation; and / or, Obtain the target file of the distributed object storage according to the object storage type, bucket name, and target object name, determine whether there is a write operation on the target file, and if so, write the target file according to the object storage type, bucket name, and target object name.

8. The method according to any one of claims 1-7, wherein, The step of reading the preset cache directory within the user-mode file system process further includes: Determine whether the preset cache directory exists; If not, construct the preset cache directory and read the preset cache directory within the user-mode file system process.

9. The method according to any one of claims 1-7, wherein, The step of reading the preset cache directory within the user-mode file system process further includes: Determine whether the preset cache directory exists and whether there is an operation permission for the preset cache directory; If not, the user-mode file system process fails to start.

10. A user-mode file system device for operating on distributed object storage files, which includes: An initialization module adapted to initialize the user-mode file system process; A mapping module adapted to read the preset cache directory within the user-mode file system process and map the first inode of the mount directory of the user-mode file system to the second inode of the preset cache directory; The mount directory includes the user access path of the target file in the distributed object storage; A directory reading module adapted to recursively read in sequence according to the user access path and determine whether the preset cache directory contains each level of directory of the user access path; A directory creation module adapted to, if it is determined that the preset cache directory does not contain each level of directory of the user access path, create corresponding directories under the preset cache directory according to the user access path, and determine the second inode in the corresponding directories; A file operation module adapted to, according to the mapping relationship between the first inode and the second inode, read each level of directory under the preset cache directory by the user-mode file system process to access the target file of the distributed object storage according to each level of directory; The second inode stores metadata, and the metadata includes file type and file operation permission.

11. A computing device, comprising: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the user-mode file system method for operating on distributed object storage files according to any one of claims 1-9.

12. A computer storage medium storing at least one executable instruction, the executable instruction causing a processor to perform an operation corresponding to the method for operating a distributed object storage file by the user-mode file system according to any one of claims 1-9.

13. A computer program product including at least one executable instruction, the executable instruction causing a processor to perform an operation corresponding to the method for operating a distributed object storage file by the user-mode file system according to any one of claims 1-9.

Images