File system management method, device, equipment and product based on eBPF program

Abstract

The present invention discloses a file system management method, apparatus, device, and product based on an eBPF program, comprising establishing a data table; linking the eBPF program with a VFS program within an operating system kernel, monitoring input file events, and updating the data table accordingly; and searching the data table in response to a file ID input by an external search application, concatenating the file path of the input file ID, and returning the result to the external search application. The present invention discloses using an eBPF program to link with a VFS program within an operating system kernel, monitoring input file events, and simultaneously establishing a data table. Using the file ID as a unique identifier for the file, the device searches for the file path required by the external search application. The method has the characteristics of high indexing accuracy, good file monitoring effect when files change frequently, and reliable file system management support for multiple external applications.

Description

Technical Field

[0001] The present invention belongs to the field of file system management, and in particular relates to a file system management method, apparatus, device and product based on an eBPF program. Background Art

[0002] Under existing technical conditions, the establishment of existing file indexes requires traversing the directory and parsing the files to generate the index during the traversal process. This process is very time-consuming and consumes a lot of system resources. Especially when files change frequently in the computer system, the inotify program is usually used to monitor file changes and update the index in real time. When the inotify program monitors a large number of file changes, there are obvious performance issues, which will affect the user experience when searching for files through the index. Summary of the Invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a file system management method, device, equipment and product based on eBPF program.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] In a first aspect, the present invention discloses a file system management method based on an eBPF program, which establishes a data table including the file ID, inode number, and parent directory inode number corresponding to all files;

[0006] Hook the eBPF program with the VFS program in the operating system kernel to monitor input file events and update the data table accordingly;

[0007] In response to a file ID input by an external search application, searching the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID;

[0008] The file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number are concatenated to form the file path of the input file ID and returned to the external search application.

[0009] In one embodiment of the present invention, the eBPF program is hooked to the VFS program in the kernel to monitor the input file events, and the method further includes: sending the monitored file events to an external file monitoring application.

[0010] In one embodiment of the present invention, a metadata database of files is established, the metadata database including file IDs and metadata of files corresponding to the file IDs;

[0011] In response to the file path input by the external metadata service application, the stat function is used to obtain the inode number corresponding to the input file path, the data table is searched, and the file ID corresponding to the inode number corresponding to the input file path is obtained. According to the file ID corresponding to the input file path, the metadata database is searched to obtain the corresponding metadata, and the metadata is returned to the external metadata service application.

[0012] In one embodiment of the present invention, sending the monitored file event to the external file monitoring application includes: sending the monitored file event to the external file monitoring application with different permissions according to different permissions of the external file monitoring application.

[0013] In one embodiment of the present invention, the method further includes: establishing a corresponding data table for each mount point in the operating system.

[0014] In one embodiment of the present invention, the table name of the data table created by the mount point is the uuid of the corresponding mount point.

[0015] In a second aspect, the present invention discloses a file system management device based on an eBPF program, the device comprising:

[0016] The data table creation module is used to create a data table, which includes the file ID, inode number and parent directory inode number corresponding to all files;

[0017] The file monitoring and update module is used to hook the eBPF program with the VFS program in the operating system kernel, monitor the input file events, and update the data table accordingly;

[0018] A search module for searching the data table in response to a file ID input by an external search application to determine a file inode number and a parent directory inode number corresponding to the input file ID;

[0019] The result return module is used to concatenate the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form the file path of the input file ID, and return it to the external search application.

[0020] In a third aspect, the present invention discloses an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute any one of the above methods.

[0021] In a fourth aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0022] In a fifth aspect, the present invention discloses a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] The present invention discloses a file system management method, device, equipment and product based on an eBPF program, comprising establishing a data table; linking the eBPF program with a VFS program in an operating system kernel, monitoring input file events, and updating the data table accordingly; searching the data table in response to a file ID input by an external search application; concatenating the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form a file path for the input file ID, and returning the result to the external search application; the present invention discloses a file system management method, device, equipment and product based on an eBPF program, which can utilize the eBPF program to link with the VFS program in an operating system kernel, monitor input file events, establish a data table at the same time, use the file ID as the unique identifier of the file, search for the file path required by the external search application, and have the characteristics of high index accuracy, good file monitoring effect when files change frequently, and reliable file system management support for multiple external applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] In the attached figure:

[0027] Figure 1 This is a schematic diagram of a file system management method based on an eBPF program according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a file system management method based on an eBPF program supporting external search applications according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a file system management method based on an eBPF program supporting an external file monitoring application according to an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a file system management method based on an eBPF program supporting an external metadata service application according to an embodiment of the present invention;

[0031] Figure 5Establishing data representation intent for a mount point of a file system management method based on an eBPF program according to an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of a file system management device based on an eBPF program according to an embodiment of the present invention;

[0033] Figure 7 A schematic diagram of an electronic device for file system management based on an eBPF program according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other.

[0035] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0036] In the description of the present invention, it should be further clarified that the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0037] Under existing technical conditions, the inotify program is usually used to monitor file changes and update the index in real time. When monitoring a large number of file changes, the inotify program has obvious performance problems, which will affect the user experience when searching for files through the index. The present invention discloses a file system management method, device, equipment and product based on the eBPF program, which links the eBPF program with the VFS program in the operating system kernel, monitors the input file events, and establishes a data table at the same time. The file ID is used as the unique identifier of the file to search for the file path required by the external search application. The method has the advantages of high index accuracy, good file monitoring effect when files change frequently, and provides reliable file system management support for various external applications.

[0038] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0039] like Figure 1 and2 As shown, in one embodiment of the present invention, a file system management method based on an eBPF program includes:

[0040] Step S101: Create a data table, which includes the file ID, inode number and parent directory inode number corresponding to all files;

[0041] In this embodiment, the file ID is a natural number that is assigned to each piece of data and increases incrementally when the data table is created.

[0042] In another embodiment, the data table further includes the name of the file;

[0043] Step S102: Link the eBPF program to the VFS program in the operating system kernel, monitor the input file events, and update the data table accordingly;

[0044] In this embodiment, the eBPF program and the VFS program run in the Linux kernel layer. The VFS program serves as a unified interface for file operations in the Linux operating system, and the eBPF program monitors file events. The eBPF program is linked to the VFS program to monitor input file events.

[0045] In this embodiment, the use of the eBPF program to monitor file events has better performance when dealing with a large number of file changes than the technical solution of using the inotify program to monitor file changes commonly used in the prior art.

[0046] In this embodiment, the eBPF program recursively obtains the file path of the file event relative to the mount point root directory through the in-kernel directory entry dentry, and then obtains the uuid and mount point path of the device where the file event is located through the device-related information in the directory entry dentry, and combines them to obtain the file path of the file event.

[0047] Step S103, in response to the file ID input by the external search application, searching the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID;

[0048] In this embodiment, the external search application is a search application running at the application layer. An index is pre-established, and the file ID is synchronized with the file ID in the data table. Its function is to search for files in the Linux file system and output the corresponding file ID based on the input search keyword. For example, when searching for "car" in the file name, the file IDs of several files with the keyword "car" are output.

[0049] Existing search applications generally use file paths as the search return results. Since this embodiment adds a file ID and file path conversion intermediate layer, external search applications can search and return file IDs instead of file paths, and then convert them into file paths based on the file ID through the intermediate layer of this embodiment, and then return the file path returned by this embodiment as the final search result. This solves the problem that existing search tools directly return file paths, but after the parent directory is renamed, the index is not updated in time, which easily leads to partial loss of search results.

[0050] Step S104: concatenate the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form the file path of the input file ID, and return it to the external search application.

[0051] In this embodiment, the file paths corresponding to the file IDs queried multiple times can be cached to improve performance.

[0052] For example, since this method determines the corresponding file path through the file ID and then returns it to the external search application, it is a management mechanism based on the file ID, which avoids the existing external search application, which is generally based on the index management mechanism of the file path. The index management mechanism based on the file path is prone to errors when dealing with situations such as the renaming of the file parent directory or the change of the file system mount point, which brings a low-quality search application experience to the user.

[0053] The file system management method based on the eBPF program disclosed by this method can provide high-quality support for external search applications, file monitoring applications, and metadata service applications.

[0054] In another embodiment of the present invention, Figure 3 As shown, the eBPF program is hooked with the VFS program in the kernel to monitor the input file events, and the monitored file events are also sent to an external file monitoring application.

[0055] Exemplarily, the external file monitoring application may be, for example, a data manager application.

[0056] In another embodiment of the present invention, Figure 4As shown, the method further includes: establishing a metadata database for the file, the metadata database including a file ID and metadata of the file corresponding to the file ID; in response to a file path input by an external metadata service application, using a stat function to obtain an inode number corresponding to the input file path, searching a data table to obtain a file ID corresponding to the inode number corresponding to the input file path, searching the metadata database based on the file ID corresponding to the input file path, obtaining corresponding metadata, and returning the metadata to the external metadata service application.

[0057] For example, the metadata database may include thumbnails of pictures and videos, summaries, contents, and tags of files, and the like.

[0058] In this embodiment, the file ID is used as a unique identifier, so that the external search application and the external metadata service application can be decoupled from the file path and do not directly depend on the file path, thereby improving the use effect of the application.

[0059] In this embodiment, the input file path is converted into a file ID, and then the file ID is used to obtain the corresponding metadata in the metadata database, avoiding the existing external metadata service application that directly uses the file path to obtain metadata, and is not updated in time when the parent folder is moved, which may cause metadata acquisition errors.

[0060] Based on the previous embodiment, in another embodiment of the present invention, the monitored file events are sent to the external file monitoring application, including: sending the monitored file events to the external file monitoring application with different permissions according to the different permissions of the external file monitoring application.

[0061] In this embodiment, an external file monitoring application can register a monitoring client by registering, and the eBPF program verifies the permissions of the registered client and sends the monitored file events that meet the client permissions to the corresponding monitoring client.

[0062] like Figure 5 As shown, a corresponding data table is created for each mount point in the operating system.

[0063] In this embodiment, the table name of the data table created by the mount point is the uuid of the corresponding mount point.

[0064] Since the directory structure is fixed when the operating system is installed, which is equivalent to the file path being fixed, the data table and the index of the external search application can be directly preset when the operating system is installed, reducing the time and system performance overhead caused by the need to initialize the index and traverse the files when installing the operating system. However, when installing the operating system, the distribution of inode numbers will be different due to different installation methods, so the inode number and the parent directory inode number in the preset data table are empty. Furthermore, when the operating system is started for the first time after installation, a one-time transition data table is preset. The file ID and the corresponding file path are preset in the one-time transition data table. According to the file path recorded in the one-time transition data table, the stat function is used to obtain the corresponding accurate inode number and parent directory inode number, and the obtained inode number and parent directory inode number are added to the preset data table. In this way, when installing the operating system, a data table including accurate inode number and parent directory inode number information is preset, avoiding the unnecessary time and performance overhead caused by completely re-establishing the data table when installing the operating system multiple times, thereby improving the efficiency of the method.

[0065] like Figure 6 As shown, the present invention also discloses a file system management device for eBPF program, including:

[0066] The data table creation module 601 is used to create a data table, which includes the file ID, inode number and parent directory inode number corresponding to all files;

[0067] The file monitoring and updating module 602 is used to link the eBPF program with the VFS program in the operating system kernel, monitor the input file events, and update the data table accordingly;

[0068] Search module 603, for searching the data table in response to a file ID input by an external search application, and determining the file inode number and parent directory inode number corresponding to the input file ID;

[0069] The result return module 604 is used to concatenate the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form the file path of the input file ID, and return it to the external search application.

[0070] The present invention also discloses an electronic device, such as Figure 7 As shown, an embodiment is disclosed, which is a block diagram of an electronic device suitable for the above-mentioned eBPF program-based file system management.

[0071] The electronic device 70 of this embodiment includes a processor 701, which can perform various appropriate actions and processes according to the program stored in the ROM 702 or the program loaded from the storage part 708 into the RAM 703. The processor 701 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor, etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present invention.

[0072] The RAM 703 stores various programs and data required for the operation of the electronic device 70. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the programs in the ROM 702 and / or RAM 703 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703, and the processor 701 may also execute the programs stored in one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0073] According to an embodiment of the present invention, the electronic device 70 may further include an I / O interface 705, which is also connected to the bus 704. The electronic device 70 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube, a liquid crystal display, and a speaker; a storage portion 708 including a hard disk; and a communication portion 709 including a network interface card such as a LAN card or a modem. The communication portion 709 performs communication processing via a network such as the Internet. A drive 7010 is also connected to the I / O interface 705 as needed. A removable medium 7011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 7010 as needed, so that a computer program read therefrom can be installed into the storage portion 708 as needed.

[0074] The present invention also provides a computer-readable storage medium.

[0075] The computer-readable storage medium may be included in the electronic device / device system described in the above embodiments, or may exist independently and not be incorporated into the electronic device / device. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0076] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0077] Embodiments of the present invention also include a computer program product.

[0078] The computer program product includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0079] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal over a network medium. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0080] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written by any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages. Programming languages ​​include, but are not limited to, Java, C++, Python, C language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of the present invention.

[0082] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described above separately, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A file system management method based on eBPF program, characterized in that: include: Create a data table, which includes the file ID, inode number and parent directory inode number corresponding to all files; Hook the eBPF program with the VFS program in the operating system kernel to monitor input file events and update the data table accordingly; In response to a file ID input by an external search application, searching the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID; The file path of the input file ID is formed by concatenating the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number, and returned to the external search application; The external search application is a search application that runs on the application layer. It pre-builds indexes, keeps the file IDs synchronized with the file IDs in the data table, and outputs the corresponding file IDs based on the search keywords entered. When installing the operating system, the data table and the index of the external search application are directly preset. When the operating system is started for the first time after installation, a one-time transition data table is preset. The file ID and the corresponding file path are preset in the one-time transition data table. According to the file path recorded in the one-time transition data table, the stat function is used to obtain the corresponding accurate inode number and parent directory inode number, and the obtained inode number and parent directory inode number are added to the preset data table.

2. A file system management method based on eBPF program according to claim 1, characterized in that: The eBPF program is hooked to the VFS program in the kernel to monitor the input file events, and the monitored file events are sent to an external file monitoring application.

3. The file system management method based on eBPF program according to claim 1, characterized in that: Establish a metadata database for the file, which includes the file ID and metadata of the file corresponding to the file ID; In response to the file path input by the external metadata service application, the stat function is used to obtain the inode number corresponding to the input file path, the data table is searched, and the file ID corresponding to the inode number corresponding to the input file path is obtained. According to the file ID corresponding to the input file path, the metadata database is searched to obtain the corresponding metadata, and the metadata is returned to the external metadata service application.

4. The file system management method based on eBPF program according to claim 2, characterized in that: The monitored file events are sent to the external file monitoring application, including: according to different permissions of the external file monitoring application, the monitored file events are sent to the external file monitoring application with different permissions.

5. The file system management method based on eBPF program according to claim 1, characterized in that: The method further includes: establishing a corresponding data table for each mount point in the operating system.

6. A file system management method based on eBPF program according to claim 5, characterized in that: The table name of the data table created by the mount point is the uuid of the corresponding mount point.

7. A file system management device based on an eBPF program, characterized by: The device includes: The data table creation module is used to create a data table, which includes the file ID, inode number and parent directory inode number corresponding to all files; The file monitoring and update module is used to hook the eBPF program with the VFS program in the operating system kernel, monitor the input file events, and update the data table accordingly; A search module for searching the data table in response to a file ID input by an external search application to determine a file inode number and a parent directory inode number corresponding to the input file ID; The result return module is used to splice the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form the file path of the input file ID, and return it to the external search application; the external search application is a search application running on the application layer, which pre-establishes an index, keeps the file ID synchronized with the file ID in the data table, and outputs the corresponding file ID according to the input search keyword; when installing the operating system, the data table and the index of the external search application are directly preset, and when the operating system is started for the first time after installation, a one-time transition data table is preset, and the file ID and the corresponding file path are preset in the one-time transition data table. According to the file path recorded in the one-time transition data table, the stat function is used to obtain the corresponding accurate inode number and parent directory inode number, and the obtained inode number and parent directory inode number are added to the preset data table.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images