File system management method and device based on eBPF program, equipment and product

Abstract

The invention discloses a file system management method and device based on an eBPF program, equipment and a product. The method comprises the steps that a data table is established; an eBPF program is hooked with a VFS program in an operating system core, an input file event is monitored, and a data table is correspondingly updated; and in response to a file ID input by the external search application program, searching the data table, splicing to form a file path of the input file ID, and returning to the external search application program. An eBPF program is hooked with a VFS program in an operating system core, an input file event is monitored, meanwhile, a data table is established, a file ID is used as a unique identifier of a file, a file path needed by an external search application program is searched, and the file path is searched. The method has the characteristics that the indexing accuracy is high, the file monitoring effect is good when the file changes frequently, and reliable file system management support can be provided for various external application programs.

Description

Technical Field

[0001] The present invention belongs to the field of file system management, and in particular relates to a file system management method, device, equipment and product based on an eBPF program. Background Art

[0002] Under the existing technical conditions, for the establishment of the existing file index, it is necessary to traverse the directory, and parse the files to generate the index during the traversal process. This process is very time-consuming and resource-intensive. Especially when the files in the computer system change frequently, the inotify program is usually used to monitor the file changes and update the index in real time. When the inotify program monitors a large number of file changes, there are obvious performance problems, which will affect the user experience when searching for files through the index. Summary of the Invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a file system management method, device, equipment and product based on an eBPF program.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows: In a first aspect, the present invention discloses a file system management method based on an eBPF program, which establishes a data table. The data table includes file IDs, inode numbers, and parent directory inode numbers corresponding to all files; Hook the eBPF program to the VFS program in the operating system kernel, monitor the input file events, and update the data table correspondingly; In response to the file ID input by an external search application program, search the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID; Concatenate the file paths recorded in the inodes corresponding to the file inode number and the parent directory inode number corresponding to the input file ID to form the file path of the input file ID, and return it to the external search application program.

[0005] In an embodiment of the present invention, hooking the eBPF program to the VFS program in the kernel and monitoring the input file events further includes: sending the monitored file events to an external file monitoring application program.

[0006] In an embodiment of the present invention, a metadata database of files is established. The metadata database includes file IDs and metadata of files corresponding to the file IDs; In response to the file path input by an external metadata service application, use the stat function to obtain the inode number corresponding to the input file path, search the data table to obtain the file ID corresponding to the inode number of the input file path, and based on the file ID corresponding to the input file path, search the metadata database to obtain the corresponding metadata, and return it to the external metadata service application.

[0007] In an embodiment of the present invention, sending the monitored file event to an external file monitoring application includes: according to different permissions of the external file monitoring application, sending the monitored file event to external file monitoring applications with different permissions.

[0008] In an embodiment of the present invention, the method further includes: establishing a corresponding data table for each mount point in the operating system.

[0009] In an embodiment of the present invention, the table name of the data table established by the mount point is the uuid of the corresponding mount point.

[0010] In a second aspect, the present invention discloses a file system management device based on an eBPF program, and the device includes: A data table establishment module, configured to establish a data table, where the data table includes file IDs, inode numbers, and parent directory inode numbers corresponding to all files; A file monitoring and updating module, configured to hook the eBPF program to the VFS program in the operating system kernel, monitor the input file events, and update the data table correspondingly; A search module, configured to, in response to a file ID input by an external search application, search the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID; A result return module, configured to splice the file paths recorded in the inodes corresponding to the file inode number and the parent directory inode number of the input file ID to form the file path of the input file ID, and return it to the external search application.

[0011] In a third aspect, the present invention discloses an electronic device, including: one or more processors; a storage device, configured to store one or more programs, where when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute any one of the above methods.

[0012] In a fourth aspect, the present invention discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0013] In a fifth aspect, the present invention discloses a computer program product, including a computer program which, when executed by a processor, implements the above method.

[0014] Compared with the prior art, the present invention has the following advantages: The present invention discloses a file system management method, apparatus, device, and product based on an eBPF program, including establishing a data table; hooking the eBPF program to the VFS program in the operating system kernel to listen for input file events and correspondingly update the data table; in response to a file ID input by an external search application, searching the data table; concatenating the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the parent directory inode number to form the file path of the input file ID, and returning it to the external search application; the present invention discloses a file system management method, apparatus, device, and product based on an eBPF program, which can hook the eBPF program to the VFS program in the operating system kernel to listen for input file events, and at the same time establish a data table, use the file ID as the unique identifier of the file, search for the file path required by the external search application, and has the characteristics of high indexing accuracy, good file listening effect when files change frequently, and can provide reliable file system management support for various external application programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0016] In the drawings: Figure 1 is a schematic diagram of a file system management method based on an eBPF program according to an embodiment of the present invention; Figure 2 is a schematic diagram of a file system management method based on an eBPF program according to an embodiment of the present invention supporting an external search application; Figure 3 is a schematic diagram of a file system management method based on an eBPF program according to an embodiment of the present invention supporting an external file listening application; Figure 4 is a schematic diagram of a file system management method based on an eBPF program according to an embodiment of the present invention supporting an external metadata service application; Figure 5 is a schematic diagram of a data table for establishing a mount point in a file system management method based on an eBPF program according to an embodiment of the present invention; Figure 6 is a schematic diagram of a file system management apparatus based on an eBPF program according to an embodiment of the present invention; Figure 7 Schematic diagram of an electronic device for file system management based on eBPF program in an embodiment of the present invention. Detailed implementation manners

[0017] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0018] In the description of the present invention, it should be noted that, unless otherwise clearly defined and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0019] In the description of the present invention, it should be further noted that the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0020] Under the existing technical conditions, the inotify program is usually used to monitor file changes and update the index in real time. When the inotify program monitors a large number of file changes, there are obvious performance problems, which will affect the user experience when searching for files through the index. The present invention discloses a file system management method, device, equipment, and product based on the eBPF program, which hooks the eBPF program with the VFS program in the operating system kernel, monitors the input file events, and simultaneously establishes a data table, uses the file ID as the unique identifier of the file, and searches for the file paths required by external search applications. It has the advantages of high index accuracy, good file monitoring effect when files change frequently, and providing reliable file system management support for various external applications.

[0021] The present invention will be described in detail below with reference to the drawings and in combination with embodiments.

[0022] As Figure 1 and 2 shown, in an embodiment of the present invention, a file system management method based on the eBPF program includes: Step S101, establish a data table, where the data table includes the file ID, inode number, and parent directory inode number corresponding to all files; In this embodiment, the file ID is a natural number that increases for each piece of data when the data table is established.

[0023] In another embodiment, the data table further includes the name of the file; Step S102: Hook the eBPF program to the VFS program in the operating system kernel, listen for the input file events, and update the data table accordingly; In this embodiment, the eBPF program and the VFS program run in the linux kernel layer. The role of the VFS program is to serve as a unified interface for file operations in the linux operating system. The role of the eBPF program is to listen for file events. The eBPF program is hooked to the VFS program to listen for the input file events.

[0024] In this embodiment, using the eBPF program to listen for file events has better performance effects when dealing with a large number of file changes compared to the prior art solution that generally uses the inotify program to listen for file changes.

[0025] In this embodiment, the eBPF program recursively obtains the file path of the file event relative to the root directory of the mount point through the in-kernel directory entry dentry, and then obtains the uuid and mount point path of the device where the file event is located through the device-related information in the directory entry dentry, and combines them to obtain the file path of the file event.

[0026] Step S103: In response to the file ID input by an external search application, search the data table to determine the file inode number and the parent directory inode number corresponding to the input file ID; In this embodiment, the external search application is a search application running in the application layer. An index is established in advance, and the file ID is synchronized with the file ID in the data table. Its role is to perform file search in the linux file system. According to the input search keyword, the corresponding file ID is output. For example, when searching for "car" in the file name, several file IDs with the keyword "car" in the file name are output.

[0027] Existing search applications generally use the file path as the search return result. In this embodiment, due to the addition of the intermediate layer for file ID and file path conversion, the external search application can search and return the file ID instead of the file path, and then based on the file ID, it is converted into the file path through the intermediate layer of this embodiment, and then the file path returned by this embodiment is returned as the final search result, solving the problem that existing search tools directly return the file path, but after the parent directory is renamed, due to the failure to update the index in time, it is easy to cause partial loss of search results.

[0028] Step S104: Concatenate the file inode number corresponding to the input file ID and the file path recorded in the inode corresponding to the inode number of the parent directory, and return the file path of the input file ID to the external search application program.

[0029] In this embodiment, the file paths corresponding to multiple queried file IDs can be cached to improve performance.

[0030] Exemplarily, since this method determines the corresponding file path through the file ID and then returns it to the external search application program, it is a management mechanism based on the file ID, avoiding the existing external search application programs, which generally rely on the index management mechanism based on the file path. The index management mechanism based on the file path is prone to errors when dealing with situations such as the renaming of the file parent directory or the change of the file system mount point, bringing a low-quality search application usage experience to users.

[0031] The file system management method based on the eBPF program disclosed in this method can provide high-quality support for external search application programs, file listening application programs, and metadata service application programs.

[0032] In another embodiment of the present invention, as Figure 3 shown, hook the eBPF program to the in-kernel VFS program to monitor the input file events, and further include: sending the monitored file events to the external file listening application program.

[0033] Exemplarily, the external file listening application program can be, for example, a data steward application program.

[0034] In another embodiment of the present invention, as Figure 4 shown, this method further includes: establishing a metadata database for files, where the metadata database includes file IDs and the metadata of the files corresponding to the file IDs; in response to the file path input by the external metadata service application program, use the stat function to obtain the inode number corresponding to the input file path, search the data table to obtain the file ID corresponding to the inode number of the input file path, and search the metadata database according to the file ID corresponding to the input file path to obtain the corresponding metadata, and return it to the external metadata service application program.

[0035] Exemplarily, the metadata database can be thumbnails of pictures and videos, abstracts, contents, and tags of files, etc.

[0036] In this embodiment, using the file ID as the unique identifier enables both the external search application program and the external metadata service application program to be decoupled from the file path and not directly rely on the file path, improving the usage effect of the application program.

[0037] In this embodiment, the input file path is converted into a file ID, and then the file ID is used to obtain the corresponding metadata in the metadata database, avoiding the possible metadata acquisition errors that may occur when the existing external metadata service application directly uses the file path to obtain metadata and fails to update in a timely manner when the parent folder is moved.

[0038] Based on the previous embodiment, in another embodiment of the present invention, the monitored file events are sent to an external file monitoring application, including: according to different permissions of the external file monitoring application, the monitored file events are sent to external file monitoring applications with different permissions.

[0039] In this embodiment, the external file monitoring application can register a monitoring client through registration. The eBPF program verifies the permissions of the registered client and sends the monitored file events that meet the client permissions to the corresponding monitoring client.

[0040] Such as Figure 5 shown, a corresponding data table is established for each mount point in the operating system.

[0041] In this embodiment, the table name of the data table established by the mount point is the uuid of the corresponding mount point.

[0042] Since the directory structure is fixed during the installation of the operating system, which is equivalent to the file path of the file being fixed, the data table and the index of the external search application can be directly preset during the installation of the operating system, reducing the time and system performance overhead caused by the need to initialize the index and traverse the files during the installation of the operating system. However, due to different installation methods, the distribution of inode numbers will be different during the installation of the operating system. Therefore, the inode numbers and the inode numbers of the parent directories in the preset data table are empty. Further, when the operating system is started for the first time after installation, a one-time transition data table is preset. The one-time transition data table presets the file ID and the corresponding file path. According to the file path recorded in the one-time transition data table, the corresponding accurate inode number and the inode number of the parent directory are obtained by using the stat function, and the obtained inode numbers and the inode numbers of the parent directory are filled into the preset data table. In this way, a data table including accurate inode numbers and inode number information of the parent directory is preset during the installation of the operating system, avoiding the unnecessary time and performance overhead caused by completely rebuilding the data table during multiple installations of the operating system and improving the usage efficiency of the method of the present invention.

[0043] Such as Figure 6 shown, the present invention also discloses a file system management device based on an eBPF program, including: The data table creation module 601 is used to create a data table, which includes the file ID, inode number, and parent directory inode number corresponding to all files; The file monitoring and updating module 602 is used to hook the eBPF program to the VFS program in the operating system kernel, monitor the input file events, and update the data table accordingly; The search module 603 is used to search the data table in response to the file ID input by an external search application, and determine the file inode number and parent directory inode number corresponding to the input file ID; The result return module 604 is used to splice the file paths recorded in the inodes corresponding to the file inode number and parent directory inode number of the input file ID to form the file path of the input file ID, and return it to the external search application.

[0044] The present invention also discloses an electronic device, as Figure 7 shown, a block diagram of an embodiment of an electronic device applicable to the above file system management based on the eBPF program is disclosed.

[0045] The electronic device 70 in this embodiment includes a processor 701, which can perform various appropriate actions and processes according to the program stored in the ROM 702 or the program loaded from the storage section 708 into the RAM 703. The processor 701 can include, for example, a general microprocessor, an instruction set processor, and / or a related chipset and / or a dedicated microprocessor, etc. The processor 701 can also include on-board memory for caching purposes. The processor 701 can include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present invention.

[0046] In the RAM 703, various programs and data required for the operation of the electronic device 70 are stored. The processor 701, ROM 702, and RAM 703 are connected to each other through a bus 704, and the processor 701 performs various operations of the method flow according to the embodiments of the present invention by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than the ROM 702 and RAM 703, and the processor 701 can also perform various operations of the method flow according to the embodiments of the present invention by executing the programs stored in one or more memories.

[0047] According to an embodiment of the present invention, the electronic device 70 may further include an I / O interface 705, and the I / O interface 705 is also connected to the bus 704. The electronic device 70 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube, a liquid crystal display, a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. The drive 7010 is also connected to the I / O interface 705 as needed. A removable medium 7011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 7010 as needed so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0048] The present invention also provides a computer-readable storage medium.

[0049] The computer-readable storage medium may be included in the electronic device / apparatus system described in the above embodiments; or it may exist separately without being assembled into the electronic device / apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0050] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory RAM, a read-only memory ROM, an erasable programmable read-only memory EPROM or a flash memory, a portable compact disk read-only memory CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0051] An embodiment of the present invention further includes a computer program product.

[0052] The computer program product includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present invention. When the computer program product runs on an electronic device, the program code is used to cause the electronic device to implement the method provided by the embodiments of the present invention.

[0053] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium. The program code included in the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0054] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedures and / or object-oriented programming languages. Programming languages include but are not limited to, such as Java, C++, python, C language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0055] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented using a dedicated hardware-based system for performing the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions. Those skilled in the art can understand that the features described in various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in various embodiments and / or claims of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0056] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present invention is defined by the appended claims and their equivalents, and without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A file system management method based on eBPF programs, characterized in that, including: establish a data table, which includes the file ID, inode number, and parent directory inode number corresponding to all files; hook the eBPF program to the VFS program in the operating system kernel to monitor the input file events and update the data table accordingly; in response to the file ID input by an external search application, search the data table to determine the file inode number and parent directory inode number corresponding to the input file ID; concatenate the file path recorded in the inode corresponding to the file inode number and parent directory inode number of the input file ID to form the file path of the input file ID, and return it to the external search application.

2. The file system management method based on eBPF programs according to claim 1, characterized in that, hooking the eBPF program to the in-kernel VFS program to monitor the input file events further includes: sending the monitored file events to an external file monitoring application.

3. The file system management method based on eBPF programs according to claim 1, characterized in that, establish a metadata database for files, which includes file IDs and the metadata of the files corresponding to the file IDs; in response to the file path input by an external metadata service application, use the stat function to obtain the inode number corresponding to the input file path, search the data table to obtain the file ID corresponding to the inode number of the input file path, and search the metadata database according to the file ID corresponding to the input file path to obtain the corresponding metadata, and return it to the external metadata service application.

4. The file system management method based on eBPF programs according to claim 2, characterized in that, sending the monitored file events to an external file monitoring application includes: sending the monitored file events to external file monitoring applications with different permissions according to the different permissions of the external file monitoring application.

5. The file system management method based on eBPF programs according to claim 1, characterized in that, The method further includes: establishing a corresponding data table for each mount point in the operating system.

6. The file system management method based on eBPF programs according to claim 5, characterized in that, The table name of the data table established for the mount point is the uuid of the corresponding mount point.

7. A file system management device based on eBPF programs, characterized in that: The apparatus includes: a data table establishment module for establishing a data table, which includes the file ID, inode number, and parent directory inode number corresponding to all files; a file monitoring and updating module for hooking the eBPF program to the VFS program in the operating system kernel to monitor the input file events and update the data table accordingly; a search module for searching the data table in response to the file ID input by an external search application to determine the file inode number and parent directory inode number corresponding to the input file ID; a result return module for concatenating the file path recorded in the inode corresponding to the file inode number and parent directory inode number of the input file ID to form the file path of the input file ID, and returning it to the external search application.

8. An electronic device, characterized in that, including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that , when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6.

10. A computer program product, including a computer program, characterized in that, , when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6.

Images