File system implementation method and system

Abstract

A file system implementation method and system are disclosed. A first process and at least one second process are run in a user state space, and the first process is used to interact with a kernel to implement a file management function. The second process communicates with the first process by using a shared memory as a channel and transmits a file access request to the first process to access a file. The file access request includes file information of the file requested to be accessed by the second process. The first process writes processing result information of the corresponding file access request in the shared memory. The shared memory can be divided into a plurality of request processing units, each of which is used for a file access request. In this way, the performance of the file system can be improved, and the problem that multiple processes need to share the file system status and resources can also be solved.

Description

Technical Field

[0001] The present disclosure relates to a file system implemented based on a user-mode space, and more particularly to a method and system for implementing a file system based on a user-mode space. Background Art

[0002] The File System in User Space (FUSE) is a file system framework implemented in user space. The file system data and metadata are provided by user-space processes.

[0003] There are currently two ways to implement a user-mode file system. The first is to implement it based on the FUSE kernel module, and the second is to write programs in user mode to manage the disk yourself, thereby realizing the function of the file system.

[0004] Most common user-mode file systems are implemented in the first way, that is, based on the FUSE kernel module. With the support of the FUSE kernel module, users only need to implement specific file operations according to the interface provided by FUSE to implement a file system, and multiple processes can share the state resources of the file system. However, when implementing a user-mode file system based on the FUSE kernel module, due to the inherent design of the FUSE mechanism, the performance of the user-mode file system implemented using the FUSE kernel module is destined to be poor. In the scenario where the upper layer is a database, it cannot meet the performance requirements.

[0005] If we bypass the FUSE kernel module and do not use the FUSE mechanism, and adopt the second method, that is, the user-mode process manages the disk by itself to implement the file system, it is certainly within an independent process, and the relevant functions of the file system can be implemented, which can solve the problem of poor performance. However, when multiple processes need to share the state and resources of the same user-mode file system, this solution cannot meet the requirements. For example, when PostgreSQL (an open source object-relational database management system) uses a user-mode file system, it faces the problem that multiple processes need to share the state and resources of the file system.

[0006] Therefore, there is still a need for an improved file system based on user-mode space implementation to solve at least one of the above problems. Summary of the invention

[0007] A technical problem to be solved by the present disclosure is to provide a file system implementation solution based on user-mode space, which can solve at least one of the above problems.

[0008] According to a first aspect of the present disclosure, a file system implementation method is provided, comprising: running a first process in a user state space, the first process being used to interact with a kernel to implement a file management function; running at least one second process in the user state space; the second process communicating with the first process by using a shared memory as a channel and delivering a file access request to the first process to implement access to a file, wherein the file access request includes file information of the file requested to be accessed by the second process; and the first process writing processing result information of the corresponding file access request in the shared memory.

[0009] Optionally, the shared memory is divided into a plurality of request processing units, each request processing unit being used for a file access request.

[0010] Optionally, each request processing unit includes a data cache block for a file access request, and is assigned a corresponding request information field and feedback information field. The data cache block is used to cache data that the second process requests the first process to write to the disk or data that the first process reads from the disk in response to the file access request of the second process. The request information field is used for the second process to write request information related to the file access request, and the feedback information field is used for the first process to write execution feedback information of the file access request.

[0011] Optionally, the shared memory is divided into at least one block, each block is used to process file access requests of different size ranges; each block is divided into at least one channel, each channel includes multiple request processing units, and the request processing unit is suitable for processing file access requests corresponding to the size range processed by the block to which it belongs.

[0012] Optionally, each channel may include: a channel header, including a size field and an idle state bitmap of a request processing unit, the size field recording the size range of file access requests that the channel is used to process, and the idle state bitmap indicating whether the multiple request processing units included in the channel are in an idle state or an occupied state; a request information array, including request information fields of multiple request processing units; a feedback information array, including feedback information fields of multiple request processing units; and a data cache area, including data cache blocks of multiple request processing units.

[0013] Optionally, the method may also include: maintaining current status information of the request processing unit in association with the request processing unit, the status of the request processing unit including at least one of the following: an idle state, the request processing unit has not yet been assigned to a file access request; the request processing unit has been acquired by the second process, but the second process has not yet filled the file access request to the request processing unit; the second process has filled the file access request to the request processing unit and is waiting for a reply from the first process; the first process has acquired the file access request of the second process and is processing it; and the first process has completed processing the file access request.

[0014] Optionally, the method may also include: the second process creates a mount information file in a specified folder and writes mount parameters for the disk; the first process senses the existence of the mount information file, retrieves the mount parameters from the mount information file, and interacts with the kernel to perform a disk mount operation in the file system; after the disk mount operation is successful, the first process assigns a connection identifier to the second process, and writes the mount result information into the mount information file; the second process obtains the mount result and connection identifier from the mount information file, and carries the connection identifier in a subsequent file access request to the disk.

[0015] Optionally, the second process applies a file lock to the mount information file it created, and maintains the file lock during the life of the second process, wherein only one file lock can be applied to a mount information file, and the method further includes: when the second process actively issues a disk unmount instruction, the second process releases the file lock, and when the first process senses through the inotify mechanism that the mount information file is closed and the file lock is released, the first process obtains mount parameters and connection identifiers, and after the file access request being executed is processed, interacts with the kernel to perform a disk unmount operation in the file system; and / or when the second process fails unexpectedly, the operating system closes the mount information file and releases the file lock, the first process attempts to add a file lock to the mount information file, and when the file lock can be added to the mount information file, the first process obtains the mount parameters and connection identifiers, and after the file access request being executed is processed, interacts with the kernel to perform a disk unmount operation in the file system.

[0016] According to a second aspect of the present disclosure, a system for implementing a file system is provided, comprising: a first device, for running a first process in a user state space, the first process being used to interact with a kernel state to implement a file system management function; a second device, for running at least one second process in the user state space; and a shared memory, serving as a communication channel between the second process and the first process, wherein the second process communicates with the first process via the shared memory, and passes a file access request to the first process to implement access to the file, the file access request including file information of the file requested to be accessed by the second process, and the first process writes processing result information of the corresponding file access request in the shared memory.

[0017] According to a third aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable codes are stored, and when the executable codes are executed by the processor, the processor executes the method described in the first aspect above.

[0018] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising an executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described in the first aspect above.

[0019] According to a fifth aspect of the present disclosure, a non-temporary machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first aspect above.

[0020] Therefore, the present invention solves the aforementioned problem of balancing performance and file system status resources, and after improving the basic capabilities of the file system, it can be used in a variety of databases and a variety of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0022] Figure 1 The system architecture of a file system implemented in one embodiment of the present disclosure is schematically shown.

[0023] Figure 2 It is a schematic flow chart of a method for implementing a file system according to an embodiment of the present disclosure.

[0024] Figure 3 is a schematic block diagram of a system for implementing a file system according to an embodiment of the present disclosure.

[0025] Figure 4The overall layout of the shared memory according to the embodiment of the present disclosure is schematically shown.

[0026] Figure 5 is a schematic flowchart of a disk mounting operation according to an embodiment of the present disclosure.

[0027] Figure 6 is a schematic flowchart of a disk unloading operation performed under normal circumstances according to an embodiment of the present disclosure.

[0028] Figure 7 is a schematic flowchart of a disk unloading operation performed under abnormal circumstances according to an embodiment of the present disclosure.

[0029] Figure 8 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned file system implementation method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] First, refer to Figure 1 and Figure 2 , describes a file system implementation scheme according to an embodiment of the present disclosure.

[0032] Figure 1 The system architecture of a file system implemented in an embodiment of the present disclosure is schematically shown. The file system is implemented based on the user space, so it can also be called a "user file system" or an "improved user file system", but it is different from the existing user file system architecture.

[0033] The kernel 140 running in the kernel space can control and manage multiple disks 150 (storage disks, generally, magnetic disks), and implement file system operations such as reading, writing, opening, and closing files on the disks.

[0034] The independent process 110 (hereinafter referred to as the "first process") running in the user space can interact with the kernel 140 to manage the disk 150 and implement the relevant logic of the file system.

[0035] The user space also runs at least one upper-level process 120 (hereinafter referred to as a "second process", which may be an application process or a user process, for example), which may be used to implement corresponding application functions or user-defined functions.

[0036] The first process 110 and the second process 120 communicate by using the shared memory 130 as a channel. Furthermore, the first process 110 interacts with the kernel 140, thereby achieving access to the file requested by the first process 110.

[0037] Figure 2 It is a schematic flow chart of a method for implementing a file system according to an embodiment of the present disclosure.

[0038] like Figure 2 As shown, in step S210, the first process 110 is run in the user state space, and the first process 110 can interact with the kernel 140 to implement the file management function.

[0039] In step S220, the second process 120 is run in the user state space.

[0040] In step S230, at least one second process 120 communicates with the first process 110 by using the shared memory 130 as a channel, and transmits a file access request to the first process 110 to access the same file system. Here, the file access request may include file information of the file requested to be accessed by the second process 120, for example, data that may be used to describe which file is to be accessed, and further, which location or location range of the file is to be accessed.

[0041] In step S240, the first process 110 writes the processing result information of the corresponding file access request into the shared memory 130. The processing result may include, for example, whether the processing is successful, and the data read from the file on the disk.

[0042] Figure 3 is a schematic block diagram of a system that can be used to implement a file system according to an embodiment of the present disclosure.

[0043] like Figure 3 As shown, the system may include a first device 310 and at least one second device 320 .

[0044] The first device 310 is used to run the first process 110 in the user state space, and the second device 320 is used to run the second process 120 in the user state space.

[0045] As described above, the system further includes a shared memory 130. The first device 310 and the second device 320 are both connected to the shared memory 130, so that the first process 110 and the second process 120 communicate through the shared memory 130.

[0046] It should be understood that Figure 3 The second devices 320 respectively used to run different second processes 120 may be the same device or different devices.

[0047] Likewise, the first device 310 running the first process 110 and the second device 320 running part or all of the second process 120 may be the same device or different devices.

[0048] The device here may be, for example, a computing device such as a computer or a server.

[0049] Next, how the upper layer second process 120 communicates with the first process 110 through the shared memory 130 is further described in detail.

[0050] First, the overall layout that the shared memory 130 may adopt is described in detail.

[0051] The shared memory 130 may be divided into a plurality of request processing units, each of which may be used to process a file access request.

[0052] Each request processing unit may include a data cache block for a file access request. The data cache block may be used to cache data that the second process 120 requests the first process 110 to write to the disk or data that the first process 110 reads from the disk in response to the file access request of the second process 120.

[0053] In addition, each request processing unit may be allocated a corresponding request information field and feedback information field.

[0054] The request information field may be used, for example, for the second process 120 to write request information related to the file access request, and the feedback information field may be used, for example, for the first process 110 to write execution feedback information on the file access request.

[0055] Figure 4 The overall layout of the shared memory 130 according to the embodiment of the present disclosure is schematically shown.

[0056] like Figure 4 As shown, the shared memory 130 may be divided into at least one block (also referred to as a "shared memory block"), each shared memory block being used to process file access requests of different size ranges.

[0057] For example, a whole block of shared memory 130, such as 500M, 1G or 2G shared memory, can be divided into multiple (N, N is a positive integer) blocks of different types according to the size range of the specified IO size (the size of the data requested to be accessed by the file access request). For example, shared memory block 1, shared memory block 2, ..., shared memory block N shown in the figure can be set to process file access requests with IO size ranges within 4k, within 8k, within 16k, ... respectively. In other words, in each shared memory block, the size of the request processing unit allocated to each file access request can correspond to the upper limit of the IO size range it processes, that is, 4k, 8k, 16k, ... respectively.

[0058] Each shared memory block may have a header, which may have a fixed size of 4KB, for example. The shared memory block header may include, for example, a channel number field nch and a processed IO size range field req_size. Figure 4 As shown, the number of channels of shared memory block 1 is 2, that is, nch=2, and the processed IO size range is within 16k, that is, req_size=16k.

[0059] Accordingly, except for the header, each shared memory block can be divided into at least one (ie, nch, nch is a positive integer) channel, such as channel 1 and channel 2.

[0060] Each shared memory block may have the same number of channels, for example, 64, or may have a different number of channels.

[0061] Each channel may include a plurality of request processing units. The request processing unit is adapted to process file access requests corresponding to the size range of the block processing to which it belongs. In other words, data cache space corresponding to the upper limit of the size range of the block processing to which it belongs may be allocated to each request processing unit.

[0062] In this way, by introducing channels, the concurrency of file access operations can be improved.

[0063] like Figure 4 As shown, each channel may include: a channel header, a request information array, a feedback information array, and a data buffer area.

[0064] The channel header may include, for example, a size field req_size and a free status bitmap (freebitmap) of the request processing unit.

[0065] The size field records the size range of the file access request processed by this channel. For example, req_size=16k indicates that the size range of the file access request processed by this channel is within 16k.

[0066] The idle state bitmap indicates whether the multiple request processing units included in the channel are in an idle state or an occupied state.

[0067] The idle state bitmap may include, for example, m bits, where m is the number of request processing units included in the channel, and m is a positive integer.

[0068] For example, the idle state bitmap may include 64 bits, i.e., the word length of a long integer variable in a 64-bit system. Thus, a long integer variable may be used as the idle state bitmap.

[0069] The m bits of the idle state bitmap correspond to m request processing units respectively. When one of the bits is set to a predetermined value, such as "0", it indicates that the request processing unit corresponding to the bit is in an idle state, has not been occupied by the second process 120, and can be applied for use. On the contrary, when one of the bits is set to a predetermined value, such as "1", it indicates that the request processing unit corresponding to the bit is in an occupied state, has been occupied by a second process 120, and can no longer be applied for use.

[0070] The shared memory 130 may not need to record which second process 120 occupies which bit. The second process 120 may record which request processing unit is occupied by itself, and then write data or status information to the corresponding request processing unit, or read data or status information.

[0071] The request information array may include multiple (m) request information fields of the request processing units. In other words, the multiple elements of the request information array may be request information fields of multiple request processing units. For example, the request information field may include the type of access operation requested to be performed (e.g., read, write, file open, file close), the disk requested to be accessed, the file name of the file requested to be accessed, the location of the data requested to be accessed in the file, the current request processing status, etc.

[0072] The feedback information array may include feedback information fields of multiple (m) request processing units. In other words, multiple elements of the feedback information array may be feedback information fields of multiple request processing units. For example, the feedback information field may include whether the access operation (read / write / file open / file close) is successful, how much data is written or read, etc.

[0073] The data cache area includes multiple (m) data cache blocks of the request processing unit. The size of each data cache block corresponds to the upper limit of the IO size range processed by the shared memory block. For example, Figure 4 Channel 1 of the shared memory block 1 shown in detail in FIG. 1 includes 64 16k data cache blocks.

[0074] The data cache blocks of each request processing unit can be used to cache the data of the file requested to be written into the disk by the second process 120 occupying the request processing unit. Accordingly, the first process 110 can read the data from the data cache block and interact with the kernel 140 to write the data to the desired location in the desired file on the desired disk, thereby implementing the data writing operation of the second process 120 on the file in the disk.

[0075] Alternatively, the data cache blocks of each request processing unit can be used to cache data read from a specified location of a specified file in a specified disk by the first process 110 in response to a read request of the second process 120 occupying the request processing unit, interacting with the kernel 140. Accordingly, the second process 120 can read data from the data cache block, thereby implementing the data reading operation of the second process 120 on the file in the disk.

[0076] The m bits in the idle state bitmap correspond to the m elements of the request information array, the m elements of the feedback information array, and the m data cache blocks of the data cache area. The corresponding elements of the request information array and the feedback information array and the corresponding data cache blocks of the data cache area are managed by setting "0" or "1" to the corresponding positions in the idle state bitmap.

[0077] Thus, when a second process 120 needs to access a file on the disk, it can first query the idle state bitmap to request a request processing unit that has not been occupied, and set the corresponding position of the request processing unit in the idle state bitmap to indicate that the request processing unit has been occupied. Then, the second process 120 can write the type of access operation requested to be performed and the disk, file, location, etc. to be accessed in the request information field corresponding to the request processing unit. When the request is to perform a write operation, the second process 120 can also write the data expected to be written in the file on the disk into the data cache block corresponding to the request processing unit.

[0078] After the first process 110 senses that a certain request processing unit is occupied, it obtains the request information in the corresponding request information field, interacts with the kernel 140, performs the corresponding write operation, and then writes the execution result feedback information in the corresponding feedback information field. When the request is to perform a write operation, the first process 110 reads data from the corresponding data cache block, interacts with the kernel 140, writes the data to the specified location of the specified file on the specified disk, and implements the data write request operation of the second process 120.

[0079] When the request is for a read operation, the first process 110 can obtain the disk, file and location information desired to be read from the request information field, interact with the kernel 140 to read the corresponding data, and write the data read from the specified location in the file on the disk into the data cache block corresponding to the request processing unit. The second process 120 can read the data written by the first process 110 from the corresponding data cache block, thereby implementing the data read request operation of the second process 120.

[0080] As described above, the upper layer multiple second processes 120 and the first process 110 that independently manages the disk to implement the file system function are all performing read and write access to the shared memory 130. In order to ensure the correctness of input / output (IO), a CAS mechanism (Compare And Swap, an atomic operation mechanism) can be used to mark the status of the request processing unit on the shared memory 130, thereby ensuring the correctness of input / output (IO).

[0081] The current status information of the request processing unit may be maintained in association with the request processing unit, for example, may be recorded in a request information field corresponding to the request processing unit.

[0082] The status of the request processing unit may include, for example, at least one of the following:

[0083] REQ_FREE=0:initial idle state, the request processing unit has not been assigned to a file access request, and the request can be assigned for use;

[0084] REQ_ALLOC=1: the request processing unit has been acquired by the second process 120 of the upper layer, but the second process 120 has not yet filled the request processing unit with a file access request;

[0085] REQ_WAIT_REPLY=2: the second process 120 has already filled the file access request to the request processing unit and is waiting for a reply from the first process 110;

[0086] REQ_IN_PROGRESS=3: the first process 110 has obtained the file access request of the second process 120 and is processing it;

[0087] REQ_WAIT_RELEASE=4: The first process has completed processing the file access request.

[0088] The above describes the solution of the present disclosure that the second thread 120 accesses the file on the disk by using the shared memory 130 as a communication channel to communicate between the second thread 120 and the first thread 110 .

[0089] The following describes the process of mounting / unmounting the disk before executing the second thread's access operation to the file on the disk.

[0090] The file system often manages multiple disks, and the kernel may not be able to connect to all the disks that need to be managed at the same time. Therefore, disk mounting (mount) and unmounting (umount) operations are required to mount the disks that need to be accessed and unmount the disks that are no longer needed.

[0091] The second process 120 needs to inform the first process 110 which disk needs to be accessed. The first process 110 interacts with the kernel 140 to perform disk mount initialization operations and authenticate the second process 120.

[0092] Shared memory lacks a notification mechanism, and the processing of mounting in multi-process mode is relatively complicated. The present disclosure uses inotify (a framework for discovering file system changes in Linux) as the basic communication mechanism, based on which the first process 110 can perceive the upper layer second process 120, establish a "connection", and perform a disk mounting operation to establish a "logical connection".

[0093] Figure 5 is a schematic flowchart of a disk mounting operation according to an embodiment of the present disclosure.

[0094] like Figure 5 As shown, in step S510, the second process 120 creates a mount information file in a designated folder and writes mount parameters for the disk.

[0095] The designated folder may be a folder managed by the system's own file management system. The mounting parameters may include, for example, information of the disk to be accessed, such as disk ID information, and permissions for accessing the disk this time.

[0096] The second process 120 can lock the mount information file it created. On the one hand, using the file lock can prevent other processes from using the file. On the other hand, the first process 110 can verify whether the file is released by checking whether the file is locked.

[0097] In step S520, the first process 110 may sense the existence of the mount information file, for example, through the inotify mechanism, and retrieve the mount parameters from the mount information file, and interact with the kernel to perform a disk mount operation in the file system.

[0098] In step S530, after the disk mount operation is successful, the first process 110 allocates a connection identifier ConnectID to the second process 120, and writes the connection identifier and the mount result information into the mount information file. The connection identifier is equivalent to the disk access credential issued to the second process 120. The mount result information may include, for example, whether the mount operation is successful.

[0099] In step S540, the second process 120 obtains the mount result and the connection identifier from the mount information file, and carries the connection identifier in the subsequent file access request to the file on the disk. In this way, the second process 120 can inform the first process 110 that it will access and has the right to access the disk.

[0100] Thus, the disk mounting operation is realized.

[0101] Below, reference Figure 6 and 7 Describes the disk unmount operation.

[0102] Figure 6 is a schematic flowchart of a disk unloading operation performed under normal circumstances according to an embodiment of the present disclosure.

[0103] As described above, the second process 120 locks the mount information file it creates and keeps the file lock during the life of the second process 120. Only when the second process 120 actively performs a disk unloading operation (actively issues a disk unloading instruction) will the second process 120 release the file lock.

[0104] Here, only one file lock can be set for a mount information file. That is, at the same time, a mount information file can only be occupied by one process (the second process 120 or the first process 110).

[0105] like Figure 6 As shown, in step S610, the second process 120 actively issues a disk unloading instruction.

[0106] Accordingly, in step S620, the second process 120 releases the file lock.

[0107] In step S630, the first process 110 senses through the inotify mechanism that the mount information file is closed and the file lock is released.

[0108] In this case, in step S640, the first process 110 obtains the mount parameters and the connection identifier from the mount information file.

[0109] Then, in step S650, after the processing of the file access request being executed (the request carries a connection identifier, which can identify whether it is an access request to the disk) is completed, interaction with the kernel is performed to perform a disk unmounting operation in the file system.

[0110] Thereby, the disk unloading operation under normal circumstances is realized.

[0111] Figure 7 is a schematic flowchart of a disk unloading operation performed under abnormal circumstances according to an embodiment of the present disclosure.

[0112] As described above, the upper-layer second process 120 always locks the disk mount file during its existence, and the child process created (forked) by the second process 120 itself will also inherit this file lock.

[0113] like Figure 7 As shown, in the case where the second process 120 unexpectedly fails (crashes), in step S710, the operating system closes the mount information file and releases the file lock.

[0114] At this time, the first process 110 cannot sense that the file is closed and released through the inotify mechanism.

[0115] In step S720 , the first process 110 attempts to add a file lock to the mount information file.

[0116] As mentioned above, a mount information file can only be occupied by one process at a time (with file lock).

[0117] In step S730, when the first process 110 is able to add a file lock to the mount information file, it can be determined that the mount information file is closed and the file lock is released.

[0118] Therefore, in step S740, the first process may obtain the mount parameters and the connection identifier from the mount information file.

[0119] In step S750, after the processing of the file access request being executed is completed, the first process 110 interacts with the kernel 140 to perform a disk unmount operation in the file system.

[0120] Therefore, even when the second process 120 unexpectedly fails, the disk unloading operation can be implemented.

[0121] Based on the above solution, the upper-layer second process 120 can communicate with the first process 110 that independently manages the disk to implement the file system function, thereby enabling the upper-layer process to access the user-mode file system.

[0122] So far, the file system implementation method and system according to the present disclosure have been described in detail.

[0123] The present disclosure avoids the use of the FUSE mechanism by using a user-mode process to manage the disk itself, thereby avoiding the problem of poor performance.

[0124] Based on the shared memory 130, the upper layer second process 120 is decoupled from the first process 110 that independently manages the disk to implement the file system function, so that multiple upper layer second processes 120 can access the same file system.

[0125] Therefore, the present disclosure can solve the problem of poor FUSE performance and the problem that multiple upper-layer second processes need to access the same file system at the same time.

[0126] Figure 8 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned file system implementation method according to an embodiment of the present invention is shown.

[0127] See also Figure 8 , the computing device 800 includes a memory 810 and a processor 820 .

[0128] The processor 820 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 820 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 820 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0129] The memory 810 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 820 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 810 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 810 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0130] The memory 810 stores executable codes, and when the executable codes are processed by the processor 820 , the processor 820 can execute the file system implementation method mentioned above.

[0131] The file system implementation method and system according to the present invention have been described in detail above with reference to the accompanying drawings.

[0132] In addition, the method according to the present invention can also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present invention. In other words, the computer program product may include executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the method as described above.

[0133] Alternatively, the present invention may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present invention.

[0134] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0135] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for implementing a file system, comprising: Running a first process in a user state space, the first process is used to interact with the kernel to implement a file management function; Running at least one second process in the user space; The second process communicates with the first process by using the shared memory as a channel and transmits a file access request to the first process to access the file, wherein the file access request includes file information of the file requested to be accessed by the second process; and The first process writes the processing result information of the corresponding file access request in the shared memory; The second process creates a mount information file in the specified folder and writes mount parameters for the disk; The first process senses the existence of the mount information file, retrieves mount parameters from the mount information file, and interacts with the kernel to perform a disk mount operation in the file system; After the disk mounting operation is successful, the first process allocates a connection identifier to the second process, and writes the connection identifier together with the mounting result information into the mounting information file; The second process obtains the mounting result and the connection identifier from the mounting information file, and carries the connection identifier in a subsequent file access request to the disk.

2. The method according to claim 1, wherein: The shared memory is divided into a plurality of request processing units, each of which is used for a file access request.

3. The method according to claim 2, wherein: Each request processing unit includes a data cache block for a file access request, and is allocated with a corresponding request information field and feedback information field. The data cache block is used to cache data that the second process requests the first process to write into the disk or data that the first process reads from the disk in response to the file access request of the second process. The request information field is used for the second process to write request information related to the file access request. The feedback information field is used for the first process to write execution feedback information of the file access request.

4. The method according to claim 3, wherein: The shared memory is divided into at least one block, each block is used to process file access requests of different size ranges; Each block is divided into at least one channel, each channel includes a plurality of the request processing units, and the request processing units are suitable for processing file access requests corresponding to the size range processed by the block to which they belong.

5. The method according to claim 4, wherein: Each channel includes: A channel header, including a size field and an idle state bitmap of a request processing unit, wherein the size field records the size range of the file access request processed by the channel, and the idle state bitmap indicates whether the multiple request processing units included in the channel are in an idle state or an occupied state; A request information array, comprising the request information fields of a plurality of the request processing units; a feedback information array, comprising the feedback information fields of a plurality of the request processing units; and A data cache area includes the data cache blocks of multiple request processing units.

6. The method according to claim 2, further comprising: Maintaining current status information of the request processing unit in association with the request processing unit, the status of the request processing unit including at least one of the following: Idle state: the request processing unit has not been assigned to a file access request; The request processing unit has been acquired by the second process, but the second process has not yet filled the request processing unit with a file access request; The second process has already filled the file access request to the request processing unit and is waiting for a reply from the first process; The first process has obtained the file access request of the second process and is processing it; as well as The first process has completed processing the file access request.

7. The method according to claim 1, wherein: The second process applies a file lock to the mount information file created by the second process, and maintains the file lock during the life of the second process, wherein only one file lock can be applied to one mount information file, and the method further includes: When the second process actively issues a disk unmount instruction, the second process releases the file lock, and the first process senses through the inotify mechanism that the mount information file is closed and the file lock is released, obtains the mount parameters and the connection identifier, and interacts with the kernel to perform a disk unmount operation in the file system after the file access request being executed is processed; and / or In the event that the second process fails unexpectedly, the operating system closes the mount information file and releases the file lock. The first process attempts to add a file lock to the mount information file, and if the file lock can be added to the mount information file, obtains the mount parameters and the connection identifier, and after the file access request being executed is processed, interacts with the kernel to perform a disk unmount operation in the file system.

8. A system for implementing a file system, comprising: A first device is used to run a first process in a user state space, and the first process is used to interact with a kernel state to implement a file system management function; A second device, used to run at least one second process in a user state space; as well as Shared memory, as a channel for the second process to communicate with the first process, The second process communicates with the first process through the shared memory and transmits a file access request to the first process to access the file. The file access request includes file information of the file requested to be accessed by the second process. The first process writes the processing result information of the corresponding file access request in the shared memory; The second process creates a mount information file in the specified folder and writes mount parameters for the disk; The first process senses the existence of the mount information file, retrieves mount parameters from the mount information file, and interacts with the kernel to perform a disk mount operation in the file system; After the disk mounting operation is successful, the first process allocates a connection identifier to the second process, and writes the connection identifier together with the mounting result information into the mounting information file; The second process obtains the mounting result and the connection identifier from the mounting information file, and carries the connection identifier in a subsequent file access request to the disk.

9. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer program product, comprising executable codes, which, when executed by a processor of an electronic device, cause the processor to perform the method according to any one of claims 1 to 7.

11. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images