Access method and equipment of network file system

Abstract

The embodiment of the invention provides an access method and equipment for a network file system. The method comprises the following steps: receiving a plurality of operation requests for a network file system, and determining a target custom function associated with the plurality of operation requests from a preset custom library function; calling the target custom function to merge the plurality of operation requests to obtain a merged operation request; the merging operation request is sent to a network file system server side through a network file system client side, and the network file system server side analyzes the multiple operation requests from the merging operation request and executes the multiple operation requests. The method can achieve the effects of reducing response delay and improving NFS performance.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for accessing a network file system. Background Art

[0002] The Network File System (NFS) allows computers in a network to share files through the TCP / IP network. In NFS applications, the NFS client can transparently read and write files on the NFS server just like accessing local files.

[0003] With the rapid development of high-tech technologies such as cloud computing, big data, and artificial intelligence, all kinds of data have also increased dramatically. Accessing NFS data through local and remote networks, especially when facing access to a large number of small files, has a large response delay, resulting in poor NFS performance. Summary of the invention

[0004] The embodiments of the present application provide a method and device for accessing a network file system, so as to reduce the delay of NFS file access response and thereby improve NFS performance.

[0005] In a first aspect, an embodiment of the present application provides a method for accessing a network file system, comprising: receiving multiple operation requests for a network file system, determining a target custom function associated with the multiple operation requests from a preset custom library function; calling the target custom function to merge the multiple operation requests to obtain a merged operation request; sending the merged operation request to a network file system server through a network file system client, wherein the network file system server parses the multiple operation requests from the merged operation request and executes the multiple operation requests.

[0006] In a possible implementation, calling the target custom function to merge the multiple operation requests to obtain a merged operation request includes:

[0007] The target user-defined function is called to perform vectorized processing on the multiple operation requests, and a merged operation request including the vectorized multiple operation requests is generated.

[0008] In a possible implementation manner, the multiple operation requests include at least one of the following:

[0009] Multiple operation requests for the same file;

[0010] At least one operation request was made to a different file.

[0011] In a possible implementation manner, the at least one operation request for each of the different files includes one or more of the following:

[0012] Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

[0013] In a possible implementation, the target custom function includes one or more of the following:

[0014] Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

[0015] In a possible implementation manner, sending the merge operation request to the network file system server through the network file system client includes:

[0016] The network file system client sends the merge operation request to the network file system server based on an RPC request that is independent of transmission.

[0017] In a possible implementation manner, sending the merge operation request to the network file system server through the network file system client includes:

[0018] Obtaining a file handle of at least one file related to the multiple operation requests from metadata stored in the network file system server;

[0019] The merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

[0020] In a possible implementation, calling the target user-defined function to perform vectorized processing on multiple operation requests to generate a merged operation request including the vectorized multiple operation requests includes:

[0021] Constructing a vector of I / O structures according to at least one operation request of the plurality of files to be requested, wherein each element of the vector of I / O structures corresponds to a file to be requested;

[0022] The vector of the I / O structure is passed into the target custom function through the target custom function interface to obtain the merge operation request.

[0023] In a possible implementation, each vector element in the vector of the I / O structure includes the following data: path, offset, length, buffer, and flag.

[0024] In a possible implementation, the at least one operation request includes a read operation or a write operation for the multiple files to be requested; and the target custom function includes a custom read function or a custom write function.

[0025] In a second aspect, an embodiment of the present application provides an access device for a network file system, comprising: a receiving module, used to receive multiple operation requests for the network file system, and determine a target custom function associated with the multiple operation requests from a preset custom library function; a calling module, used to call the target custom function to merge the multiple operation requests to obtain a merged operation request; a sending module, used to send the merged operation request to a network file system server through a network file system client, wherein the network file system server parses the multiple operation requests from the merged operation request and executes the multiple operation requests.

[0026] In a possible implementation manner, the calling module is specifically used to:

[0027] The target user-defined function is called to perform vectorized processing on the multiple operation requests, and a merged operation request including the vectorized multiple operation requests is generated.

[0028] In a possible implementation manner, the multiple operation requests include at least one of the following:

[0029] Multiple operation requests for the same file;

[0030] The same operation request is for different files.

[0031] In a possible implementation manner, the at least one operation request for different files includes one or more of the following:

[0032] Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

[0033] In a possible implementation, the target custom function includes one or more of the following:

[0034] Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

[0035] In a possible implementation manner, the sending module is further used for:

[0036] The network file system client sends the merge operation request to the network file system server based on an RPC request that is independent of transmission.

[0037] In a possible implementation manner, the sending module is further used for:

[0038] Obtaining a file handle of at least one file related to the multiple operation requests from metadata stored in the network file system server;

[0039] The merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

[0040] In a possible implementation, the calling module is further used for:

[0041] Constructing a vector of I / O structures according to at least one operation request for a plurality of files to be requested, wherein each vector element of the vector of I / O structures corresponds to a file to be requested;

[0042] The vector of the I / O structure is passed into the target custom function through a target custom function interface to obtain the merge operation request.

[0043] In a possible implementation, each vector element in the vector of the I / O structure includes the following data: path, offset, length, buffer, and flag.

[0044] In a possible implementation, the at least one operation request includes a read operation or a write operation for the multiple files to be requested; and the target custom function includes a custom read function or a custom write function.

[0045] In a third aspect, an embodiment of the present application provides a network file system access device, including: a memory, a processor;

[0046] The memory stores computer-executable instructions;

[0047] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0050] The access method and device of the network file system provided by the embodiment of the present application, by receiving multiple operation requests for the network file system, determining the target custom function associated with the multiple operation requests from the preset custom library function; calling the target custom function to merge the multiple operation requests to obtain a merged operation request; sending the merged operation request to the network file system server through the network file system client, wherein the network file system server parses the multiple operation requests from the merged operation request and executes the multiple operation requests, and there is no need to call the virtual file system conversion layer once for each operation request and construct a remote procedure call request on the user terminal side, only need to call the virtual file system conversion layer once for the merged operation request, and construct a remote procedure call request, thereby reducing the operation of the NFS client calling the virtual file system conversion layer and constructing the remote procedure call request. The server can receive a remote procedure call request from the NFS client to execute multiple operation requests without the need to interact with the NFS client for multiple operation requests separately. Since the interaction process is reduced, the load of the server due to the above-mentioned interaction process can be reduced, which is conducive to the rapid response of multiple operation requests of the server, and the delay time of the server responding to the access request of the file can be reduced, thereby improving the performance of NFS. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] Figure 1 A schematic diagram of the interaction between the network file system client and the server provided for this application;

[0053] Figure 2 Schematic diagram of the process of accessing the network file system provided by this application Figure 1 ;

[0054] Figure 3 Schematic diagram of the process of accessing the network file system provided by this application Figure 2 ;

[0055] Figure 4 A schematic diagram of the interaction process between the client and the server in the network file system access method provided by this application;

[0056] Figure 5This is a schematic diagram of an application scenario;

[0057] Figure 6 This is a schematic diagram of an application scenario;

[0058] Figure 7 A schematic diagram of the structure of the access device of the network file system provided by the present application;

[0059] Figure 8 A schematic diagram of the structure of the access device of the network file system provided in this application.

[0060] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0061] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0062] First, the terms involved in this application are explained:

[0063] Network File System (NFS) is a distributed file system protocol that allows client systems to transparently access remote storage resources over the network, just like accessing local file systems, without having to know the actual location of the files on the physical storage device.

[0064] Remote Procedure Call (RPC) is a technology that allows a program running on one computer to call a subroutine on another computer. This technology shields the underlying network communication details, making remote communication of program components as simple as local calls. The RPC mechanism enables developers to build distributed computing systems in which different components can be distributed on different computers, but they can call each other as if they were on the same electronic device.

[0065] Portable Operating System Interface (POSIX) is a set of operating system interface standards designed to ensure the portability of operating systems on multiple platforms. POSIX defines a set of operating system interface specifications, including system call interfaces, command line tools, threads and synchronization mechanisms, library function interfaces, and file system interfaces. Portable Operating System Interface (POSIX API).

[0066] A file handle is an opaque data structure that uniquely identifies a specific file or directory in the file system and is used to ensure that the client can correctly access and operate resources on the remote file system. The specific content of the file handle is opaque, which means that its internal structure is unknown to the NFS client and should not be parsed directly. The file handle includes but is not limited to the following elements: File system ID: identifies the file system where the file is located, File ID: uniquely identifies the file or directory in the file system, File version number: used to detect file changes and ensure that the data cached by the client is the latest, Other information: other auxiliary information, such as permissions, timestamps, etc., which help improve performance or security. The file status corresponding to the client (such as open, upload, download, etc.). The role of the file handle: First, the unique identification of the file or directory. Each file or directory has a unique file handle on the NFS server. This handle is passed between the client and the server to identify the specific file or directory. Second, location transparency. By using the file handle, the NFS client can interact with the file without knowing the actual storage location of the file, which provides location transparency. Third, permission control: file handles can also implicitly carry permission information to help the NFS server decide whether to allow the client to perform certain operations. Fourth, efficient access: file handles can contain optimized information, such as the location or metadata of the file, to access the file more quickly.

[0067] Virtual Filesystem Switch (VFS) is an abstraction layer in modern operating system kernels that unifies the interfaces of different types of file systems. VFS provides a set of standard, abstract file operation functions, allowing upper-layer applications to interact with any supported file system in a consistent manner without having to worry about the underlying specific file system implementation details.

[0068] Please refer to the following Figure 1 , Figure 1 FIG. 1 is a schematic diagram of the interaction between the network file system client and the server in the related art. Figure 1 As shown, the NFS client runs in the user's terminal device 101, and the NFS server 102 runs in the remote server.

[0069] Applications can be run in the terminal device 101, and users can initiate file operation instructions to the network file system through the application, such as opening files, reading files, writing file contents, closing files, etc. The application sends a file operation request through the POSIX API. The POSIX API call is passed to the VFS layer of the operating system kernel. VFS determines which specific file system the target file or directory belongs to based on the mount point and path information. If the path points to a mounted NFS shared directory, VFS will forward the request to the NFS client module. After receiving the request from the VFS layer, the NFS client constructs a corresponding RPC request. The process of constructing the corresponding RPC request includes: first, obtaining a file handle: using the file handle previously obtained from the NFS server to identify a specific file or directory. Second, serializing parameters, serializing the request parameters according to the NFS protocol format. The NFS client sends the constructed remote procedure call request to the NFS server through the network.

[0070] Each step of the file operation requires information to be passed between the NFS client and the server. For example, the read operation of a file includes the steps of finding, opening, reading, and closing. The information of each step needs to pass through the POSIX API. Due to the serialization characteristics of the POSIX API, multiple operations cannot be merged.

[0071] In the related art, different access requests are merged at a server or network side other than a client, one of which is to merge at the server side and the other is to merge at the network device side.

[0072] The server's disk I / O scheduler combines many small requests into several larger requests to minimize searches. For small files, disk reading and writing takes less time, while the file opening operation (open()) takes up most of the system time, resulting in very low disk effective service time and poor disk performance. Therefore, the server can accumulate multiple operations from the client on a small file and process the accumulated multiple operations in a unified manner. When the server accumulates multiple operations for unified processing, some operation requests are placed in a waiting state, resulting in a long delay time, which is not conducive to metadata operations and access requests with high real-time requirements.

[0073] Merging on the network device side means that the network device reduces the number of small data packets sent through a preset packet sending algorithm to improve bandwidth utilization, thereby avoiding network congestion. However, the way to reduce small data packets is to accumulate the operation requests sent by the client to obtain larger data packets before transmitting them to the server. This merging method on the network side, on the one hand, the above-mentioned packet sending algorithm is not applicable to all operation requests, and on the other hand, this merging method will also cause a large delay, which is not conducive to applications and small files that require fast response and are sensitive to delays.

[0074] Therefore, the current NFS system has a large delay in responding to operation requests for small files, resulting in poor performance of NFS.

[0075] The solution provided by the present application combines multiple operation requests for NFS on the client side and sends the combined operation requests to the NFS server through the network for processing. Therefore, the number of interactions between the client and the server is reduced. In addition, the server and the network side do not need to accumulate operation requests and then process them uniformly, which can reduce the delay time for responding to operation requests, which is conducive to improving the response speed of small file operations and improving NFS performance.

[0076] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0077] Please refer to Figure 2 , Figure 2 Schematic diagram of the process of accessing the network file system provided by this application Figure 1 ,like Figure 2 As shown, the method includes:

[0078] S201: receiving a plurality of operation requests for a network file system, and determining target custom functions associated with the plurality of operation requests from a preset custom function library.

[0079] The execution subject of the access method of the network file system (for example, a terminal device running an NFS client) can receive an operation request for the network file system initiated by an application. The objects of the multiple operation requests can be, for example, small files. Small files can refer to files whose file size is less than a preset threshold. The above preset threshold can be, for example, 1M bytes.

[0080] The above operation request may be a file creation operation, a directory creation operation, a file search operation request, an open operation request, a read operation request, a write operation request, a copy operation request, an attribute search request, an attribute setting operation request, etc.

[0081] In an example, the multiple operation requests may include multiple dependent operations, such as multiple operations that are logically sequential, or the same operation on different files.

[0082] The above-mentioned multiple operations having a logical sequence relationship may be, for example, a search operation, an open operation, a read operation and a close operation on the same file.

[0083] In some application scenarios, the above-mentioned multiple operation requests may be multiple different operations for the same file, such as operations such as searching, opening, reading, and closing the same file. The merging of multiple operation requests may be merging multiple operations for the same file. In these application scenarios, a target custom function that can merge multiple different operations for the same file may be determined in the target custom function.

[0084] In some application scenarios, the above-mentioned multiple operation requests may also be at least one operation for different files, such as read operations performed on file 1, file 2, and file 3; for example, it may be an open operation and a read operation on file 1, an open operation and a read operation on file 2, and an open operation and a read operation on file 3. In these application scenarios, the above-mentioned execution subject may determine the target custom function associated with the multiple operation requests from the preset custom function library. For example, when the above-mentioned multiple operation requests are read operations on multiple files, the target custom function is a multi-file read function (Nread function).

[0085] S202: calling a target user-defined function to merge multiple operation requests to obtain a merged operation request.

[0086] In an example, the target custom function may encapsulate multiple operation requests to obtain an encapsulated merged operation request.

[0087] Specifically, the target custom function can arrange multiple operations in a sequential order, and encapsulate the obtained operation sequence into a merged operation request. The operation sequence can indicate which operation to perform in the first step, which operation to perform in the second step, and so on.

[0088] The merge operation request still includes multiple operation requests for the file in the NFS.

[0089] In some application scenarios, at least one operation request for different files includes one or more of the following:

[0090] Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

[0091] In some application scenarios, the custom function library runs at the application layer. The custom function library may include one or more of the following:

[0092] Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

[0093] Among them, the multi-file opening function Nopen() can open multiple files (specified by path) in one RPC request. In order to locate the search required for their parent directory, Noopen_simple is provided in the external merge sort. Noopen_simple uses a set of common flags and modes for all files: an integer index of an internal table is defined and used when the file is created (file cursor, status identifier, sequence, etc.).

[0094] The multiple file closing function Nclose() can close multiple opened files and release their resources.

[0095] The multi-file multi-attribute reading function Ngetattrs() can read multiple attributes of multiple files, combining multiple system calls (chmod, chown, time and truncation, etc.). Not only can multiple attributes be obtained at one time, but the attribute acquisition of multiple files can be processed in one RPC request. File collections and collections use attribute structure arrays as input and output. Each structure contains an Nfile structure (used to set the file path), all attributes (mode, size, etc.), and a bitmap showing which attributes are displayed.

[0096] The multi-file multi-attribute setting function NSetattrs() can set multiple attributes of multiple files, combining multiple system calls (chmod command (used to change the permission settings of files or directories), chown command (used to change the owner and group of files or directories), time and truncation, etc.). Not only can multiple attributes be set at once, but the attribute settings of multiple files can be processed in one RPC request. File collections and collections use attribute structure arrays as input and output. Each structure contains an Nfile structure, all attributes (mode, size, etc.), and a bitmap showing which attributes are which.

[0097] The multi-file multi-attribute acquisition function Ngetattrs() can set multiple attributes for multiple files.

[0098] The multi-directory creation function Nmkdir() can create multiple directories (such as a directory tree). Use Nmkdir() to ensure that a deep directory and all its ancestors exist. For example, when creating a directory, Ngetattrs finds out which ancestors exist, and then uses Nmkdir to create the missing directory.

[0099] Multiple directory read function Nlistdir(), this function speeds up directory listing and implements the following improvements: (1) multiple directories can be listed at once; (2) no opendir call is required before starting to list the directory; (3) Nlistdir retrieves attributes and directory entries, saving subsequent statistics; (4) Nlistdir can work recursively (breadth-first, directories at the same level in the tree will be listed in arbitrary order). This operation can be seen as a fast vectorized solution that uses as few rpc requests as possible to read NFS directory contents.

[0100] Nlistdir can accept five parameters: (1) an array of directories to list; (2) a bitmap indicating the desired attributes; (3) a flag to select recursive listing; (4) a user-defined callback function; and (5) an opaque pointer provided by the user to be passed to the callback.

[0101] For each directory entry listed, the callback function is called with three arguments: (1) the file path and the file's attribute structure; (2) the top-level ancestor directory that appeared in the first argument to Nlistdir; and (3) the opaque pointer (the last argument to Nlistdir).

[0102] The multi-link creation function Nsymlink() and the multi-link reading function Nreadlink() allow many links to be created or read at once, and both accept a path vector and a buffer vector containing the target paths.

[0103] The multi-file or multi-directory deletion function Nremove() can delete multiple files and directories at once. Nrmove itself does not support recursive deletion, but it can be achieved through recursive Nlistdir and appropriately ordered Nremove.

[0104] The multi-file renaming function Nrename() can implement batch renaming and rename the source path vector to the target path vector using as few RPCs as possible.

[0105] Multiple custom functions in the custom function library can replace the POSIX function library. Therefore, it is not necessary to send each operation step to the server through the NFS client in a serialized form through the POSIX API. Since each custom function in the custom function library can encapsulate multiple operations into a merge operation, an RPC request can be constructed for the merge operation request, and then sent by the NFS client to the NFS server based on the RPC protocol and the network.

[0106] S203: Sending the merge operation request to the network file system server through the network file system client, wherein the network file system server parses the merge operation request to obtain multiple operation requests and executes the multiple operation requests.

[0107] The custom function library can communicate with the kernel's VFS and obtain the path corresponding to the target file or directory from the VFS. The path corresponding to the target file or directory is sent to the NFS client. The NFS client constructs an RPC request and sends the RPC request to the NFS server through the network.

[0108] In some application scenarios, the above step S203 includes: the NFS client sends the merge operation request to the NFS server through the network through an RPC request that is unrelated to the transmission.

[0109] Transport-independent RPC separates application logic from its underlying transport protocol, allowing the RPC mechanism to work independently of the specific network transport layer. Transport-independent RPC allows applications to switch between different transport protocols without modifying the application code. Transport-independent RPC decouples application logic and transport protocols by providing an abstraction layer. RPC requests can be transmitted through different transport protocols, which improves the flexibility of RPC request transmission and helps shorten the transmission path for file operations on NFS.

[0110] The NFS client can send the merge operation request to the NFS server through the network device and the network. After receiving the merge operation request, the NFS server parses the merge operation request into multiple operation requests and executes the multiple operation requests.

[0111] For example, the execution order of each operation request may be parsed from the vectorized merge operation request, and the multiple operation requests may be executed in sequence according to the execution order.

[0112] For example, a vectorized merge operation request includes four operations of finding, opening, reading, and closing for file 1 arranged in the order of execution. The NFS server can parse the four operations of finding, opening, reading, and closing to be executed in sequence from the merge operation request, and execute the four operations of finding, opening, reading, and closing for file 1 in sequence.

[0113] In this embodiment, when multiple operation requests for NFS are received, a target custom function interface is determined from a custom function library, multiple operation requests are merged using the target custom function interface, and the merged operation request obtained by merging is sent to the NFS client, which is then sent to the NFS server by the NFS client. Since multiple operation requests are encapsulated by custom functions in the terminal device, multiple operation requests are sent to the NFS server at one time for processing. On the user terminal side, there is no need to call VFS once for each operation request and construct an RPC request. Only VFS needs to be called once for the merged operation request and an RPC request needs to be constructed, thereby reducing the operations of the NFS client calling VFS and constructing RPC requests. The server can receive an RPC request from the NFS client to execute multiple operation requests without having to interact with the NFS client for multiple operation requests. Since the interaction process is reduced, the load of the server caused by the above-mentioned interaction process can be reduced, which is conducive to the server to quickly respond to multiple operation requests, and the delay time for the server to respond to the file access request can be shortened, thereby improving the performance of NFS.

[0114] Please refer to Figure 3 , Figure 3 Schematic diagram of the process of accessing the network file system provided by this application Figure 2 ,like Figure 3 As shown, the method includes:

[0115] S301: receiving multiple operation requests for a network file system, and determining target custom functions associated with the multiple operation requests from a preset custom function library.

[0116] The specific implementation of step S301 can refer to Figure 1 The step S201 shown is not repeated here.

[0117] S302: Calling a target user-defined function to perform vectorized processing on multiple operation requests, and generating a merged operation request including the multiple vectorized operation requests.

[0118] Vectorizing multiple operations means representing multiple operations in the form of vectors. That is, the merged operation request includes multiple operations represented by a vector. Each operation can be an element in the vector.

[0119] By representing multiple operations in a vector form, multiple operations that need to be transmitted to the NFS server separately can be sent to the NFS server at one time.

[0120] In addition, when multiple operations are vectorized, the multiple operations may be vectorized according to a rule that the NFS server can identify a sequential execution order.

[0121] Each function in the custom function library can provide a vectorized API. Through each vectorized API, multiple operations executed in steps can be converted into operations represented in vector form that can be executed simultaneously.

[0122] S303: Sending the merge operation request to the network file system server through the network file system client, wherein the network file system server parses the merge operation request to obtain multiple operation requests and executes the multiple operation requests.

[0123] In this embodiment, it is described that a target custom function vectorizes multiple operations to obtain a merge operation request, which can simplify the implementation process of merging multiple operations and is conducive to improving the speed of merging multiple operations.

[0124] In some embodiments, step S303 includes the following steps:

[0125] First, a file handle of at least one file related to the multiple operation requests is obtained from metadata stored in the network file system server.

[0126] Secondly, a merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

[0127] The network file system server may be a parallel network file system (Parallel NFS, pNFS), in which metadata and file content data may be stored. The metadata may include file handles of multiple files. The file handle may include access rights, file content storage paths, etc.

[0128] In these embodiments, the file handle may be obtained from metadata through the current merge operation request, or may be obtained from metadata through historical interactions.

[0129] The operation objects of the multiple operation requests for the network file system may be one file or multiple files. For example, the multiple operation requests may be multiple operation requests for one file. Alternatively, the multiple operation requests include operation requests for different files.

[0130] In some application scenarios, for at least one file related to multiple operation requests, the file handle of each file can be obtained from metadata. Specifically, the network file system client can send the file identifier of each file related to the operation request to the network file system server, and the network file system server sends the file handle obtained from the metadata to the network file system client. The network file system client generates a merged operation request including the file handles of each file.

[0131] In these embodiments, by obtaining the file handle of the file from the metadata, the NFS client can send a merge operation request carrying the file handle to the network file system server, so that the network file system server directly operates the file through the file handle. This helps to realize parallel reading and writing of data to different files, thereby significantly improving file access performance. Since there is no need to obtain the storage path of the file once for each operation on the file, it helps to reduce the load of the network file system server.

[0132] exist Figure 3 In some possible implementations of the illustrated embodiment, step S302 includes:

[0133] According to at least one operation request of the plurality of files to be requested, construct a vector of I / O structures, wherein each vector element of the vector of I / O structures corresponds to a file to be requested;

[0134] The vector of the I / O structure is passed to the target custom function through the target custom function interface to obtain a merge operation request.

[0135] Vector I / O (also called scatter / gather I / O) is a technique that allows a single system call to process multiple non-contiguous memory areas. This technique provides an array (i.e., vector) containing multiple buffers, allowing applications to read or write blocks of data scattered in different locations at one time without calling I / O operations multiple times.

[0136] The vector of I / O structures may be represented by, for example, Struct nfio iostr[3], indicating that the vector includes 3 vector elements, each of which may correspond to a file.

[0137] Each vector element in the vector of I / O structures can contain the Nfile structure (used to set the file path), offset, length, buffer and flags of the corresponding file.

[0138] Among them, each vector element can be represented as follows:

[0139] { nfile = {.type = PATH,

[0140] path=" / root / Test / test.txt"},

[0141] offset = 0,

[0142] length = 64×1024,

[0143] data=buf1, / / includes a file output end bit

[0144] flags=0,

[0145] }

[0146] Among them, the Nfile structure nfile={} is used to write the file path. Offset is used to set the offset, and the offset above is 0. Length is the length of the file content to be read or the file content to be written. Data is used to set the buffer, and the above buffer can be the buffer in the user terminal device. Flags is used to set the corresponding status of the server.

[0147] In these implementations, by constructing a vector of I / O structures including a plurality of vector elements, each vector element corresponds to data of a file to be requested, thereby achieving merging of operation requests for a plurality of files.

[0148] Each vector element in a vector of I / O structures includes the following data: path, offset, length, buffer, and flags.

[0149] The above path may be a storage path in VFS. The length may be the length of the file involved in the operation request. The buffer may be a buffer on the terminal device side. The flag may be a flag of the requested file obtained from the NFS server. By setting the above data in the vector element, the server can parse each operation request according to each vector element and execute the operation corresponding to each original operation request.

[0150] In some implementations, at least one operation request includes a read operation or a multi-file write operation for multiple requested files; and the target custom function includes a custom read function or a custom write function.

[0151] According to the read request or write request of multiple files to be requested, a vector of I / O structure is constructed; the vector of I / O structure is passed into the custom read function through the custom read function interface to obtain the read operation of multiple files; or, the vector of I / O structure is passed into the custom write function through the custom read function interface to obtain the write operation of multiple files.

[0152] Custom read functions and custom write functions can accept vectors of I / O structures.

[0153] The following is an example of a case where multiple operation requests are multiple file read requests. The application can construct a vector of I / O structures for read requests of multiple files. Then the custom read function interface is called to pass the vector of the I / O structure of the read request to the custom read function. The application sends the custom read function to which the vectorized multiple operations are passed to the NFS client. The NFS client constructs an RPC request that is independent of transmission based on the custom read function, and sends the RPC request to the NFS server through the network. Exemplarily, the call to the custom read function for the vector of the I / O structure including 3 elements can be as follows:

[0154] struct nres r = nread(iostr, 3).

[0155] For multi-file write requests, the relevant NFS protocol does not support appending. If multiple processes append data to a file at the same time, an empty append (0_APPEND) may cause file corruption on the NFS file system. In the related art, the kernel NFS client appends by writing an offset equal to the current known file size. This behavior is inefficient because the file size must first be read in a separate RPC and is vulnerable to attacks through security vulnerabilities. The custom write function nwrite() in this application uses a special offset value as an append marker in the vector of the I / O structure, and records the file size to be written as the valid offset after the append marker.

[0156] The vector of I / O structures includes elements whose number can be adjusted according to actual conditions.

[0157] In these embodiments, by constructing an I / O structure vector for a read operation or a write operation of multiple files, and using a corresponding multi-file read function or a multi-file write function to generate a write operation request or a multi-file read request for multiple files, it is possible to simultaneously send the read requests or write requests of the multiple files to the NFS server for processing. When the multiple files are small files, the response speed for reading or writing the small files can be improved.

[0158] Please refer to Figure 4 , Figure 4 The following is a schematic diagram of the interaction process between the client and the server in the network file system access method provided by this application. Figure 4 As shown, the NFS client runs in the user's terminal device 401, and the NFS server 102 runs in the remote server.

[0159] Applications (referred to as applications) can be run in terminal devices, and users can use applications to initiate file operation instructions to the network file system, such as opening files, reading files, writing file contents, closing files, and other operations. The application encapsulates multiple operation requests into a merged operation request by calling the target custom function in the custom function library running in the application layer. Specifically, the target custom function can obtain the path of the target file or directory corresponding to the operation request from the VFS. The target custom function constructs a merged operation request including the path and multiple operation requests (vectorizing multiple operation requests), and sends the merged operation request to the NFS client. The NFS client constructs an RPC request that is independent of transmission, and the NFS client sends the constructed PPC request to the NFS server through the network device and the network.

[0160] and Figure 1 compared to, Figure 4 In the interaction process between the client and the server shown in the figure, there is no need for the POSIX API to send the access operation request to the VFS. Instead, the target custom function in the custom library function interacts with the VFS, avoiding the problem that multiple operations cannot be merged due to the serialization processing of the POSIX protocol. Through the functions provided in the custom function library, multiple operations can be encapsulated into a merged operation request and sent to the NFS client, which can reduce the number of times the terminal device side application calls the VFS and the number of times the RPC request is constructed. For multiple operation requests, the number of interactions between the client and the server is also reduced, which is conducive to reducing the response time of the operation request.

[0161] Please refer to Figure 5 , Figure 5 A schematic diagram of an application scenario is shown. Figure 5 As shown, multiple operation requests for the / root / Test / test.txt file can be merged using the target custom function to obtain a merged operation request. Figure 5 The multiple operation requests shown include: searching for the root directory "root", searching for the parent directory "Test" of test.txt, opening the test.txt file, reading the test.txt file (the read offset is 4096 bytes) and closing the test.txt file.

[0162] Merge operation request Figure 5 "PUTROOTFHP; LOOKUP "root"; GETTH; GETATTR; LOOKUP "Test"; GETTH; GETATTR; OPEN "test.txt"; READ 0 4096; CLOSE; GETTH; GETATTR" shown.

[0163] The NFS server can return the content of / root / Test / test.txt read by the root attributes and the test attributes.

[0164] Please refer to Figure 6 , Figure 6 A schematic diagram of an application scenario is shown. Figure 6 As shown, the operation requests for file a, file b, file c and file d can be merged into one merged operation request. As shown in the figure, the operation request for file a includes: root directory "root" search and parent directory "Test" search.

[0165] The operation requests for file b (text.txt) include: an operation of opening the text.txt file, an operation of reading part of the content of the text.txt file, and an operation of closing the text.txt file.

[0166] The operation requests for file c (text2.txt) include: an operation of opening the text2.txt file, an operation of reading part of the content of the text2.txt file, and an operation of closing the text2.txt file.

[0167] The operation requests for file d (text3.txt) include: an operation of opening the text3.txt file, an operation of reading part of the content of the text3.txt file, and an operation of closing the text3.txt file.

[0168] Multiple operation requests for file a, file b, file c, and file d can be combined into one combined operation request, which is sent by the NFS client to the NFS server. This further reduces the number of RPC requests constructed by the client and the process of information interaction with the server, which can further improve the server's response speed to operation requests and optimize NFS performance.

[0169] Figure 7 A schematic diagram of the structure of the access device of the network file system provided by this application, such as Figure 7 As shown, the network file system access device 70 provided in this embodiment includes:

[0170] The receiving module 701 is used to receive multiple operation requests for the network file system, and determine the target custom functions associated with the multiple operation requests from the preset custom library functions;

[0171] A calling module 702 is used to call a target custom function to merge multiple operation requests to obtain a merged operation request;

[0172] The sending module 703 is used to send the merge operation request to the network file system server through the network file system client, wherein the network file system server parses the merge operation request to obtain multiple operation requests and executes the multiple operation requests.

[0173] In a possible implementation, the calling module 702 is specifically used for:

[0174] The target user-defined function is called to perform vectorized processing on the multiple operation requests, and a merged operation request including the vectorized multiple operation requests is generated.

[0175] In a possible implementation, the multiple operation requests include at least one of the following:

[0176] Multiple operation requests for the same file;

[0177] At least one operation request is made for each of the different files.

[0178] In a possible implementation, at least one operation request for different files includes one or more of the following:

[0179] Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

[0180] In a possible implementation, the target custom function includes one or more of the following:

[0181] Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

[0182] In a possible implementation, the sending module 703 is further configured to:

[0183] The network file system client sends a merge operation request to the network file system server based on an RPC request that is independent of the transport.

[0184] In a possible implementation, the sending module 703 is further configured to:

[0185] Obtaining a file handle of at least one file related to the multiple operation requests from metadata stored in the network file system server;

[0186] The merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

[0187] In a possible implementation, the calling module 702 is specifically used for:

[0188] Constructing a vector of I / O structures according to at least one operation request of the plurality of files to be requested, wherein each element of the vector of I / O structures corresponds to a file to be requested;

[0189] The vector of the I / O structure is passed to the target custom function through the target custom function interface to obtain a merge operation request.

[0190] In a possible implementation, each vector element in the vector of I / O structures includes the following data: path, offset, length, buffer, and flag.

[0191] In a possible implementation, the at least one operation request includes a read operation or a write operation for the multiple files to be requested; and the target custom function includes a custom read function or a custom write function.

[0192] The network file system access device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0193] Figure 8 A schematic diagram of the structure of the access device of the network file system provided in this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the device 80 also includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0194] In a specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above method.

[0195] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0196] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0197] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0198] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0199] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0200] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0201] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0202] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0203] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0204] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0205] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0206] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0207] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0208] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for accessing a network file system, characterized in that: include: Receiving a plurality of operation requests for a network file system, and determining a target custom function associated with the plurality of operation requests from a preset custom library function; Calling the target custom function to merge the multiple operation requests to obtain a merged operation request; The merge operation request is sent to a network file system server through a network file system client, wherein the network file system server parses the multiple operation requests from the merge operation request and executes the multiple operation requests.

2. The method according to claim 1, characterized in that The calling of the target custom function to merge the multiple operation requests to obtain a merged operation request includes: The target user-defined function is called to perform vectorized processing on the multiple operation requests, and a merged operation request including the vectorized multiple operation requests is generated.

3. The method according to claim 1, characterized in that The multiple operation requests include at least one of the following: Multiple operation requests for the same file; At least one operation request for different files.

4. The method according to claim 3, characterized in that The at least one operation request for different files includes one or more of the following: Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

5. The method according to claim 1, characterized in that The target custom function includes one or more of the following: Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

6. The method according to claim 1, characterized in that The step of sending the merge operation request to the network file system server through the network file system client includes: The network file system client sends the merge operation request to the network file system server based on an RPC request that is independent of transmission.

7. The method according to any one of claims 1 to 6, characterized in that: The step of sending the merge operation request to the network file system server through the network file system client includes: Obtaining a file handle of at least one file related to the multiple operation requests from metadata stored in the network file system server; The merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

8. The method according to claim 2, characterized in that: The calling of the target custom function to perform vectorized processing on the multiple operation requests to generate a merged operation request including the vectorized multiple operation requests includes: Constructing a vector of I / O structures according to at least one operation request for a plurality of files to be requested, wherein each vector element of the vector of I / O structures corresponds to a file to be requested; The vector of the I / O structure is passed into the target custom function through a target custom function interface to obtain the merge operation request.

9. The method according to claim 8, characterized in that Each vector element in the vector of the I / O structure includes the following data: path, offset, length, buffer and flag.

10. The method according to claim 8 or 9, characterized in that: The at least one operation request includes a read operation or a write operation for the multiple files to be requested; and the target custom function includes a custom read function or a custom write function.

11. A network file system access device, characterized in that: include: A receiving module, configured to receive a plurality of operation requests for a network file system, and determine a target custom function associated with the plurality of operation requests from a preset custom library function; A calling module, configured to call the target custom function to merge the multiple operation requests to obtain a merged operation request; The sending module is used to send the merge operation request to the network file system server through the network file system client, wherein the network file system server parses the multiple operation requests from the merge operation request and executes the multiple operation requests.

12. The device according to claim 11, characterized in that The calling module is specifically used for: The target user-defined function is called to perform vectorized processing on the multiple operation requests, and a merged operation request including the vectorized multiple operation requests is generated.

13. The device according to claim 11, characterized in that The multiple operation requests include at least one of the following: Multiple operation requests for the same file; At least one operation request for different files.

14. The device according to claim 13, characterized in that The at least one operation request for different files includes one or more of the following: Read, write, open, close, get properties, set properties, create directory, list targets, create link, read link, file or directory delete, rename.

15. The device according to claim 11, characterized in that The target custom function includes one or more of the following: Multi-file read operation function, multi-file write operation function, multi-file open function, multi-file close function, multi-file multi-attribute read function, multi-file multi-attribute setting function, multi-directory create function, multi-directory read function, multi-link create function, multi-link read function, multi-file or multi-directory delete function, multi-file rename function.

16. The device according to claim 11, characterized in that The sending module is further used for: The network file system client sends the merge operation request to the network file system server based on an RPC request that is independent of transmission.

17. The device according to any one of claims 11 to 16, characterized in that The sending module is further used for: Obtaining a file handle of at least one file related to the multiple operation requests from metadata stored in the network file system server; The merge operation request including the file handle is sent to the network file system server, so that the network file system server responds to each operation request of the at least one file.

18. The device according to claim 12, characterized in that The calling module is further used for: Constructing a vector of I / O structures according to at least one operation request for a plurality of files to be requested, wherein each vector element of the vector of I / O structures corresponds to a file to be requested; The vector of the I / O structure is passed into the target custom function through a target custom function interface to obtain the merge operation request.

19. The device according to claim 18, characterized in that Each vector element in the vector of the I / O structure includes the following data: path, offset, length, buffer and flag.

20. The device according to claim 18 or 19, characterized in that The at least one operation request includes a read operation or a write operation for the multiple files to be requested; and the target custom function includes a custom read function or a custom write function.

21. A network file system access device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

23. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images