File Processing Method, Apparatus, Electronic Device, and Storage Medium

Abstract

An embodiment of the present invention discloses a file processing method, apparatus, electronic device, and storage medium. The method includes: determining a to-be-processed file block and a to-be-processed write request from the received to-be-processed message; determining whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block; if it is determined to enable the write-through mode, performing a structural transformation on the index of the to-be-processed file block, and controlling the CephFS client to directly write the to-be-processed file block after the structural transformation into the storage device according to the to-be-processed write request. By adopting the solution of the present application, when continuously writing new files into the file system and each file is only written once and will not be read in a short time, by performing a structural transformation on the index of the to-be-processed file block, the CephFS client can support the write request pass-through from the NFS server, so as to directly write the to-be-processed file block into the storage device, improving the average response speed and write efficiency of the file.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of distributed storage, and in particular, to a file processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Ceph File System (CephFS) is a distributed file system implemented based on the RADOS architecture. It stores the metadata and data of files or directories in respective storage pools in the form of objects. Currently, considering the compatibility and performance issues of Cephfs-client, it is usually embedded in the operating system in the form of a kernel module (ko), providing a VFS-compatible interface upward to receive requests and following the RADOS communication protocol downward to complete the forwarding of write requests and message response processing. However, although the above method solves the compatibility and performance issues, it requires a relatively high kernel version, and most devices do not meet this requirement. Summary of the Invention

[0003] The embodiments of the present invention provide a file processing method, apparatus, electronic device, and storage medium to improve the write efficiency and response speed of files and enhance system stability.

[0004] In a first aspect, the embodiments of the present invention provide a file processing method applied to an NFS server, where the NFS server is deployed in front of a CephFS client, and the method includes:

[0005] Determine a to-be-processed file block and a to-be-processed write request from the received to-be-processed message;

[0006] Determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block;

[0007] If it is determined to enable the write-through mode, perform a structural transformation on the index of the to-be-processed file block, and control the CephFS client to directly write the to-be-processed file block with the transformed index structure into the storage device according to the to-be-processed write request.

[0008] In a second aspect, the embodiments of the present invention further provide a file processing apparatus configured in an NFS server, where the NFS server is deployed in front of a CephFS client, and the apparatus includes:

[0009] A message processing module, configured to determine a to-be-processed file block and a to-be-processed write request from the received to-be-processed message;

[0010] A mode determination module, configured to determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block;

[0011] A file passthrough module, which, if it is determined to enable the passthrough mode, performs a structural transformation on the index of the to-be-processed file block, and controls the CephFS client to directly write the to-be-processed file block with the index structure transformation into the storage device according to the to-be-processed write request.

[0012] Thirdly, an electronic device is further provided in an embodiment of the present invention, including:

[0013] One or more processors;

[0014] A storage device for storing one or more programs;

[0015] The one or more programs are executed by the one or more processors, so that the one or more processors implement the file processing method as described in any embodiment of the present invention.

[0016] Fourthly, a computer-readable storage medium is further provided in an embodiment of the present invention, on which a computer program is stored, and when the program is executed by a processor, it implements the file processing method as described in any embodiment of the present invention.

[0017] A file processing method is provided in an embodiment of the present invention. After receiving a to-be-processed message, the to-be-processed file block and the corresponding to-be-processed write request are determined from the to-be-processed message, and it is judged whether to enable the passthrough mode to perform a write processing operation on the to-be-processed file block; if it is determined to enable the passthrough mode, the index of the to-be-processed file block is structurally transformed, and the CephFS client is controlled to directly write the to-be-processed file block with the index structure transformation into the storage device according to the to-be-processed write request. By adopting the solution of the present application, when continuously writing new files into the file system, each file is written only once and will not be read in a short time, by structuring the index of the to-be-processed file block, the CephFS client can support the write request passthrough from the NFS server, so as to directly write the to-be-processed file block into the storage device, improving the average response speed and writing efficiency of the file; and, by the NFS server proxying the CephFS client to provide services, the problem of too high kernel version requirements for the CephFS client can be avoided on the premise of solving compatibility and performance problems.

[0018] The above-mentioned invention content is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above-mentioned and other purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically exemplified below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present invention. Also, the same reference symbols are used throughout the drawings to represent the same parts. In the drawings:

[0020] Figure 1 It is a schematic diagram of accessing metadata and data based on the Ceph file system provided in an embodiment of the present invention;

[0021] Figure 2 is a flow chart of a file processing method provided in an embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of the architecture of a kernel-mode NFS and CephFS provided in an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of an NFS server providing services on behalf of a CephFS client provided in an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of a mode switching between cache write and write-through provided in an embodiment of the present invention;

[0025] Figure 6a is a schematic diagram of a kvec-based structure provided in an embodiment of the present invention;

[0026] Figure 6b is a schematic diagram of a bio_vec-based structure provided in an embodiment of the present invention;

[0027] Figure 7 is a flow chart of performing structural transformation on a file block to be processed provided in an embodiment of the present invention;

[0028] Figure 8 is a structural block diagram of a file processing device provided in an embodiment of the present invention;

[0029] Figure 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0031] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts depict the operations (or steps) as sequential processes, many of the operations (or steps) can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but it can also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0032] To better understand the technical solution of the present application, the metadata and data of files or directories in the Ceph file system will be briefly described below. Figure 1 It is a schematic diagram of accessing metadata and data based on the Ceph file system provided in an embodiment of the present invention. Refer to Figure 1 , for the Ceph file system, the CephFS client cannot directly access the metadata pool. It needs to send a request to the configured MDS process, and the MDS indirectly completes the modification of the metadata and saves it to the metadata pool. However, the CephFS client can directly read and write objects in the data pool, which means that the modification of the file content does not need to be completed via other processes.

[0033] Currently, there are mainly the following implementation schemes for the CephFS client: The first one is to provide a file system operation API interface in the form of a user-mode module (so) for user processes to call. The user program needs to reference this module when writing, but most file operation programs are written for a general system architecture, so the compatibility of this scheme is relatively poor; The second one is to start a ceph fuse process on the user device, which calls the cephfs API. The user's file operation program still accesses the file system through system calls, and the kernel will automatically forward the request to the fuse process to complete the docking work. However, this scheme will have a serious impact on performance and is not suitable for the production environment; The third one is to be embedded in the operating system in the form of a kernel module (ko), providing a VFS-compatible interface upward to receive requests, and following the RADOS communication protocol downward to complete request forwarding and message response processing. However, although the above method solves the compatibility and performance problems, it requires a relatively high kernel version, and most devices do not meet this requirement.

[0034] Next, the file processing method, device, electronic device, and storage medium provided in the present application will be elaborated in detail through the following various embodiments and their optional solutions.

[0035] Figure 2is a flowchart of a file processing method provided in an embodiment of the present invention. The embodiment of the present invention is applicable to the case of fast file writing based on Ceph file system. The method can be executed by a file processing device, which can be implemented in software and / or hardware and can be integrated in an electronic device with network communication function. Figure 2 As shown, the file processing method in the embodiment of the present invention may include the following steps:

[0036] S210: Determine the to-be-processed file blocks and the to-be-processed write requests from the received to-be-processed messages.

[0037] In this embodiment, Figure 3 Schematic diagram of the architecture of a kernel-mode NFS and CephFS provided in an embodiment of the present invention. Figure 3 In the implementation scheme for CephFS client, considering the compatibility and performance issues of CephFS client, and the problem that the kernel version required by CephFS client is relatively high, but most devices usually do not meet this requirement, NFS server can be used as a proxy for CephFS client to provide services. Among them, NFS server can be deployed at the front end of CephFS client, and provide services through NFS server proxy CephFS client, so that the user-side device does not need to have a higher kernel version, avoiding the inability to provide services effectively due to most devices not meeting this requirement.

[0038] In an optional solution of this embodiment, it can be combined with various optional solutions provided in one or more of the above embodiments. Among them, determining the to-be-processed file blocks and the to-be-processed write requests from the received to-be-processed messages may include the following operations:

[0039] Parse the pending message received from the NFS client, and write the pending file block and pending write request obtained by parsing into the temporarily applied data buffer.

[0040] In this embodiment, Figure 4 Schematic diagram of an NFS server providing services on behalf of a CephFS client provided in an embodiment of the present invention. Figure 2 and Figure 4 , the NFS client can send a pending message to the NFS server through Socket communication, and the pending file blocks and pending write requests carried in the pending message can be obtained by parsing the pending message. The NFS server can temporarily apply for a data buffer as required, and put the parsed pending file blocks and pending write requests into the data buffer.

[0041] S220: Determine whether to enable the write-through mode to perform a write operation on the file block to be processed.

[0042] In this embodiment, refer to Figure 4 , for the Ceph file system, two modes, namely the write-through mode and the cache write mode, can be set to implement the write processing of the file blocks to be processed. The cache write mode supports writing the file to be processed into the general file system cache first, and then flushing the written file blocks to be processed from the general file system cache into the storage device. In contrast, the write-through mode supports directly writing the file blocks to be processed with a specific index structure into the storage device after the index structure conversion.

[0043] In this embodiment, when continuously writing new files to the file system and the pressure is high enough, it will cause the file system to not have enough idle time to flush the cache. After a period of time, the cache will be exhausted. At this time, it is necessary to wait for the previous dirty data to be flushed before allowing new data to continue to be written to the cache, which reduces the response speed. In addition, when each file is written only once and the content will not be read in a short time, the setting of the cache is not very meaningful. Compared with directly writing the data to the backend storage device, the cache write mode also has an additional data copying process, which will reduce the writing efficiency. Especially when the file block size is larger, its impact on the writing efficiency is more obvious. Based on the above situation, it can be seen that the cache write mode and the write-through mode have different adaptabilities to scenarios. In some scenarios, there are some defects in the cache write mode. Therefore, it is necessary to decide whether to enable the write-through mode according to the actual situation to implement the write processing operation of the file blocks to be processed.

[0044] In an alternative solution of this embodiment, it can be combined with each alternative solution provided in one or more of the above embodiments. Among them, determining whether to enable the write-through mode to perform the write processing operation on the file blocks to be processed may include the following steps A1 - A2:

[0045] Step A1: If it is determined that the file block to be processed is written for the first time, determine the block size of the file block to be processed.

[0046] Step A2: If it is determined that the block size of the file block to be processed is greater than the preset block size, select to enable the write-through mode from the write-through mode and the cache write mode to perform the write processing operation on the file block to be processed.

[0047] In this embodiment, Figure 5 is a schematic diagram of the mode switching between cache write and write-through provided in the embodiment of the present invention. Refer to Figure 5, it is possible to determine whether the file block to be processed belongs to the initial write by judging whether the position to be written for writing the file block to be processed in the storage device contains valid data. If the position to be written for writing the file block to be processed in the storage device does not contain valid data, it is determined that the file block to be processed belongs to the initial write; if the position to be written for writing the file block to be processed in the storage device contains valid data, it is determined that the file block to be processed does not belong to the initial write and is very likely to belong to the overwrite write.

[0048] In this embodiment, in the case of overwrite write, due to the possibility of write merging in the CephFS client, it is more efficient to integrate multiple issued write requests and issue them once. At this time, the cached write mode is better than the direct write mode. However, in the case of initial write, there may be no possibility of write merging in the CephFS client. For example, only one write request needs to be issued; or when the file block is large, only the original write request needs to be issued. At this time, the direct write mode is better than the cached write mode. Therefore, if it is determined that the file block to be processed belongs to the initial write, continue with the block size of the file block to be processed; if it is determined that the file block to be processed does not belong to the initial write, directly enable the cached write mode.

[0049] In this embodiment, when the file block to be processed is small, the cost of data replication is low. Using the cached write mode can effectively reduce the response time and also improve the efficiency by merging and issuing. In addition, due to the relatively small overall traffic, the file system has enough free write caches. At this time, the direct write mode should be abandoned and the cached write mode should be used. Therefore, it is possible to determine whether the block size of the file block to be processed is greater than the preset block size. If it is determined to be greater than the preset block size, select and enable the direct write mode from the direct write mode and the cached write mode to perform the write processing operation on the file block to be processed; otherwise, select and enable the cached write mode from the direct write mode and the cached write mode to perform the write processing operation on the file block to be processed.

[0050] In this embodiment, the preset block size can be a configuration value set according to the direct write performance generated by file blocks of different block sizes. For example, this configuration value can be 128K. Because for block sizes below 128K during the mass picture storage test, the direct write hardly improves the write performance, and for block sizes above 128K, the write performance can be improved by 16%-22%.

[0051] S230. If it is determined to enable the direct write mode, perform a structural transformation on the index of the file block to be processed, and control the CephFS client to directly write the file block to be processed with the transformed index structure into the storage device according to the write request to be processed.

[0052] In this embodiment, Figure 6aIt is a schematic diagram of a kvec - form structure provided in an embodiment of the present invention. In the data buffer temporarily applied for by the NFS server, the index of the file block to be processed is given in the kvec form, that is, only the start address and length of the buffer are given here, without specifying the pages it covers. When the CephFS client sends a write request to the backend storage, data needs to be transmitted in units of pages. Therefore, a structural conversion of the index of the file block to be processed is required here to support directly sending the file block to be processed from the buffer applied for by the NFS server to the backend storage device for writing.

[0053] In this embodiment, Figure 6b It is a schematic diagram of a bio_vec - form structure provided in an embodiment of the present invention. The bio_vec array can be selected as the target conversion format for the index structure conversion. Among them, bio_vec represents a paragraph within a page in the form of page index, segment length, and offset within the page, which can meet the requirement of page - by - page transmission. Through the bio_vec array, it is possible to support directly sending the file block to be processed from the buffer applied for by the NFS server to the backend storage device for writing.

[0054] In an alternative solution of this embodiment, it can be combined with each alternative solution provided in one or more of the above - mentioned embodiments. Figure 7 It is a flowchart of a structural conversion of the file block to be processed provided in an embodiment of the present invention. Among them, performing an index structure conversion on the file block to be processed specifically includes the following steps B1 - B2:

[0055] Step B1: Calculate the page address, offset within the page, and segment length of the file block to be processed in the kvec structure, and query the page based on the page address to obtain the page index of the file block to be processed in the kvec structure.

[0056] Step B2: Based on the calculated page index, offset within the page, and segment length, convert the index of the file block to be processed from the kvec structure to the bio_vec array for page - by - page transmission.

[0057] In an alternative solution of this embodiment, it can be combined with each alternative solution provided in one or more of the above - mentioned embodiments. Among them, controlling the CephFS client to directly write the file block to be processed with the index structure conversion into the storage device may include steps C1 - C2:

[0058] Step C1: Through the virtual file system VFS operation interface, send a write request to be processed for the file block to be processed to the CephFS client.

[0059] Step C2: Through the to-be-processed write request, control the CephFS client to directly send the to-be-processed file blocks transformed by the index structure from the NFS server to the storage device for writing.

[0060] In this embodiment, referring to Figure 4 , in the write-through mode enabled, the CephFS client docks with the NFS server. The NFS server can transform the index structure of the to-be-processed file blocks in the buffer, and then issue the to-be-processed write request to the CephFS client through the VFS interface. The CephFS client can bypass the traffic bottleneck brought by the file system cache, and directly send the to-be-processed file blocks transformed by the index structure from the NFS server to the storage device for writing according to the to-be-processed write request.

[0061] In this embodiment, referring to Figure 4 , in the cache write mode enabled, the NFS server issues a write request to the CephFS client through the VFS interface, and the data is written into the file system cache. Then the CephFS client returns an acknowledgement message of successful writing upward. The data will eventually be written into the backend storage in the form of dirty data flushing at an appropriate time.

[0062] Adopting the file processing solution provided in the embodiment of the present application, when continuously writing new files into the file system, each file is written only once and will not be read in a short time, by structuring the index of the to-be-processed file blocks, the CephFS client can support the pass-through of write requests from the NFS server, realize directly writing the to-be-processed file blocks into the storage device, improve the response speed and writing efficiency of the files; and, through the NFS server acting as an agent to provide services for the CephFS client, the problem of overly high kernel version requirements for the CephFS client can be avoided on the premise of solving compatibility and performance problems.

[0063] Figure 8 is a structural block diagram of a file processing device provided in an embodiment of the present invention. The embodiment of the present invention is applicable to the situation of fast file writing based on the Ceph file system. The file processing device can be implemented in software and / or hardware, and can be integrated on an electronic device with network communication functions. As Figure 8 shown, the file processing device in the embodiment of the present invention may include the following: a message processing module 810, a mode determination module 820, and a file write-through module 830. Among them:

[0064] The message processing module 810 is configured to determine the to-be-processed file blocks and the to-be-processed write request from the received to-be-processed messages;

[0065] The mode determination module 820 is configured to determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file blocks;

[0066] A file passthrough module 830, which is configured to, if it is determined to enable the passthrough mode, perform a structural transformation on the index of the to-be-processed file block, and control the CephFS client to directly write the to-be-processed file block with the index structure transformation into a storage device according to the to-be-processed write request; the NFS server is deployed in front of the CephFS client.

[0067] Optionally, based on the above embodiments, the message processing module 810 includes:

[0068] Parse a to-be-processed message received from an NFS client, and write the to-be-processed file block and the to-be-processed write request obtained by the parsing into a data buffer temporarily allocated.

[0069] Optionally, based on the above embodiments, the mode determination module 820 includes:

[0070] If it is determined to write the to-be-processed file block for the first time, determine the block size of the to-be-processed file block;

[0071] If it is determined that the block size of the to-be-processed file block is greater than a preset block size, select to enable the passthrough mode from the passthrough mode and the cache write mode to perform a write processing operation on the to-be-processed file block.

[0072] Optionally, based on the above embodiments, the file passthrough module 830 includes:

[0073] Calculate the page address, the in-page offset, and the segment length of the to-be-processed file block in the kvec structure, and query the page according to the page address to obtain the page index of the to-be-processed file block in the kvec structure;

[0074] Based on the page index, the in-page offset, and the segment length, transform the index of the to-be-processed file block from the kvec structure to a bio_vec array transmitted in units of pages.

[0075] Optionally, based on the above embodiments, the file passthrough module 830 includes:

[0076] Send the to-be-processed write request of the to-be-processed file block to the CephFS client through a virtual file system VFS operation interface;

[0077] Control the CephFS client to directly send the to-be-processed file block with the index structure transformation from the NFS server to a storage device for writing through the to-be-processed write request.

[0078] The file processing device provided in the embodiments of the present invention can execute the file processing method provided in any of the above embodiments of the present invention, and has the corresponding functions and beneficial effects for executing the file processing method. For the specific process, reference can be made to the embodiments of the foregoing file processing method.

[0079] Figure 9 It is a schematic structural diagram of an electronic device provided in the embodiments of the present invention. As Figure 9 shown in the structure, the electronic device provided in the embodiments of the present invention includes: one or more processors 910 and a storage device 920; the processor 910 in the electronic device can be one or more, Figure 9 and here one processor 910 is taken as an example; the storage device 920 is used to store one or more programs; the one or more programs are executed by the one or more processors 910, so that the one or more processors 910 implement the file processing method described in any one of the embodiments of the present invention.

[0080] The electronic device may further include: an input device 930 and an output device 940.

[0081] The processor 910, the storage device 920, the input device 930, and the output device 940 in the electronic device can be connected through a bus or other means, Figure 9 and here taking the connection through the bus as an example.

[0082] The storage device 920 in the electronic device, as a computer-readable storage medium, can be used to store one or more programs, and the programs can be software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the file processing method provided in the embodiments of the present invention. The processor 910 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the storage device 920, that is, implements the file processing method in the above method embodiments.

[0083] The storage device 920 may include a storage program area and a storage data area. Among them, the storage program area can store an operating system and application programs required for at least one function; the storage data area can store data created according to the use of the electronic device, etc. In addition, the storage device 920 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the storage device 920 may further include a memory remotely set relative to the processor 910, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0084] The input device 930 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the electronic device. The output device 940 can include display devices such as a display screen.

[0085] Moreover, when one or more programs included in the above-mentioned electronic device are executed by the one or more processors 910, the programs perform the following operations:

[0086] Determine a to-be-processed file block and a to-be-processed write request from the received to-be-processed message;

[0087] Determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block;

[0088] If it is determined to enable the write-through mode, perform a structural transformation on the index of the to-be-processed file block, and control the CephFS client to directly write the to-be-processed file block with the index structure transformation into the storage device according to the to-be-processed write request.

[0089] Of course, those skilled in the art can understand that when one or more programs included in the above-mentioned electronic device are executed by the one or more processors 910, the programs can also perform related operations in the file processing method provided in any embodiment of the present invention.

[0090] An embodiment of the present invention provides a computer-readable medium, on which a computer program is stored. When the program is executed by a processor, it is used to execute a file processing method, and the method includes:

[0091] Determine a to-be-processed file block and a to-be-processed write request from the received to-be-processed message;

[0092] Determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block;

[0093] If it is determined to enable the write-through mode, perform a structural transformation on the index of the to-be-processed file block, and control the CephFS client to directly write the to-be-processed file block with the index structure transformation into the storage device according to the to-be-processed write request.

[0094] Optionally, when the program is executed by a processor, it can also be used to execute the file processing method provided in any embodiment of the present invention.

[0095] The computer storage medium of an embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium that includes or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0097] The program code included on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0098] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0099] In the description of this specification, the description of reference terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0100] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments may be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A file processing method, characterized in that, applied to an NFS server, the NFS server is deployed in front of a CephFS client, and the method includes: Determining a to-be-processed file block and a to-be-processed write request from the received to-be-processed message; Determining whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block; If it is determined to enable the write-through mode, perform a structural transformation on the index of the to-be-processed file block, and control the CephFS client to directly write the to-be-processed file block with the index structure transformation into a storage device according to the to-be-processed write request; Among them, the determining whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block includes: If it is determined to write the to-be-processed file block for the first time, determining the block size of the to-be-processed file block; If it is determined that the block size of the to-be-processed file block is greater than a preset block size, select to enable the write-through mode from the write-through mode and the cache write mode to perform a write processing operation on the to-be-processed file block; Among them, the performing a structural transformation on the index of the to-be-processed file block includes: Selecting a bio_vec array as the target transformation format for the index structure transformation, where the bio_vec array represents a paragraph within a page in the form of a page index, a segment length, and an offset within the page.

2. The method according to claim 1, characterized in that, Determining a to-be-processed file block and a to-be-processed write request from the received to-be-processed message includes: Performing message parsing on the received to-be-processed message from the NFS client, and writing the parsed to-be-processed file block and to-be-processed write request into a temporarily allocated data buffer.

3. The method according to claim 1, characterized in that, Performing a structural transformation on the index of the to-be-processed file block includes: Calculating the page address, the offset within the page, and the segment length of the to-be-processed file block in the kvec structure, and querying the page according to the page address to obtain the page index of the to-be-processed file block in the kvec structure; Based on the page index, the offset within the page, and the segment length, transforming the index of the to-be-processed file block from the kvec structure to a bio_vec array transmitted in units of pages.

4. The method according to claim 1, characterized in that, Controlling the CephFS client to directly write the to-be-processed file block with the index structure transformation into a storage device according to the to-be-processed write request includes: Issuing the to-be-processed write request of the to-be-processed file block to the CephFS client through a virtual file system VFS operation interface; Controlling the CephFS client to directly send the to-be-processed file block with the index structure transformation from the NFS server to a storage device for writing through the to-be-processed write request.

5. A file processing device, characterized in that, configured in an NFS server, the NFS server is deployed in front of a CephFS client, and the device includes: A message processing module, configured to determine a to-be-processed file block and a to-be-processed write request from the received to-be-processed message; A mode determination module, configured to determine whether to enable the write-through mode to perform a write processing operation on the to-be-processed file block; A file passthrough module, which is used to perform a structural transformation on the index of the to-be-processed file block if it is determined to enable the passthrough mode, and control the CephFS client to directly write the to-be-processed file block with the index structure transformation into the storage device according to the to-be-processed write request The mode determination module includes: if it is determined to write the to-be-processed file block for the first time, determining the block size of the to-be-processed file block; if it is determined that the block size of the to-be-processed file block is greater than the preset block size, selecting to enable the passthrough mode from the passthrough mode and the cache write mode to perform a write processing operation on the to-be-processed file block The file passthrough module is specifically used to select a bio_vec array as the target transformation format for the index structure transformation, where the bio_vec array represents a paragraph within a page in the form of a page index, a segment length, and an offset within the page 6. The apparatus according to claim 5, wherein, The file passthrough module includes: Issuing a to-be-processed write request for the to-be-processed file block to the CephFS client through a virtual file system VFS operation interface; Controlling the CephFS client to directly send the to-be-processed file block with the index structure transformation from the NFS server to the storage device for writing through the to-be-processed write request 7. An electronic device, wherein, including: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the file processing method according to any one of claims 1-4 8. A computer-readable storage medium, on which a computer program is stored, wherein, The program, when executed by a processor, implements the file processing method according to any one of claims 1-4

Images