File operation method and device, computer device, readable storage medium and program product

Abstract

The application relates to a file operation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: initializing a first negotiation message; converting the first negotiation message by using a target byte sequence and sending the first negotiation message to a data processor; receiving a second negotiation message returned by the data processor; analyzing the second negotiation message based on a host byte sequence; when the value of a second byte sequence negotiation identifier field obtained by analysis is a target value, converting a data processor byte sequence field according to the target byte sequence to obtain a data processor byte sequence; wherein the target value is equal under a big-endian byte sequence and a little-endian byte sequence; and performing byte sequence negotiation when the data processor byte sequence and the host byte sequence are inconsistent. The method can prevent cross-CPU architecture compatibility problems and ensure the accuracy of file operation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a file operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] A DPU (Data Processing Unit) is a dedicated processor built for data centers, providing accelerated processing for network, storage, and other services. With the rise of technologies such as AI and big data, increasingly higher demands are being placed on data centers, and the market for DPUs is expected to grow significantly.

[0003] The file system is one of the core functions of storage, providing a standard interface for upper-layer applications to access storage resources. Traditionally, the file system is one of the core components provided by the operating system kernel. Kernel file systems typically store data on several physical disks, thus limiting their capacity. Moreover, being in kernel mode makes development and debugging difficult. Most importantly, a file system failure can render the entire operating system unusable. Therefore, user-space file systems emerged. With user-space file systems, developers can develop and debug file systems in user space, greatly improving development efficiency. Furthermore, the core logic of the file system runs in user space, reducing the risk of kernel crashes. In recent years, with the rise of distributed storage systems and big data applications, user-space file systems can act as proxies for distributed storage systems on the server side, providing POSIX (Portable Operating System Interface of UNIX) semantic access interfaces.

[0004] In traditional technologies, user-space file systems are implemented based on FUSE (Filesystem in userspace). The FUSE framework includes two core components: one is the fuse kernel module located in the Linux kernel, which receives file system requests from applications, encapsulates them using the FUSE protocol, and forwards them to the user-space file system service process; the other is the LIBFUSE library, which supports the development of user-space file system services. It obtains file system requests from the fuse kernel module and writes the request responses from the user-space file system service back to the fuse kernel module.

[0005] Typically, the fuse kernel module and the user-space file system service run on the same operating system. However, in a DPU scenario, the fuse kernel module runs on the host machine's operating system, while the file system service runs on the DPU's operating system. If the host machine and the DPU have heterogeneous CPU architectures and their CPU byte order is inconsistent—for example, the host machine uses big-endian byte order while the DPU uses little-endian byte order—then all multi-byte fields in the file system operation structures will be incorrectly parsed by the file system service process on the DPU, leading to operation failure. Summary of the Invention

[0006] Therefore, it is necessary to provide a file operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can ensure the accuracy of file operations in response to the above-mentioned technical problems.

[0007] Firstly, this application provides a file manipulation method applied to a host, the method comprising:

[0008] Initialize the first negotiation message, which includes a host byte order field and a first byte order negotiation identifier field;

[0009] The first negotiation message is converted using the target byte order, and the converted first negotiation message is sent to the data processor;

[0010] The system receives a second negotiation message returned by the data processor when the first byte order negotiation identifier field is a target value. The second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field.

[0011] The second negotiation message is parsed based on the host byte order;

[0012] If the value of the second byte order negotiation identifier field obtained through parsing is the target value, the data processor byte order field is converted according to the target byte order to obtain the data processor byte order; wherein, the target value is equal in both big-endian and little-endian byte order;

[0013] If the byte order of the data processor and the byte order of the host are inconsistent, the host is configured to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or the byte order of the host and the data processor is configured to be the byte order of the data processor; or the byte order of the host and the data processor is configured to be the byte order of the host.

[0014] In one embodiment, the target byte order is little-endian or big-endian.

[0015] In one embodiment, the method further includes:

[0016] The abstraction layer component converts the first operation flag in the first file operation message into a first common abstract operation flag. The first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstraction layer component, and to determine the operation type based on the second operation flag.

[0017] In one embodiment, the method further includes:

[0018] The system receives a first file operation result message sent by the data processor, the first file operation result message carrying a second common abstract operation flag;

[0019] The second public abstract operation flag is converted into a third operation flag using the abstraction layer component;

[0020] The operation type is determined based on the third operation flag.

[0021] Secondly, this application also provides a file operation method applied to a data processor, the method comprising:

[0022] The first negotiation message sent by the host is received, and the first negotiation message includes a first byte order negotiation identifier field and a host byte order field.

[0023] The first negotiation message is parsed based on the data processor byte order;

[0024] If the value of the first byte order negotiation identifier field obtained through parsing is the target value, the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or the byte order of the host and the data processor is configured to be the data processor byte order; or the byte order of the host and the data processor is configured to be the host byte order, wherein the target value is equal in both big-endian and little-endian byte order.

[0025] In one embodiment, the method further includes:

[0026] If the value of the first byte order negotiation identifier field obtained by conversion is the target value, the second byte order negotiation identifier field is set to the target value, the data processor byte order field is set to the data processor byte order, and the byte order of the second negotiation message is converted to the target byte order;

[0027] Send the second negotiation message to the host.

[0028] In one embodiment, the target byte order is little-endian or big-endian.

[0029] In one embodiment, the method further includes:

[0030] The abstraction layer component converts the first operation result flag in the first file operation result message into a second common abstract operation flag. The second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstraction layer component, and to determine the operation type based on the third operation flag.

[0031] In one embodiment, the method further includes:

[0032] The receiver sends a first file operation message, which carries a first common abstract operation flag.

[0033] The first public abstract operation flag is converted into a second operation flag using the abstraction layer component;

[0034] The operation type is determined based on the second operation flag.

[0035] Thirdly, this application also provides a file manipulation device applied to a host computer, the device comprising:

[0036] An initialization module is used to initialize a first negotiation message, which includes a host byte order field.

[0037] The first conversion module is used to convert the first negotiation message according to the target byte order and send the converted first negotiation message to the data processor.

[0038] The first receiving module is configured to receive a second negotiation message returned by the data processor based on the first negotiation message, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field.

[0039] The second conversion module is used to parse the second negotiation message based on the host byte order;

[0040] The negotiation determination module is used to convert the data processor byte order field according to the target byte order to obtain the data processor byte order when the value of the parsed second byte order negotiation identifier field is the target value; wherein, the target value is equal in both big-endian and little-endian byte order;

[0041] The first byte order negotiation module is configured to, when the byte order of the data processor and the byte order of the host are inconsistent, configure the host to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or configure the byte order of the host and the data processor to be the byte order of the data processor; or configure the byte order of the host and the data processor to be the byte order of the host.

[0042] Fourthly, this application also provides a file manipulation apparatus for use in a data processor, the apparatus comprising:

[0043] The second receiving module is used to receive a first negotiation message sent by the host, the first negotiation message including a host byte order field and a first byte order negotiation identifier field;

[0044] The third conversion module is used to parse the first negotiation message based on the data processor byte order;

[0045] The second negotiation determination module is configured to, when the value of the parsed first byte order negotiation identifier field is the target value, configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order, wherein the target value is equal in both big-endian and little-endian byte order.

[0046] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0047] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0048] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0049] The aforementioned file operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product perform byte order negotiation before file operation. Specifically, a first negotiation message is initialized, converted to the target byte order, and the converted first negotiation message is sent to the data processor. The data processor returns a second negotiation message based on the first negotiation message. The second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field. The second negotiation message is parsed based on the host byte order. If the value of the parsed second byte order negotiation identifier field is the target value, that is, if the data processor also supports byte order negotiation, the data processor byte order field is converted to the target byte order to obtain the data processor byte order. Then, the host byte order and the data processor byte order are compared. If they are inconsistent, the byte order of the two is agreed upon. This includes configuring the host to perform byte order conversion on the first file operation to be sent based on the data processor byte order; or configuring the byte order of both the host and the data processor to be the data processor byte order; or configuring the byte order of both the host and the data processor to be the host byte order, so that both the host and the data processor can accurately parse the file operation, thereby ensuring the accuracy of the operation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a diagram illustrating the application environment of a file operation method in one embodiment;

[0052] Figure 2 This is a flowchart illustrating a file operation method in one embodiment;

[0053] Figure 3 This is a flowchart illustrating a file operation method in another embodiment;

[0054] Figure 4 This is a structural block diagram of a file operation device in one embodiment;

[0055] Figure 5 This is a structural block diagram of the file operation device in another embodiment;

[0056] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] Specifically, in combination Figure 1 As shown, Figure 1 This is a framework diagram of a DPU scenario in one embodiment. In this embodiment, the fuse kernel module runs on the host machine's operating system, while the user-space file system service runs on the DPU's operating system. The host machine runs a Linux operating system, but in a cloud computing environment, it also supports other operating systems such as Windows.

[0059] The FUSE (Filesystem in userspace) protocol was not designed with byte order in mind. The FUSE protocol defines dozens of file system operations; taking the write operation as an example, its protocol format is shown in Table 1 below:

[0060] Table 1: FUSE WRITE Operation Protocol Format

[0061]

[0062] The `fuse_in_header` describes the FUSE header information present in all FUSE requests, while `fuse_write_in` describes basic information about a WRITE operation, including the file being operated on, the offset and length of the write operation within the file, etc. The protocol definition for the WRITE operation described above uses host byte order, and the Linux kernel's `fuse` module also uses host byte order to assign values ​​to structures such as `fuse_in_header` and `fuse_write_in` when generating the FUSE request description message for the WRITE operation. The DPU obtains the FUSE message data, including `fuse_in_header` and `fuse_write_in`, through the hardware-implemented virtio-fs channel between itself and the Host Machine, and then hands it over to the user-space file system service on the DPU to complete the WRITE request. If the Host Machine and DPU have heterogeneous CPU architectures and their CPU byte order is inconsistent, such as the Host Machine using big-endian byte order and the DPU using little-endian byte order, then all multi-byte fields in the fuse_in_header and fuse_write_in structures will be incorrectly parsed by the file system service process on the DPU, causing the WRITE operation to fail and potentially causing other undefined errors.

[0063] The FUSE protocol was not designed with compatibility in mind for different CPU architectures within the same operating system. Specifically, the FUSE protocol heavily relies on the Linux kernel file system programming interface, and the Linux kernel file system has developed compatibility issues with different CPU architectures over its long evolution. The severity of this problem can be illustrated by taking the OPEN operation defined in the FUSE protocol as an example:

[0064] Table 2: FUSE OPEN Operation Protocol Format

[0065]

[0066] The `flags` field of `fuse_open_in` describes how files are opened, but these flags have different values ​​even across different CPU architectures within the same operating system. For example, for x86 and ARM64 architectures:

[0067] Table 3: FUSE OPEN Operation Protocol Format

[0068]

[0069] by Figure 1 Taking the architecture as an example, if the Host Machine is an x86 architecture and the DPU is an ARM64 architecture, when the Host Machine's fuse kernel module constructs the OPEN operation request for the FUSE protocol, if the upper-layer application needs to open the file in DIRECT IO mode, the fuse kernel module will fill the flags field of fuse_open_in with O_DIRECT (00040000) under the x86 architecture. However, since the user-space file system service on the DPU is based on the ARM64 architecture, it will parse the 00040000 value in the flags field of fuse_open_in into O_LARGEFILE (0400000) under the ARM64 architecture. At this time, a serious problem occurs: the user cannot open the file in the correct way. At best, the file opening will fail; at worst, it will lead to various undefined behaviors.

[0070] File system services are a core function provided by the DPU. The two defects mentioned above have severely impacted the file system unloading service provided by the DPU, making it almost unusable in scenarios where the host machine and DPU have heterogeneous CPUs. In the cloud computing field, customers can freely choose the CPU architecture and operating system type of cloud servers; therefore, the aforementioned defects will also severely affect the DPU's ability to provide file system unloading services in the cloud computing domain.

[0071] Therefore, this application provides a file manipulation method that can be applied to, for example... Figure 1 The application environment shown.

[0072] The FUSE protocol defines dozens of file system operations. The host machine's fuse kernel module needs to describe the file system operations initiated by the application using the FUSE protocol and encapsulate them into FUSE messages to send to the file system service on the DPU. If the host machine and the DPU are heterogeneous CPUs and the byte order used by the CPUs is inconsistent, there will be a byte order problem. However, if the byte order of the host machine and the DPU is consistent, there is no need to perform byte order conversion of the FUSE messages.

[0073] In the current FUSE specification, the Host Machine's fuse kernel module and the DPU's user-space file system service negotiate initial file system features through fuse_init_in and fuse_init_out in the FUSE protocol. Fuse_init_in is passed from the Host Machine to the DPU, and fuse_init_out is passed from the DPU to the Host Machine.

[0074] Table 4: Protocol Format of FUSE Feature Negotiation Message

[0075]

[0076] To address the byte order issue of FUSE messages in heterogeneous CPU scenarios, this embodiment extends the FUSE protocol:

[0077] 1) Add two fields to the fuse_init_in message to obtain the first negotiation message. The two fields to be added are the first byte order negotiation identifier field uint16_t nego_endianness and the host byte order field uint16_tendianness.

[0078] 2) Add two fields to the fuse_init_out message to obtain the second negotiation message. The two fields to be added are the second byte order negotiation identifier field uint16_t endianness_negoed and the data processor byte order field uint16_t endianness.

[0079] In one exemplary embodiment, such as Figure 2 As shown, a file operation method is provided, which can be applied to... Figure 1 Taking the host machine as an example, the explanation includes the following steps 202 to 212. Wherein:

[0080] S202: Initialize the first negotiation message, which includes a host byte order field and a first byte order negotiation identifier field.

[0081] The first negotiation message is fuse_init_in, which adds the first byte order negotiation identifier field uint16_t nego_endianness and the host byte order field uint16_t endianness.

[0082] The first byte order negotiation identifier field, uint16_t nego_endianness, is used to indicate whether the host supports byte order negotiation. If the value of the first byte order negotiation identifier field is the target value, it means that the host supports byte order negotiation; otherwise, the host does not support byte order negotiation.

[0083] The target value is equal in both big-endian and little-endian byte order. Optionally, the target value is set to 0xdededede. This target value is to ensure that the first byte order negotiation flag field nego_endianness will have the same value regardless of whether the CPU is big-endian or little-endian. Thus, the same target value can be obtained regardless of the byte order used for parsing. After obtaining this target value, it can be determined that the other end supports byte order negotiation.

[0084] In this context, byte order refers to the storage order of data types exceeding one byte in a storage device (such as memory). Big-endian byte order means that the most significant byte is stored at the lower address of the storage device, and the least significant byte is stored at the higher address of the storage device; little-endian byte order means that the least significant byte is stored at the lower address of the storage device, and the most significant byte is stored at the higher address of the storage device.

[0085] S204: Convert the first negotiation message using the target byte order and send the converted first negotiation message to the data processor.

[0086] In one optional embodiment, the target byte order is little-endian or big-endian. Specifically, in this application, the byte order of the first and second negotiation messages is fixed as little-endian or big-endian, rather than using host byte order or data processor byte order.

[0087] For example, this application can use little-endian byte order as the target byte order, in which case the host byte order may be big-endian. Whether subsequent FUSE messages undergo byte order conversion depends on the byte order negotiation result of fuse_init_in and fuse_out_init.

[0088] After extending the FUSE protocol as described above, the byte order issue between the Host Machine and the Data Processor (DPU) can be correctly handled. The Host Machine initializes the first negotiation message `fuse_init_in` with the following configuration: `nego_endianness = 0xdededede; endianness = the host machine's byte order`. Then, the initialized first negotiation message is converted to the target byte order and sent to the Data Processor.

[0089] S206: Receive a second negotiation message returned by the data processor when the first byte order negotiation identifier field is the target value. The second negotiation message includes the data processor byte order field and the second byte order negotiation identifier field.

[0090] After receiving the first negotiation message, the data processor parses the message based on its byte order and reads the value of the first byte order negotiation field. If the value of the first byte order negotiation field is not the target value (e.g., not 0xdededede), it indicates that the host machine's fuse kernel module does not support endianness negotiation. Subsequently, the host machine and the data processor DPU will not negotiate any byte order. User-space file system services based on FUSE will encounter errors when the byte order of the host machine and the DPU is inconsistent.

[0091] If the first byte order negotiation field in the first negotiation message has the target value, such as 0xdededede, it indicates that the host machine's fuse kernel module supports byte order endianness negotiation. The user-space file system service on the data processor (DPU) side generates a second negotiation message. In this second negotiation message, the second byte order negotiation field `endianness_negoed` in `fuse_init_out` is set to the target value, such as 0xdededede, to indicate that the data processor (DPU) also supports byte order negotiation. The `endianness` field in the second negotiation message `fuse_init_out` is then set to the data processor's byte order. Conversely, if the data processor (DPU) does not support byte order negotiation, the `endianness_negoed` field in the second negotiation message `fuse_init_out` is set to a non-target value. Finally, the second negotiation message `fuse_init_out` is converted to the target byte order and passed to the host machine's fuse kernel module.

[0092] S208: Parse the second negotiation message based on the host byte order.

[0093] After receiving the second negotiation message, the host parses the second negotiation message according to the host byte order in order to read the contents of each field in the second negotiation message.

[0094] S210: If the value of the parsed second byte order negotiation identifier field is the target value, convert the data processor byte order field according to the target byte order to obtain the data processor byte order; wherein, the target value is equal in both big-endian and little-endian byte order.

[0095] If the value of the second byte order negotiation identifier field in the parsed second negotiation message is not the target value, it means that the data processor does not support byte order negotiation. Therefore, the host machine and the data processor DPU will not negotiate any byte order. The user-space file system service based on FUSE will encounter an error when the byte order of the host machine and the DPU is inconsistent.

[0096] If the value of the second byte order negotiation identifier field in the parsed second negotiation message is the target value, it means that the data processor supports byte order negotiation. In this case, the data processor byte order field of the parsed second negotiation message is converted according to the target byte order to obtain the correct data processor byte order for subsequent processing.

[0097] S212: In the case that the host byte order and the data processor byte order are inconsistent, configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; or configure the host and the data processor byte order to be both the data processor byte order; or configure the host and the data processor byte order to be both the host byte order.

[0098] If the host byte order and the data processor byte order are consistent, then subsequent host machine operations on other file system operations defined by the FUSE protocol do not require byte order conversion.

[0099] If the host byte order and the data processor byte order are inconsistent, byte order conversion is required. This includes the following scenarios: configuring the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; configuring both the host and the data processor byte order to be in the data processor byte order; or configuring both the host and the data processor byte order to be in the host byte order.

[0100] In other words, if both the host and the data processor support byte order negotiation, a common byte order can be agreed upon. For example, in subsequent messages, whether sent or received, both the host and the data processor can use either the data processor's byte order or the host's byte order. Alternatively, subsequent messages can use the other party's byte order when sent, but use their own byte order when received, thus avoiding byte order confusion.

[0101] In one alternative embodiment, the configuration host performs byte order conversion on received or sent messages based on the data processor's byte order, while the data processor does not need to perform byte order conversion.

[0102] In one optional embodiment, both the host and the data processor are configured with host byte order. That is, the data processor performs byte order conversion on received or sent messages according to host byte order, while the host processes messages according to its own byte order without byte order conversion. Specifically, the data processor, after receiving the first negotiation message and determining that the first byte order negotiation identifier field is the target value, parses the host byte order field in the first negotiation message using the target byte order to obtain the host byte order. In other words, when both the host and the data processor are configured with host byte order, the host needs to inform the data processor so that the data processor can subsequently perform byte order conversion on received or sent messages according to host byte order.

[0103] In one optional embodiment, the configuration host performs byte order conversion on the first file operation message to be sent based on the data processor's byte order; the configuration data processor performs byte order conversion on the first file operation result message to be sent based on the host's byte order. In this way, the other party's byte order is used when sending, while its own byte order is used when receiving, thereby avoiding byte order confusion.

[0104] The above file operation method involves negotiating the byte order before the file operation. Specifically, a first negotiation message is initialized, converted to the target byte order, and then sent to the data processor. The data processor returns a second negotiation message based on the first negotiation message. The second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field. The second negotiation message is parsed based on the host byte order. If the value of the parsed second byte order negotiation identifier field is the target value, that is, if the data processor also supports byte order negotiation, the data processor byte order field is converted to the target byte order to obtain the data processor byte order. Then, the host byte order and the data processor byte order are compared. If they are inconsistent, the byte order of the two is agreed upon. This includes configuring the host to perform byte order conversion for the first file operation to be sent based on the data processor byte order; or configuring the byte order of both the host and the data processor to be the data processor byte order; or configuring the byte order of both the host and the data processor to be the host byte order. This ensures that both the host and the data processor can accurately parse the file operation, thereby guaranteeing the accuracy of the operation. Furthermore, other FUSE operations only occur on the Host. When the machine and DPU byte order are inconsistent, conversion is performed without affecting the performance of non-heterogeneous CPU architectures. Typically, network byte order conversion will incur a slight performance overhead.

[0105] In one optional embodiment, the method further includes: converting a first operation flag in a first file operation message into a first common abstract operation flag via an abstraction layer component, wherein the first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag via the abstraction layer component, and determining the operation type based on the second operation flag.

[0106] In one optional embodiment, the method further includes: receiving a first file operation result message sent by a data processor, the first file operation result message carrying a second public abstract operation flag; converting the second public abstract operation flag into a third operation flag through an abstraction layer component; and determining the operation type based on the third operation flag.

[0107] To address the file system compatibility issue between different CPU architectures within the same operating system, this embodiment introduces a new common intermediate abstraction layer built upon the operating system's file system interface in the FUSE protocol. This layer shields the differences in file system interfaces across different CPU architectures within the same operating system. Continuing with the OPEN operation defined in the FUSE protocol as an example, a common, architecture-independent definition is abstracted for the flag parameters required for the file open operation:

[0108] Table 5: Definition of FUSE OPEN Operation Common Flags

[0109]

[0110] Taking a Host Machine of x86 and a DPU of ARM64 as an example:

[0111] The host machine's fuse kernel module opens a file, assuming its flags are O_DIRECT (00040000) and O_LARGEFILE (00100000). The abstraction layer component uses the first operation flag in the first file operation message to convert it into the first common abstract operation flag, that is, using the common abstract flag defined above for the FUSE OPEN operation, which will be converted into hexadecimal 0xc.

[0112] The data processor (DPU) is an ARM64, with O_DIRECT value of 0200000 and O_LARGEFILE value of 0400000. After receiving the FUSE OPEN operation message, the file system service on the DPU side parses it and finds that its file open flag is 0xc. Thus, it knows that the actual file flags are FUSE_OPEN_DIRECT and FUSE_OPEN_O_LARGEFILE. After translation and conversion by the aforementioned abstraction layer component, the DPU side's ARM64's O_DIRECT (0200000) and O_LARGEFILE (0400000) are used as the actual DPU side file open flags, so that the file will be opened correctly according to the actual intent of the Host Machine side application.

[0113] Furthermore, if the host receives a first file operation result message sent by the data processor, and this first file operation result message carries a second common abstract operation flag, then the host can convert the second common abstract operation flag into a third operation flag through the abstraction layer component, so that the host can accurately determine the operation type.

[0114] As mentioned above, a similar transformation needs to be introduced in the new public abstraction layer component of the FUSE protocol for all file system operations in the FUSE protocol that have cross-CPU architecture compatibility issues. This will ensure that the actual file operation intentions of applications on the host machine are correctly processed by the file system service of the data processor DPU.

[0115] In one exemplary embodiment, such as Figure 3 As shown, a file operation method is provided, which can be applied to... Figure 1 Taking the data processor (DPU) as an example, the explanation includes steps 302 to 308. Wherein:

[0116] S302: Receive the first negotiation message sent by the host. The first negotiation message includes a first byte order negotiation identifier field and a host byte order field.

[0117] The limitations on the first byte order negotiation identifier field can be found above.

[0118] S304: Convert the first negotiation message based on the data processor byte order.

[0119] S306: If the value of the first byte order negotiation identifier field obtained by conversion is the target value, configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or configure the byte order of both the host and the data processor to be the data processor byte order; or configure the byte order of both the host and the data processor to be the host byte order, wherein the target value is equal in both big-endian and little-endian byte order.

[0120] Optionally, the target byte order can be either little-endian or big-endian. The limitations regarding the target byte order are also explained above and will not be repeated here.

[0121] After receiving the first negotiation message, the data processor converts the first negotiation message based on the data processor's byte order and reads the value of the first byte order negotiation field in the converted first negotiation message. If the value of the first byte order negotiation field in the first negotiation message is not the target value, for example, not 0xdededede, it means that the host machine's fuse kernel module does not support byte order endianness negotiation. Subsequently, the host machine and the data processor DPU will not perform any byte order negotiation. User-space file system services based on FUSE will encounter errors when the byte order of the host machine and the DPU is inconsistent.

[0122] If the value of the first byte order negotiation field in the first negotiation message is the target value, such as 0xdededede, it means that the host machine's fuse kernel module supports byte order endianness negotiation.

[0123] Optionally, in this embodiment, the byte order configuration can be on the data processor side. That is, when both the host and the data processor support byte order negotiation, and the host byte order in the host byte order field is inconsistent with the data processor byte order, the byte order on the data processor side can be directly configured to also be the host byte order, meaning that both the host and the data processor process data using the host byte order. Alternatively, the data processor can be configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order, and the host can be configured to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; or the byte order of both the host and the data processor can be configured to be the data processor byte order. When configuring the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order, or when configuring both the host and the data processor to be the data processor byte order, the byte order on the data processor side needs to be sent to the host. The specific sending method can be found below. Furthermore, it should be noted that if the data processor is configured to use the data processor byte order for both the host and the data processor, the host needs to be informed of the data processor byte order used.

[0124] In some optional embodiments, the method further includes: if the value of the first byte order negotiation identifier field is the target value, setting the second byte order negotiation identifier field to the target value, setting the data processor byte order field to the data processor byte order, and converting the byte order of the second negotiation message to the target byte order; and sending the second negotiation message to the host.

[0125] In this embodiment, the user-space file system service on the data processor DPU side generates a second negotiation message, wherein the second byte order negotiation field endianness_negoed of the second negotiation message fuse_init_out is set to the target value, such as 0xdededede, and the byte order field endianness of the second negotiation message fuse_init_out is set to the byte order of the data processor DPU.

[0126] Finally, the data processor converts the byte order of the second negotiation message fuse_init_out to the target byte order and passes it to the host machine's fuse kernel module. This informs the host data processor that it also supports byte order negotiation, so that the host side can perform byte order conversion based on the data processor's byte order and the data processor's byte order configuration result.

[0127] In one optional embodiment, the method further includes: converting a first operation result flag in a first file operation result message into a second common abstract operation flag via an abstraction layer component, wherein the second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag via the abstraction layer component, and determining the operation type based on the third operation flag.

[0128] In one optional embodiment, the method further includes: receiving a first file operation message sent by a host, the first file operation message carrying a first common abstract operation flag; converting the first common abstract operation flag into a second operation flag through an abstraction layer component; and determining the operation type based on the second operation flag.

[0129] This application introduces an abstraction layer component. Before the host and data processor send operation messages, or after receiving operation messages, this abstraction layer component converts the operation flags in the operation messages. For example, before the data processor sends the operation result message, it converts the first operation result flag in the first file operation result message into a second common abstract operation flag. After receiving the first file operation message sent by the host, the abstraction layer component converts the first common abstract operation flag into the second operation flag. Similarly, after the host receives the first file operation result message, the abstraction layer component converts the second common abstract operation flag in the first file operation result message into a third operation flag. Before sending the first file operation message, the host converts the first common abstract operation flag into the second operation flag through the abstraction layer component.

[0130] In the above embodiments, a new public intermediate abstraction layer built on top of the operating system's file system interface is introduced at the FUSE protocol layer. This layer shields the differences in file system interfaces between different CPU architectures of the same operating system. It can correctly and efficiently support the file system services provided by the DPU, avoiding cross-CPU architecture compatibility issues between the Host Machine and the DPU. When providing DPU-based file system unloading services, cloud service providers can freely choose the respective CPU architectures of the Host Machine and the DPU, providing flexibility while ensuring performance.

[0131] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] Based on the same inventive concept, this application also provides a file operation apparatus for implementing the file operation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more file operation apparatus embodiments provided below can be found in the limitations of the file operation method described above, and will not be repeated here.

[0133] In one exemplary embodiment, such as Figure 4 As shown, a file operation device is provided, including: an initialization module 401, a first conversion module 402, a first receiving module 403, a second conversion module 404, a first negotiation determination module 405, and a byte order negotiation module 406, wherein:

[0134] Initialization module 401 is used to initialize the first negotiation message, which includes a host byte order field.

[0135] The first conversion module 402 is used to convert the first negotiation message according to the target byte order and send the converted first negotiation message to the data processor;

[0136] The first receiving module 403 is used to receive a second negotiation message returned by the data processor based on the first negotiation message, the second negotiation message including a data processor byte order field and a second byte order negotiation identifier field;

[0137] The second conversion module 404 is used to parse the second negotiation message based on the host byte order;

[0138] The first negotiation determination module 405 is used to convert the data processor byte order field according to the target byte order to obtain the data processor byte order when the value of the second byte order negotiation identifier field obtained by parsing is the target value; wherein, the target value is equal in both big-endian and little-endian byte order;

[0139] The byte order negotiation module 406 is used to configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order when the data processor byte order and the host byte order are inconsistent; or to configure the byte order of both the host and the data processor to be the data processor byte order; or to configure the byte order of both the host and the data processor to be the host byte order.

[0140] In one alternative embodiment, the target byte order is little-endian or big-endian.

[0141] In one alternative embodiment, the above-described apparatus further includes:

[0142] The first flag conversion module is used to convert the first operation flag in the first file operation message into a first common abstract operation flag through the abstraction layer component. The first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstraction layer component, and to determine the operation type based on the second operation flag.

[0143] In one alternative embodiment, the above-described apparatus further includes:

[0144] The second flag conversion module is used to receive a first file operation result message sent by the data processor, the first file operation result message carrying a second common abstract operation flag; convert the second common abstract operation flag into a third operation flag through the abstraction layer component; and determine the operation type based on the third operation flag.

[0145] In one exemplary embodiment, such as Figure 5 As shown, a file operation device is provided, including: a second receiving module 501, a third conversion module 502, and a second negotiation and determination module 503, wherein:

[0146] The second receiving module 501 is used to receive a first negotiation message sent by the host, the first negotiation message including a host byte order field and a first byte order negotiation identifier field;

[0147] The third conversion module 502 is used to parse the first negotiation message based on the data processor byte order;

[0148] The second negotiation determination module 503 is configured to, when the value of the parsed first byte order negotiation identifier field is the target value, configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order.

[0149] In one optional embodiment, the above apparatus further includes: a second byte order negotiation module 406, configured to, when the value of the first byte order negotiation identifier field obtained by conversion is the target value, set the second byte order negotiation identifier field to the target value, set the data processor byte order field to the data processor byte order, and convert the byte order of the second negotiation message to the target byte order; and send the second negotiation message to the host.

[0150] In one alternative embodiment, the target byte order is little-endian or big-endian.

[0151] In one alternative embodiment, the above-described apparatus further includes:

[0152] The third flag conversion module is used to convert the first operation result flag in the first file operation result message into a second common abstract operation flag through the abstraction layer component. The second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstraction layer component, and to determine the operation type based on the third operation flag.

[0153] In one alternative embodiment, the above-described apparatus further includes:

[0154] The fourth flag conversion module is used to receive a first file operation message sent by the host, the first file operation message carrying a first common abstract operation flag;

[0155] The first public abstract operation flag is converted into a second operation flag using the abstraction layer component;

[0156] The operation type is determined based on the second operation flag.

[0157] Each module in the aforementioned file operation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0158] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 6As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a file operation method.

[0159] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is also provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device may be a data processor (DPU), which may be a DPU chip. Alternatively, the computer device may be a network interface card (NIC) including a data processor. Alternatively, the computer device may be a server including a data processor or a NIC, the server comprising a host and a data processor, the data processor being used to schedule packets to the host or to process them itself, and the host being used to process packets scheduled by the data processor.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A file operation method, characterized in that, Applied to a host, the method includes: Initialize the first negotiation message, which includes a host byte order field and a first byte order negotiation identifier field; The first negotiation message is converted using the target byte order, and the converted first negotiation message is sent to the data processor; The system receives a second negotiation message returned by the data processor when the first byte order negotiation identifier field is a target value. The second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field. The second negotiation message is parsed based on the host byte order; If the value of the second byte order negotiation identifier field obtained through parsing is the target value, the data processor byte order field is converted according to the target byte order to obtain the data processor byte order; wherein, the target value is equal in both big-endian and little-endian byte order; If the byte order of the data processor and the byte order of the host are inconsistent, the host is configured to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or the byte order of both the host and the data processor is configured to be the byte order of the data processor; or the byte order of both the host and the data processor is configured to be the byte order of the host. The abstraction layer component converts the first operation flag in the first file operation message into a first common abstract operation flag. The first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstraction layer component, and to determine the operation type based on the second operation flag.

2. The method according to claim 1, characterized in that, The target byte order is either little-endian or big-endian.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The system receives a first file operation result message sent by the data processor, the first file operation result message carrying a second common abstract operation flag; The second public abstract operation flag is converted into a third operation flag using the abstraction layer component; The operation type is determined based on the third operation flag.

4. A file operation method, characterized in that, Applied to a data processor, the method includes: The first negotiation message sent by the host is received, and the first negotiation message includes a first byte order negotiation identifier field and a host byte order field. The first negotiation message is parsed based on the data processor byte order; If the value of the first byte order negotiation identifier field obtained by parsing is the target value, the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or the byte order of the host and the data processor is configured to be the data processor byte order; or the byte order of the host and the data processor is configured to be the host byte order, wherein the target value is equal in both big-endian and little-endian byte order; The abstraction layer component converts the first operation result flag in the first file operation result message into a second common abstract operation flag. The second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstraction layer component, and to determine the operation type based on the third operation flag.

5. The method according to claim 4, characterized in that, The method further includes: If the value of the first byte order negotiation identifier field obtained by parsing is the target value, the second byte order negotiation identifier field is set to the target value, the data processor byte order field is set to the data processor byte order, and the byte order of the second negotiation message is converted to the target byte order. Send the second negotiation message to the host.

6. The method according to claim 5, characterized in that, The target byte order is either little-endian or big-endian.

7. The method according to claim 4 or 5, characterized in that, The method further includes: The receiver sends a first file operation message, which carries a first common abstract operation flag. The first public abstract operation flag is converted into a second operation flag using the abstraction layer component; The operation type is determined based on the second operation flag.

8. A file operation device, characterized in that, Applied to a host computer, the device includes: An initialization module is used to initialize a first negotiation message, which includes a host byte order field. The first conversion module is used to convert the first negotiation message according to the target byte order and send the converted first negotiation message to the data processor. The first receiving module is configured to receive a second negotiation message returned by the data processor based on the first negotiation message, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identifier field. The second conversion module is used to parse the second negotiation message based on the host byte order; The negotiation determination module is used to convert the data processor byte order field according to the target byte order to obtain the data processor byte order when the value of the parsed second byte order negotiation identifier field is the target value; wherein, the target value is equal in both big-endian and little-endian byte order; The first byte order negotiation module is configured to, when the byte order of the data processor and the byte order of the host are inconsistent, configure the host to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or configure the byte order of both the host and the data processor to be the byte order of the data processor; or configure the byte order of both the host and the data processor to be the byte order of the host. The first flag conversion module is used to convert the first operation result flag in the first file operation result message into a second common abstract operation flag through the abstraction layer component. The second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstraction layer component, and to determine the operation type based on the third operation flag.

9. A file operation device, characterized in that, Applied to a data processor, the device includes: The second receiving module is used to receive a first negotiation message sent by the host, the first negotiation message including a host byte order field and a first byte order negotiation identifier field; The third conversion module is used to parse the first negotiation message based on the data processor byte order; The second negotiation determination module is configured to, when the value of the parsed first byte order negotiation identifier field is the target value, configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order, wherein the target value is equal in both big-endian and little-endian byte order; The third flag conversion module is used to convert the first operation result flag in the first file operation result message into a second common abstract operation flag through the abstraction layer component. The second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstraction layer component, and to determine the operation type based on the third operation flag.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images