File operation method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a file operation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: initializing a first negotiation message; the target byte order is adopted to convert the first negotiation message, and the converted first negotiation message is sent to a data processor; receiving a second negotiation message returned by the data processor; analyzing the second negotiation message based on the host byte order; when the value of the second byte order negotiation identification field obtained through analysis is a target value, converting the data processor byte order field according to a target byte order to obtain a data processor byte order; wherein the target values are equal under the large-end byte order and the small-end byte order; and performing byte order negotiation under the condition that the byte order of the data processor is inconsistent with the byte order of the host. By adopting the method, the problem of cross-CPU architecture compatibility is avoided, and the accuracy of file operation can be ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a file operation method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] DPU (Data Processing Unit) is a dedicated processor built for data centers, providing accelerated processing of network, storage and other services. With the rise of technologies such as AI and big data, data centers are facing higher and higher requirements, and the market for DPU will continue to grow.

[0003] The file system is one of the core functions of storage, providing a standard interface for upper-level applications to access storage resources. Traditionally, the file system is one of the core components provided by the operating system kernel. The kernel file system usually stores data on several physical disks, so the capacity of the file system is relatively limited. In addition, it is in the kernel state of the operating system, so development and debugging are difficult. The most important thing is that the failure of the file system may cause the entire operating system to be unavailable. Therefore, the user-state file system came into being. With the help of the user-state file system, developers can develop and debug the file system in the user state, greatly improving development efficiency. The core logic of the file system runs in the user state, reducing the risk of kernel crashes. In addition, with the rise of distributed storage systems, big data and other application scenarios in recent years, the user-state file system can be deployed on the server side as a proxy for the distributed storage system, providing an access interface with POSIX (Portable Operating System Interface of UNIX) semantics.

[0004] In traditional technology, user-space file systems are all implemented based on FUSE (Filesystem in userspace). The FUSE framework includes two core components: one is the fuse kernel module located in the Linux kernel, which receives file system requests from applications, encapsulates them with the FUSE protocol, and forwards them to the user-space file system service process; the other is the LIBFUSE library, which is used to support the development of user-space file system services. It obtains file system requests from the fuse kernel module and writes the request responses of the user-space file system service back to the fuse kernel module.

[0005] Usually, the fuse kernel module and the user-mode file system service run on the same operating system. However, in the DPU scenario, the fuse kernel module runs on the operating system of the host machine, and the file system service runs on the operating system of the DPU. If the host machine and the DPU are heterogeneous CPU architectures and their CPU byte orders are inconsistent, for example, the host machine uses big-endian byte order and the DPU uses little-endian byte order, then all multi-byte fields in the file system operation structure will be incorrectly parsed by the file system service process on the DPU, resulting in operation failure. Summary of the invention

[0006] Based on this, it is necessary to provide a file operation method, apparatus, computer equipment, computer-readable storage medium and computer program product that can ensure the accuracy of file operations in response to the above technical problems.

[0007] In a first aspect, the present application provides a file operation method, applied to a host, the method comprising:

[0008] Initialize a first negotiation message, wherein the first negotiation message includes a host byte order field and a first byte order negotiation identification field;

[0009] Converting the first negotiation message using a target byte order, and sending the converted first negotiation message to a data processor;

[0010] receiving a second negotiation message returned by the data processor when the first byte order negotiation identification field is a target value, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identification field;

[0011] Parsing the second negotiation message based on the host byte order;

[0012] When the value of the second byte order negotiation identification field obtained by parsing is a target value, converting the data processor byte order field according to the target byte order to obtain a data processor byte order; wherein the target value is equal in both big endian byte order and little endian byte order;

[0013] When the byte order of the data processor is inconsistent with the byte order of the host, the host is configured to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or the byte orders of the host and the data processor are configured to be the data processor byte order; or the byte orders of the host and the data processor are configured to be the host byte order.

[0014] In one embodiment, the target byte order is little endian or big endian.

[0015] In one embodiment, the method further comprises:

[0016] Through the abstract layer component, the first operation flag in the first file operation message is converted into a first common abstract operation flag, and the first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstract layer component, and determine the operation type based on the second operation flag.

[0017] In one embodiment, the method further comprises:

[0018] receiving a first file operation result message sent by a data processor, wherein the first file operation result message carries a second public abstract operation flag;

[0019] converting the second common abstract operation flag into a third operation flag through an abstract layer component;

[0020] An operation type is determined based on the third operation flag.

[0021] In a second aspect, the present application further provides a file operation method, applied to a data processor, the method comprising:

[0022] Receive a first negotiation message sent by a host, wherein the first negotiation message includes a first byte order negotiation identification field and a host byte order field;

[0023] Parsing the first negotiation message based on the data processor byte order;

[0024] When the value of the first byte order negotiation identification field obtained by parsing is the target value, the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or the byte orders of the host and the data processor are configured to be the data processor byte order; or the byte orders of the host and the data processor are configured to be the host byte order, wherein the target value is equal in both big endian byte order and little endian byte order.

[0025] In one embodiment, the method further comprises:

[0026] When the value of the first byte order negotiation identification field obtained by conversion is a target value, the second byte order negotiation identification field is set to the target value, the data processor byte order field is set to the data processor byte order, and the byte order of the second negotiation message is converted into the target byte order;

[0027] Send the second negotiation message to the host.

[0028] In one embodiment, the target byte order is little endian or big endian.

[0029] In one embodiment, the method further comprises:

[0030] Through the abstract layer component, the first operation result flag in the first file operation result message is converted into a second common abstract operation flag, and the second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstract layer component, and determine the operation type based on the third operation flag.

[0031] In one embodiment, the method further comprises:

[0032] Receiving a first file operation message sent by a host, wherein the first file operation message carries a first public abstract operation flag;

[0033] Converting the first common abstract operation flag into a second operation flag through an abstract layer component;

[0034] An operation type is determined based on the second operation flag.

[0035] In a third aspect, the present application further provides a file operation device, applied to a host, the device comprising:

[0036] An initialization module, used to initialize a first negotiation message, wherein the first negotiation message includes a host byte order field;

[0037] a first conversion module, configured to convert the first negotiation message using a target byte order, and send the converted first negotiation message to a data processor;

[0038] A first receiving module, configured to receive a second negotiation message returned by the data processor based on the first negotiation message, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identification field;

[0039] A second conversion module, used for parsing the second negotiation message based on the host byte order;

[0040] a negotiation determination module, configured to convert the data processor byte order field according to the target byte order to obtain a data processor byte order when the value of the second byte order negotiation identification field obtained by parsing is a target value; wherein the target value is equal in both big-endian byte order and little-endian byte order;

[0041] The first byte order negotiation module is used to configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order when the data processor byte order is inconsistent with the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order.

[0042] In a fourth aspect, the present application further provides a file operation device, applied to a data processor, the device comprising:

[0043] A second receiving module, configured to receive a first negotiation message sent by a host, wherein the first negotiation message includes a host byte order field and a first byte order negotiation identification field;

[0044] A third conversion module, configured to parse the first negotiation message based on a data processor byte order;

[0045] The second negotiation determination module is used to configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order when the value of the first byte order negotiation identification field obtained by parsing is the target value; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order, wherein the target value is equal in both big endian and little endian.

[0046] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0047] In a sixth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0048] In a seventh aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0049] The above-mentioned file operation method, device, computer equipment, computer-readable storage medium and computer program product first negotiate the byte order before the file operation. Specifically, a first negotiation message is initialized, the first negotiation message is converted using the target byte order, and the converted first negotiation message is sent to the data processor. The data processor parses the second negotiation message based on the host byte order based on the second negotiation message returned by the first negotiation message, the second negotiation message including the data processor byte order field and the second byte order negotiation identification field; when the value of the second byte order negotiation identification field obtained by parsing is the target value, that is, when the data processor also supports byte order negotiation, the data processor byte order field is converted according to the target byte order to obtain the data processor byte order, and then the host byte order and the data processor byte order are compared. When the two are inconsistent, the byte orders of the two are agreed upon, including configuring the host to perform byte order conversion on the first file operation to be sent based on the data processor byte order; or configuring the byte orders of the host and the data processor to be the data processor byte order; or configuring the byte orders of the host and the data processor to be the host byte order, so that both the host and the data processor can accurately parse the file operation, thereby ensuring the accuracy of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 An application environment diagram of a file operation method in an embodiment;

[0052] Figure 2 A schematic diagram of a flow chart of a file operation method in an embodiment;

[0053] Figure 3 is a flowchart of a file operation method in another embodiment;

[0054] Figure 4 is a structural block diagram of a file operation device in one embodiment;

[0055] Figure 5 is a structural block diagram of a file operation device in another embodiment;

[0056] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] Specifically, combined Figure 1 As shown, Figure 1 The figure is a framework diagram of a DPU scenario in an embodiment. In this embodiment, the fuse kernel module runs on the operating system of the host machine, and the user-mode file system service runs on the operating system of the DPU. The host machine runs the Linux operating system, but in the actual cloud computing environment, it also supports other operating systems such as Windows.

[0059] The FUSE (Filesystem in userspace) protocol was designed without considering the byte order issue. The FUSE protocol defines dozens of file system operations. Taking the write operation as an example, its protocol format is shown in Table 1 below:

[0060] Table 1: FUSE WRITE operation protocol format

[0061]

[0062] Among them, fuse_in_header describes that all FUSE requests have FUSE message header information, and fuse_write_in describes the basic information of a WRITE operation, including the file being operated, the displacement and length of the write operation in the file, and other information. The above protocol definition of the WRITE operation is described using the host byte order, and the fuse module of the current Linux kernel also uses the host byte order to assign values ​​to structures such as fuse_in_header and fuse_write_in when generating a FUSE request description message for a WRITE operation. The DPU obtains FUSE message data such as fuse_in_header and fuse_write_in through the hardware-implemented virtio-fs channel between it and the Host Machine, and hands it over to the user-mode file system service on the DPU to complete the WRITE request. If the Host Machine and DPU are heterogeneous CPU architectures, and the CPU byte order of the two is inconsistent, for example, the Host Machine uses big-endian byte order and the DPU uses little-endian byte order, then all multi-byte fields in the fuse_in_header and fuse_write_in structures will be incorrectly parsed by the file system service process on the DPU, causing the WRITE operation to fail and possibly causing other undefined errors.

[0063] The FUSE protocol was designed without considering the compatibility between different CPU architectures of the same operating system. Specifically, the FUSE protocol was highly dependent on the Linux kernel file system programming interface during its design. However, the Linux kernel file system has had compatibility issues between different CPU architectures during its long-term evolution. The severity of this problem is described by taking the OPEN operation defined in the FUSE protocol as an example:

[0064] Table 2: FUSE OPEN operation protocol format

[0065]

[0066] The flags field of fuse_open_in is used to describe how to open a file, but these flags have different defined values ​​even between different CPU architectures of the same operating system. Take the X86 and ARM64 architectures as examples:

[0067] Table 3: FUSE OPEN operation protocol format

[0068]

[0069] by Figure 1 For example, if the Host Machine is an X86 architecture and the DPU is an ARM64 architecture, when the fuse kernel module of the Host Machine constructs the OPEN operation request of the FUSE protocol, if the upper-layer application needs to open the file in DIRECT IO mode, the fuse kernel module will use O_DIRECT (00040000) under the X86 architecture to fill the flags field of fuse_open_in. Since the user-mode file system service on the DPU is based on the ARM64 architecture, it will parse the 00040000 value in the flags field of fuse_open_in into O_LARGEFILE (0400000) on the ARM64 architecture. At this time, a serious problem occurs and the user cannot open the file in the correct way. At the least, the file opening fails, and at the worst, various undefined behaviors are caused.

[0070] The file system service is a core function provided by the DPU. The above two defects have seriously affected the file system offload service provided by the DPU, making it almost unusable in scenarios where the host machine and the DPU are heterogeneous CPUs. In the field of cloud computing, customers can freely choose the CPU architecture and operating system type of the cloud server, so the above defects will also seriously affect the DPU's provision of file system offload services in the field of cloud computing.

[0071] To this end, this application provides a file operation method that can be applied to Figure 1 In the application environment shown.

[0072] Among them, the FUSE protocol defines dozens of file system operations. The fuse kernel module of the host machine needs to describe the file system operations initiated by the application using the FUSE protocol, and encapsulate them into FUSE messages and send them to the file system service on the DPU. At this time, if the Host Machine and DPU are heterogeneous CPUs and the byte order used by the CPU is inconsistent, there is a byte order problem. If the byte order of the Host Machine and DPU is consistent, there is no need to perform byte order conversion of the FUSE message.

[0073] In the current FUSE specification, the fuse kernel module of the Host Machine and the user-mode file system service of the DPU perform initial file system feature negotiation through fuse_init_in and fuse_init_out in the FUSE protocol, where fuse_init_in is passed from the Host Machine to the DPU, and fuse_init_out is passed from the DPU to the Host Machine.

[0074] Table 4: Protocol format of FUSE feature negotiation message

[0075]

[0076] In order to solve the byte order problem of FUSE messages in heterogeneous CPU scenarios, the FUSE protocol is extended in this embodiment:

[0077] 1) Add two fields to the fuse_init_in message to obtain the first negotiation message, and the corresponding two added fields are the first byte order negotiation identification field uint16_t nego_endianness and the host byte order field uint16_tendianness.

[0078] 2) Add two fields to the fuse_init_out message to obtain the second negotiation message, and the corresponding two added fields are the second byte order negotiation identification field uint16_t endianness_negoed and the data processor byte order field uint16_t endianness.

[0079] In an exemplary embodiment, Figure 2 As shown, a file operation method is provided, which is applied to Figure 1 The host in the example is used to illustrate, including the following steps 202 to 212. Among them:

[0080] S202: Initialize a first negotiation message, where the first negotiation message includes a host byte order field and a first byte order negotiation identification field.

[0081] The first negotiation message is fuse_init_in to which the first byte order negotiation identification field uint16_t nego_endianness and the host byte order field uint16_t endianness are added.

[0082] The first byte order negotiation identification field uint16_t nego_endianness is used to indicate whether the host supports byte order negotiation. If the value of the first byte order negotiation identification field is the target value, it means that the host supports byte order negotiation. Otherwise, the host does not support byte order negotiation.

[0083] Among them, the target value is equal in big-endian and little-endian byte order. Optionally, the target value is set to 0xdededede. This target value is to ensure that the first byte order negotiation identification field nego_endianness will have the same value regardless of whether it is a CPU in big-endian byte order or a CPU in little-endian byte order, so that the same target value can be obtained regardless of the byte order parsing. After obtaining the target value, it can be determined that the other end supports byte order negotiation.

[0084] Among them, byte order refers to the storage order of data types exceeding one byte in storage devices (such as memory). Big-endian byte order means that high-order bytes are stored at low addresses of storage devices, and low-order bytes are stored at high addresses of storage devices; little-endian byte order means that low-order bytes are stored at low addresses of storage devices, and high-order bytes are stored at high addresses of storage devices.

[0085] S204: convert the first negotiation message using the target byte order, and send the converted first negotiation message to the data processor.

[0086] In one optional embodiment, the target byte order is little endian or big endian. In this application, the byte order of the first negotiation message and the second negotiation message is fixed to little endian or big endian, instead of using the host byte order or the data processor byte order.

[0087] For example, in this application, the little endian byte order can be used as the target byte order, and the host byte order may be the big endian byte order. Whether the subsequent other FUSE messages are converted to byte order depends on the byte order negotiation results of fuse_init_in and fuse_out_init.

[0088] After the above FUSE protocol extension, the byte order problem of the host machine and the data processor DPU can be correctly handled. The host machine initializes the first negotiation message fuse_init_in and makes the following configuration: nego_endianness = 0xdededede; endianness = the host byte order of the host machine. Then the initialized first negotiation message is converted through the target byte order, and the converted first negotiation message is sent to the data processor.

[0089] S206: Receive a second negotiation message returned by the data processor when the first endian negotiation identification field is a target value, where the second negotiation message includes a data processor endian field and a second endian negotiation identification field.

[0090] After receiving the first negotiation message, the data processor parses the first negotiation message based on the data processor byte order, and reads the value of the first byte order negotiation field in the parsed first negotiation message. If the value of the first byte order negotiation field in the first negotiation message is not the target value, for example, it is not 0xdededede, it means that the fuse kernel module of the host Host Machine does not support byte order negotiation. Subsequently, the host Host Machine and the data processor DPU will not perform any byte order negotiation. An error will occur in the FUSE-based user-mode file system service when the byte order of the Host Machine and the DPU are inconsistent.

[0091] If the value of the first byte order negotiation field in the first negotiation message is the target value, such as 0xdededede, it indicates that the fuse kernel module of the host Host Machine supports byte order negotiation, and the user-mode file system service on the data processor DPU side generates a second negotiation message, wherein the second byte order negotiation field endianness_negoed of the second negotiation message fuse_init_out is set to the target value, such as 0xdededede, to indicate that the data processor DPU also supports byte order negotiation, and the byte order field endianness of the second negotiation message fuse_init_out is set to the data processor byte order of the data processor DPU; in addition, if the data processor DPU does not support byte order negotiation, the second byte order negotiation field endianness_negoed of the second negotiation message fuse_init_out is set to a non-target value. Finally, after the second negotiation message fuse_init_out is converted to the target byte order, it is passed to the fuse kernel module of the host Host Machine.

[0092] S208: Parse the second negotiation message based on the host byte order.

[0093] After receiving the second negotiation message, the host parses the second negotiation message according to the host byte order to read the content of each field in the second negotiation message.

[0094] S210: When the value of the second byte order negotiation identification field obtained by parsing is the target value, convert the data processor byte order field according to the target byte order to obtain the data processor byte order; wherein the target value is equal in both the big endian byte order and the little endian byte order.

[0095] When the value of the second byte order negotiation identification field of the parsed second negotiation message is not the target value, it means that the data processor does not support byte order negotiation. Therefore, the subsequent host Host Machine and the data processor DPU will not perform any byte order negotiation. An error will occur in the FUSE-based user-mode file system service when the byte order of the Host Machine and DPU are inconsistent.

[0096] When the value of the second byte order negotiation identification field of the parsed second negotiation message is the target value, which means that the data processor supports byte order negotiation, the data processor byte order field of the parsed second negotiation message is converted according to the target byte order to obtain the correct data processor byte order for subsequent processing.

[0097] S212: When the host byte order and the data processor byte order are inconsistent, configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order.

[0098] Among them, if the host byte order is consistent with the data processor byte order, the subsequent host machine does not need to perform byte order conversion for other file system operations defined by the FUSE protocol.

[0099] If the host byte order and the data processor byte order are inconsistent, byte order conversion processing is required, including the following situations: configuring the host to convert the byte order of the first file operation message to be sent based on the data processor byte order; or configuring the byte order of the host and the data processor to be the data processor byte order; or configuring the byte order of the host and the data processor to be the host byte order.

[0100] That is to say, if both the host and the data processor support byte order negotiation, they can agree on the same byte order. For example, no matter whether the subsequent messages are sent or received, the host and the data processor use the data processor byte order or the host byte order. Or the subsequent messages use the byte order of the other party when sending, and use their own byte order when receiving, so as to avoid byte order confusion.

[0101] In one optional embodiment, the host is configured to perform byte order conversion on received or sent messages based on the byte order of the data processor, while the data processor does not need to perform byte order conversion.

[0102] In one of the optional embodiments, the byte order of the host and the data processor are both configured as the host byte order, that is, the data processor performs byte order conversion on the received or sent message according to the host byte order, and the host processes it according to its own byte order without byte order conversion. The data processor may be, after obtaining the first negotiation message, determine that the first byte order negotiation identification field is the target value, and use the target byte order to parse the host byte order field in the first negotiation message to obtain the host byte order. That is, when the host configures the byte order of the host and the data processor to be the host byte order, the host needs to inform the data processor so that the data processor can subsequently convert the byte order of the received or sent message according to the host byte order.

[0103] In one of the optional embodiments, the host is configured to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order, so that the other party's byte order is used when sending, and its own byte order is used when receiving, thereby avoiding byte order confusion.

[0104] The above-mentioned file operation method first negotiates the byte order before the file operation. Specifically, the first negotiation message is initialized, the first negotiation message is converted using the target byte order, and the converted first negotiation message is sent to the data processor. The data processor parses the second negotiation message based on the host byte order based on the second negotiation message returned by the first negotiation message, the second negotiation message including the data processor byte order field and the second byte order negotiation identification field; when the value of the second byte order negotiation identification field obtained by parsing is the target value, that is, when the data processor also supports byte order negotiation, the data processor byte order field is converted according to the target byte order to obtain the data processor byte order, and then the host byte order and the data processor byte order are compared. When the two are inconsistent, the byte orders of the two are agreed upon, including configuring the host to perform byte order conversion on the first file operation to be sent based on the data processor byte order; or configuring the byte orders of the host and the data processor to be the data processor byte order; or configuring the byte orders of the host and the data processor to be the host byte order, so that the host and the data processor can accurately parse the file operation, thereby ensuring the accuracy of the operation, and other FUSE operations are only performed on the Host. When the byte order of the Machine and DPU is inconsistent, conversion is performed without affecting the performance of non-heterogeneous CPU architectures. Usually, the conversion of network byte order will have a slight performance overhead.

[0105] In one of the optional embodiments, the method also includes: converting a first operation flag in the first file operation message into a first common abstract operation flag through an abstract layer component, the first common abstract operation flag being used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstract layer component, and determining the operation type based on the second operation flag.

[0106] In one of the optional embodiments, the method also includes: receiving a first file operation result message sent by a data processor, the first file operation result message carrying a second common abstract operation flag; converting the second common abstract operation flag into a third operation flag through an abstract layer component; and determining the operation type based on the third operation flag.

[0107] In order to solve the compatibility problem of file systems between different CPU architectures of the same operating system, this embodiment introduces a new public intermediate abstraction layer based on the operating system file system interface in the FUSE protocol, which shields the differences in the file system interfaces between different CPU architectures of the same operating system. Taking the OPEN operation defined in the FUSE protocol as an example, a public and architecture-independent definition is abstracted for the flag parameters that need to be passed in when opening a file:

[0108] Table 5: FUSE OPEN operation public flag definition

[0109]

[0110] Take the Host Machine as X86 and the DPU as ARM64 as an example:

[0111] The fuse kernel module of the host machine of the host opens a file, assuming that its flags are O_DIRECT (00040000) and O_LARGEFILE (00100000), and uses the abstract layer component to convert the first operation flag in the first file operation message into the first public abstract operation flag, that is, using the public abstract flag defined for the FUSE OPEN operation, it will be converted into hexadecimal 0xc.

[0112] The data processor DPU is ARM64, and its O_DIRECT value is 0200000, and O_LARGEFILE is 0400000. After obtaining the message of the FUSE OPEN operation, the file system service on the DPU side parses and obtains its file open flag as 0xc, and thus knows that its real file flags are FUSE_OPEN_DIRECT and FUSE_OPEN_O_LARGEFILE. It is translated and converted through the above-mentioned abstract layer components, and the O_DIRECT (0200000) and O_LARGEFILE (0400000) of the ARM64 on the DPU side are used as the actual DPU side file open flags, so that the file will be correctly opened according to the real intention of the application on the Host Machine side.

[0113] In addition, if the host side receives the first file operation result message sent by the data processor, the first file operation result message carries the second common abstract operation flag, so that the host converts the second common abstract operation flag into the third operation flag through the abstract layer component, so that the host can accurately determine the operation type.

[0114] As described above, it is necessary to introduce similar conversions to all file system operations in the FUSE protocol that have cross-CPU architecture compatibility issues in the new public abstraction layer component of the FUSE protocol, thereby ensuring that the actual file operation intentions of the application on the host machine are correctly processed by the file system service of the data processor DPU.

[0115] In an exemplary embodiment, Figure 3 As shown, a file operation method is provided, which is applied to Figure 1 The data processor DPU in the example is used to illustrate, including the following steps 302 to 308. Among them:

[0116] S302: Receive a first negotiation message sent by a host, where the first negotiation message includes a first byte order negotiation identification field and a host byte order field.

[0117] Among them, the definition of the first byte order negotiation identification field can be found above.

[0118] S304: Convert the first negotiation message based on the data processor byte order.

[0119] S306: When the value of the first byte order negotiation identification field obtained by conversion is the target value, configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order, wherein the target value is equal in both big endian byte order and little endian byte order.

[0120] Optionally, the target byte order is a little endian byte order or a big endian byte order. The definition of the target byte order can also be found above and will not be repeated here.

[0121] After receiving the first negotiation message, the data processor converts the first negotiation message based on the data processor byte order, and reads the value of the first byte order negotiation field in the converted first negotiation message. If the value of the first byte order negotiation field in the first negotiation message is not the target value, for example, it is not 0xdededede, it means that the fuse kernel module of the host Host Machine does not support byte order negotiation. Subsequently, the host Host Machine and the data processor DPU will not perform any byte order negotiation. An error will occur in the FUSE-based user-mode file system service when the byte order of the Host Machine and the DPU are inconsistent.

[0122] If the value of the first byte order negotiation field in the first negotiation message is the target value, for example, 0xdededede, it indicates that the fuse kernel module of the host machine supports byte order big and small end negotiation.

[0123] Optionally, in this embodiment, the configuration of the byte order can be on the data processor side, that is, when both the host and the data processor support byte order negotiation, and the host byte order in the host byte order field is inconsistent with the data processor byte order, the byte order on the data processor side can be directly configured to be the host byte order, that is, the host and the data processor are both processed through the host byte order. Or the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order, and the host is configured to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; or the byte order of the host and the data processor is configured to be the data processor byte order. Wherein the host is configured to perform byte order conversion on the first file operation message to be sent based on the data processor byte order; or when the byte order of the host and the data processor is configured to be the data processor byte order, the byte order on the data processor side needs to be sent to the host, and the specific sending method can be found below. In addition, it should be noted that if the data processor configures the byte order of the host and the data processor to be the data processor byte order, the host needs to be informed of the data processor byte order used.

[0124] In some optional embodiments, the method further includes: when the value of the converted first byte order negotiation identification field is the target value, setting the second byte order negotiation identification field to the target value, setting the data processor byte order field to the data processor byte order, and converting the byte order of the second negotiation message to the target byte order; and sending the second negotiation message to the host.

[0125] In this embodiment, the user-mode file system service on the data processor DPU side generates a second negotiation message, wherein the second byte order negotiation field endianness_negoed of the second negotiation message fuse_init_out is set to the target value, for example, 0xdededede, and the byte order field endianness of the second negotiation message fuse_init_out is set to the byte order of the data processor DPU.

[0126] Finally, the data processor converts the byte order of the second negotiation message fuse_init_out into the target byte order and passes it to the fuse kernel module of the host machine, thereby informing the host that the data processor also supports byte order negotiation, so that the host side can convert the byte order based on the byte order of the data processor and the byte order configuration result of the data processor.

[0127] In one of the optional embodiments, the method also includes: converting a first operation result flag in a first file operation result message into a second common abstract operation flag through an abstract layer component, the second common abstract operation flag being used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstract layer component, and determining the operation type based on the third operation flag.

[0128] In one of the optional embodiments, the method also includes: receiving a first file operation message sent by a host, the first file operation message carrying a first common abstract operation flag; converting the first common abstract operation flag into a second operation flag through an abstract layer component; and determining the operation type based on the second operation flag.

[0129] In the present application, an abstract layer component is introduced. Before the host and the data processor send an operation message, or after receiving an operation message, the operation flag in the operation message is converted through the abstract component. For example, before the data processor sends an operation result message, the first operation result flag in the first file operation result message is converted into a second common abstract operation flag. After receiving the first file operation message sent by the host, the first common abstract operation flag is converted into a second operation flag through the abstract layer component. Similarly, after receiving the first file operation result message, the host converts the second common abstract operation flag in the first file operation result message into a third operation flag through the abstract component. Before sending the first file operation message, the host converts the first common abstract operation flag into the second operation flag through the abstract layer component.

[0130] In the above embodiment, a new public intermediate abstraction layer built on the operating system file system interface is introduced in the FUSE protocol layer, which shields the differences in the file system interfaces of the same operating system between different CPU architectures. The file system service provided by the DPU can be correctly and efficiently supported, and the cross-CPU architecture compatibility problem between the Host Machine and the DPU will not occur. When providing the DPU-based file system offload service, the cloud computing service provider can freely choose the respective CPU architectures of the Host Machine and the DPU, providing flexibility while ensuring performance.

[0131] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0132] Based on the same inventive concept, the embodiment of the present application also provides a file operation device for implementing the file operation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more file operation device embodiments provided below can refer to the limitations on the file operation method above, and will not be repeated here.

[0133] In an exemplary embodiment, Figure 4 As shown, a file operation device is provided, including: an initialization module 401, a first conversion module 402, a first receiving module 403, a second conversion module 404, a first negotiation determination module 405 and a byte order negotiation module 406, wherein:

[0134] An initialization module 401 is used to initialize a first negotiation message, wherein the first negotiation message includes a host byte order field;

[0135] A first conversion module 402, configured to convert the first negotiation message using a target byte order, and send the converted first negotiation message to a data processor;

[0136] A first receiving module 403 is configured to receive a second negotiation message returned by the data processor based on the first negotiation message, where the second negotiation message includes a data processor byte order field and a second byte order negotiation identification field;

[0137] A second conversion module 404, configured to parse the second negotiation message based on a host byte order;

[0138] A first negotiation determination module 405 is configured to convert the data processor byte order field according to the target byte order to obtain a data processor byte order when the value of the second byte order negotiation identification field obtained by parsing is a target value; wherein the target value is equal in both big-endian byte order and little-endian byte order;

[0139] The byte order negotiation module 406 is used to configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order when the data processor byte order and the host byte order are inconsistent; or to configure the byte order of the host and the data processor to be the data processor byte order; or to configure the byte order of the host and the data processor to be the host byte order.

[0140] In one of the optional embodiments, the target byte order is little endian or big endian.

[0141] In one of the optional embodiments, the above device further includes:

[0142] A first flag conversion module is used to convert a first operation flag in a first file operation message into a first common abstract operation flag through an abstract layer component, wherein the first common abstract operation flag is used to instruct a data processor to convert the first common abstract operation flag into a second operation flag through the abstract layer component, and determine an operation type based on the second operation flag.

[0143] In one of the optional embodiments, the above device further includes:

[0144] The second flag conversion module is used to receive a first file operation result message sent by a data processor, wherein the first file operation result message carries a second common abstract operation flag; convert the second common abstract operation flag into a third operation flag through an abstract layer component; and determine an operation type based on the third operation flag.

[0145] In an exemplary embodiment, Figure 5 As shown, a file operation device is provided, including: a second receiving module 501, a third conversion module 502, and a second negotiation determination module 503, wherein:

[0146] The second receiving module 501 is used to receive a first negotiation message sent by a host, where the first negotiation message includes a host byte order field and a first byte order negotiation identification field;

[0147] A third conversion module 502, configured to parse the first negotiation message based on a data processor byte order;

[0148] The second negotiation determination module 503 is used to configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order when the value of the first byte order negotiation identification field obtained by parsing is the target value; or to configure the byte order of the host and the data processor to be the data processor byte order; or to configure the byte order of the host and the data processor to be the host byte order.

[0149] In one of the optional embodiments, the above-mentioned device also includes: a second byte order negotiation module 406, which is used to set the second byte order negotiation identification field to the target value, set the data processor byte order field to the data processor byte order, and convert the byte order of the second negotiation message into the target byte order when the value of the converted first byte order negotiation identification field is the target value; and send the second negotiation message to the host.

[0150] In one of the optional embodiments, the target byte order is little endian or big endian.

[0151] In one of the optional embodiments, the above device further includes:

[0152] The third flag conversion module is used to convert the first operation result flag in the first file operation result message into a second common abstract operation flag through an abstract layer component, and the second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstract layer component, and determine the operation type based on the third operation flag.

[0153] In one of the optional embodiments, the above device further includes:

[0154] a fourth flag conversion module, configured to receive a first file operation message sent by a host, wherein the first file operation message carries a first public abstract operation flag;

[0155] Converting the first common abstract operation flag into a second operation flag through an abstract layer component;

[0156] An operation type is determined based on the second operation flag.

[0157] Each module in the above-mentioned file operation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0158] In an exemplary embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a file operation method is implemented.

[0159] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] In one embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. The computer device may be a data processor DPU, and the data processor may be a DPU chip. Alternatively, the computer device may be a network interface card including a data processor. Alternatively, the computer device may be a server including a data processor or a network interface card, and the server includes a host and a data processor, wherein the data processor is used to dispatch a message to the host or the data processor itself for processing, and the host is used to process the message dispatched by the data processor.

[0161] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0162] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0163] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0164] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0165] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A file operation method, characterized in that: Applied to a host, the method comprises: Initialize a first negotiation message, wherein the first negotiation message includes a host byte order field and a first byte order negotiation identification field; Converting the first negotiation message using a target byte order, and sending the converted first negotiation message to a data processor; receiving a second negotiation message returned by the data processor when the first byte order negotiation identification field is a target value, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identification field; Parsing the second negotiation message based on the host byte order; When the value of the second byte order negotiation identification field obtained by parsing is a target value, converting the data processor byte order field according to the target byte order to obtain a data processor byte order; wherein the target value is equal in both big endian byte order and little endian byte order; When the byte order of the data processor is inconsistent with the byte order of the host, the host is configured to perform byte order conversion on the first file operation message to be sent based on the byte order of the data processor; or the byte orders of the host and the data processor are configured to be the data processor byte order; or the byte orders of the host and the data processor are configured to be the host byte order.

2. The method according to claim 1, characterized in that The target byte order is little endian or big endian.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Through the abstract layer component, the first operation flag in the first file operation message is converted into a first common abstract operation flag, and the first common abstract operation flag is used to instruct the data processor to convert the first common abstract operation flag into a second operation flag through the abstract layer component, and determine the operation type based on the second operation flag.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a first file operation result message sent by a data processor, wherein the first file operation result message carries a second public abstract operation flag; converting the second common abstract operation flag into a third operation flag through an abstract layer component; An operation type is determined based on the third operation flag.

5. A file operation method, characterized in that: Applied to a data processor, the method comprises: Receive a first negotiation message sent by a host, wherein the first negotiation message includes a first byte order negotiation identification field and a host byte order field; Parsing the first negotiation message based on the data processor byte order; When the value of the first byte order negotiation identification field obtained by parsing is the target value, the data processor is configured to perform byte order conversion on the first file operation result message to be sent based on the host byte order; or the byte orders of the host and the data processor are configured to be the data processor byte order; or the byte orders of the host and the data processor are configured to be the host byte order, wherein the target value is equal in both big endian byte order and little endian byte order.

6. The method according to claim 5, characterized in that The method further comprises: When the value of the first byte order negotiation identification field obtained by parsing is a target value, the second byte order negotiation identification field is set to the target value, the data processor byte order field is set to the data processor byte order, and the byte order of the second negotiation message is converted into the target byte order; Send the second negotiation message to the host.

7. The method according to claim 6, characterized in that The target byte order is little endian or big endian.

8. The method according to claim 5 or 6, characterized in that: The method further comprises: Through the abstract layer component, the first operation result flag in the first file operation result message is converted into a second common abstract operation flag, and the second common abstract operation flag is used to instruct the host to convert the second common abstract operation flag into a third operation flag through the abstract layer component, and determine the operation type based on the third operation flag.

9. The method according to claim 5 or 6, characterized in that: The method further comprises: Receiving a first file operation message sent by a host, wherein the first file operation message carries a first public abstract operation flag; Converting the first common abstract operation flag into a second operation flag through an abstract layer component; An operation type is determined based on the second operation flag.

10. A file operation device, characterized in that: Applied to a host, the device comprises: An initialization module, used to initialize a first negotiation message, wherein the first negotiation message includes a host byte order field; a first conversion module, configured to convert the first negotiation message using a target byte order, and send the converted first negotiation message to a data processor; A first receiving module, configured to receive a second negotiation message returned by the data processor based on the first negotiation message, wherein the second negotiation message includes a data processor byte order field and a second byte order negotiation identification field; A second conversion module, used for parsing the second negotiation message based on the host byte order; a negotiation determination module, configured to convert the data processor byte order field according to the target byte order to obtain a data processor byte order when the value of the second byte order negotiation identification field obtained by parsing is a target value; wherein the target value is equal in both big-endian byte order and little-endian byte order; The first byte order negotiation module is used to configure the host to perform byte order conversion on the first file operation message to be sent based on the data processor byte order when the data processor byte order is inconsistent with the host byte order; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order.

11. A file operation device, characterized in that: Applied to a data processor, the device comprises: A second receiving module, configured to receive a first negotiation message sent by a host, wherein the first negotiation message includes a host byte order field and a first byte order negotiation identification field; A third conversion module, configured to parse the first negotiation message based on a data processor byte order; The second negotiation determination module is used to configure the data processor to perform byte order conversion on the first file operation result message to be sent based on the host byte order when the value of the first byte order negotiation identification field obtained by parsing is the target value; or configure the byte order of the host and the data processor to be the data processor byte order; or configure the byte order of the host and the data processor to be the host byte order, wherein the target value is equal in both big endian and little endian.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images