A data writing method, a reading method, a device and an electronic equipment
By creating shared files in the file system and writing address information, the problem of data sharing in the CXL memory device partitioned memory pool mode is solved, enabling efficient data sharing and system collaboration among multiple hosts.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-03-20
AI Technical Summary
In the existing technology, the partitioned memory pool mode of CXL memory devices cannot meet the data sharing needs between multiple hosts, especially in the case of big data analysis, where intermediate data needs to be shared between multiple hosts.
By creating shared files in the file system and writing the address information and data of the shared files into the memory device based on the mapping relationship between the virtual address of the host and the physical address of the memory device, data sharing between multiple hosts can be achieved.
It enables data sharing among multiple hosts, improves resource utilization and system collaboration capabilities, and allows for efficient reading and writing of shared data in memory devices.
Smart Images

Figure CN121411708B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data processing, and in particular to a data writing method, a data reading method, a device and an electronic equipment. BACKGROUND
[0002] At present, a CXL (Compute Express Link, an open interconnection protocol) memory device can be connected to multiple physical hosts (such as computers, servers, etc.) through a CXL bus, so that the CXL memory device can provide memory expansion functions for multiple hosts as a memory pool. The related art manages the memory pool in a partitioned memory pool mode, that is, the memory pool is divided into multiple partitions, and each partition is only allocated to one host for independent use.
[0003] However, there is sometimes a need for data sharing between multiple hosts, for example, when multiple hosts are used to work together to perform big data analysis, intermediate data generated by host A may need to be provided to host B for use. However, the partitioned memory pool mode of the related art cannot meet the demand for data sharing between multiple hosts, because each partition is only used independently by one host. SUMMARY
[0004] The purpose of the embodiments of the present application is to provide a data writing method, a data reading method, a device and an electronic equipment to realize data sharing between multiple hosts. The specific technical solutions are as follows:
[0005] The embodiments of the present application first provide a data writing method applied to a first host, wherein the first host runs a file system, the file system is mounted with a memory device, the memory device stores metadata of the file system, the memory device provides shared memory for multiple hosts, and the multiple hosts all run the file system. The method comprises the following steps:
[0006] Creating a shared file in the file system;
[0007] Based on a mapping relationship between a virtual address of the first host and a physical address of the memory device, writing address information of the shared file in the metadata stored in the memory device; wherein the address information of the shared file comprises: an offset of the virtual address of the shared file in the first host compared to a starting mapping address; and the starting mapping address is a starting address of continuous virtual addresses in the first host which have a mapping relationship with the memory device.
[0008] Based on the mapping relationship and the address information of the shared file, writing shared data in the memory device.
[0009] In an embodiment of the present application, the writing of the shared data into the memory device based on the mapping relationship and the address information of the shared file comprises:
[0010] determining a virtual address of the shared file in the first host based on the address information of the shared file;
[0011] obtaining an address offset of the shared data to be written in the shared file;
[0012] determining a to-be-written virtual address of the shared data according to the obtained address offset and the virtual address of the shared file in the first host;
[0013] writing the shared data into the memory device based on the mapping relationship and the to-be-written virtual address.
[0014] In an embodiment of the present application, the metadata of the file system comprises: first metadata representing a memory device mounted by the file system, and second metadata representing a directory structure of the file system;
[0015] the writing of the address information of the shared file into the metadata stored in the memory device comprises:
[0016] writing directory information of the shared file and the address information of the shared file into the second metadata stored in the memory device; wherein the directory information of the shared file represents a directory path of the shared file in the file system.
[0017] In an embodiment of the present application, the metadata of the file system is written into the memory device by the following way:
[0018] receiving a request transferred based on an input / output control interface;
[0019] if an identifier representing the creation of a file is transferred in the received request, obtaining type information carried by the request;
[0020] if the type information represents metadata, writing the metadata of the file system running in the first host into the memory device based on a mapping relationship between a virtual address of the first host and a physical address of the memory device in response to the request;
[0021] the creation of the shared file in the file system comprises:
[0022] receiving a request transferred based on an input / output control interface;
[0023] If an identifier indicating creation of a file is passed in the received request, and the type information carried by the request indicates a shared file, a shared file is created in the file system.
[0024] In an embodiment of the present application, the method further comprises:
[0025] generating an inode of the shared file, wherein the inode records address information of the shared file;
[0026] writing shared data in the memory device based on the mapping relationship and the address information of the shared file, comprises:
[0027] writing shared data in the memory device based on the mapping relationship and the address information of the shared file recorded in the inode;
[0028] The method further comprises:
[0029] When receiving an unloading request for the file system of the first host, recursively deleting the inodes of the files contained in the virtual file system of the first host based on the directory structure of the file system.
[0030] An embodiment of the present application further provides a data reading method applied to a second host, wherein the second host runs a file system, the file system is mounted with a memory device, the memory device provides shared memory for a plurality of hosts, and the plurality of hosts all run the file system; the memory device stores metadata of the file system, the metadata contains address information of a shared file, and the address information of the shared file comprises an offset of a virtual address of the shared file in a host compared to a starting mapping address, and the starting mapping address is a starting address of continuous virtual addresses in a host having a mapping relationship with the memory device; the method comprises the following steps:
[0031] reading the metadata from the memory device based on a mapping relationship between a virtual address of the second host and a physical address of the memory device;
[0032] creating the shared file in a virtual file system of the second host based on the metadata;
[0033] reading shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file.
[0034] In an embodiment of the present application, the reading of the shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file comprises:
[0035] determining a virtual address of the shared file in the second host based on the address information of the shared file;
[0036] obtaining an address offset of shared data to be read in the shared file;
[0037] determining a virtual address to be read of the shared data according to the obtained address offset and the virtual address of the shared file in the second host;
[0038] reading the shared data from the memory device based on the mapping relationship and the virtual address to be read.
[0039] In an embodiment of the present application, the metadata of the file system comprises: first metadata representing a memory device mounted by the file system, and second metadata recording a directory path of the shared file in the file system and address information of the shared file;
[0040] creating the shared file in the file system of the second host based on the metadata, comprising:
[0041] creating the shared file in the file system of the second host according to the directory path recorded by the second metadata.
[0042] In an embodiment of the present application, after reading the metadata from the memory device, the method further comprises:
[0043] generating an inode of the shared file; wherein the inode records the address information of the shared file;
[0044] reading shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file, comprising:
[0045] reading shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file recorded in the inode;
[0046] The method further comprises:
[0047] when receiving an unloading request for the file system of the second host, recursively deleting inodes of files contained in the virtual file system of the second host based on a directory structure of the file system.
[0048] The embodiment of the present application also provides a data writing device, which is applied to a first host, the first host runs a file system, the file system is mounted with a memory device, metadata of the file system is stored in the memory device, the memory device provides shared memory for a plurality of hosts, the plurality of hosts all run the file system, and the device comprises:
[0049] a file creating module, which is used for creating a shared file in the file system;
[0050] an address information writing module, which is used for writing address information of the shared file in the metadata stored in the memory device based on a mapping relationship between a virtual address of the first host and a physical address of the memory device, wherein the address information of the shared file comprises an offset of the virtual address of the shared file in the first host compared with a starting mapping address, and the starting mapping address is a starting address of continuous virtual addresses in the first host which have the mapping relationship with the memory device;
[0051] a shared data writing module, which is used for writing shared data in the memory device based on the mapping relationship and the address information of the shared file.
[0052] The embodiment of the present application also provides a data reading device, which is applied to a second host, the second host runs a file system, the file system is mounted with a memory device, the memory device provides shared memory for a plurality of hosts, and the plurality of hosts all run the file system; metadata of the file system is stored in the memory device, the metadata comprises address information of a shared file, the address information of the shared file comprises an offset of a virtual address of the shared file in a host compared with a starting mapping address, and the starting mapping address is a starting address of continuous virtual addresses in the host which have the mapping relationship with the memory device; the device comprises:
[0053] a metadata reading module, which is used for reading the metadata from the memory device based on a mapping relationship between a virtual address of the second host and a physical address of the memory device;
[0054] a file synchronizing module, which is used for creating the shared file in a virtual file system of the second host based on the metadata;
[0055] a shared data reading module, which is used for reading shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file.
[0056] The embodiment of the present application also provides an electronic device, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.
[0057] a memory for storing a computer program;
[0058] a processor for executing the program stored on the memory to implement the steps of the data writing method or the data reading method.
[0059] The embodiment of the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data writing method or the data reading method.
[0060] The embodiment of the present application also provides a computer program product containing instructions which, when executed on a computer, cause the computer to perform the data writing method or the data reading method.
[0061] The embodiment of the present application has the following beneficial effects:
[0062] The data writing method provided by the embodiment of the present application is applied to a first host, the first host runs a file system, the file system is mounted with a memory device, the memory device stores metadata of the file system, the memory device provides shared memory for a plurality of hosts, and the plurality of hosts all run the file system, so that the first host can create a shared file in the file system; based on a mapping relationship between a virtual address of the first host and a physical address of the memory device, address information of the shared file is written in the metadata stored in the memory device, the address information of the shared file includes: an offset of the virtual address of the shared file in the first host compared to a start mapping address; the start mapping address is a start address of continuous virtual addresses in the first host which have the mapping relationship with the memory device; based on the mapping relationship and the address information of the shared file, shared data is written in the memory device.
[0063] In the scheme, the file systems of the plurality of hosts are mounted to the same memory device, and the metadata of the file system and the address information of the shared file are written in the memory device, so that any host can obtain the metadata of the file system and the address information of the shared file from the memory device to create the shared file in the file system of the host, and when data of the shared file is read and written based on the file system, the shared data can be written or read in the memory device in the memory read-write mode based on the mapping relationship between the virtual address of the host and the physical address of the memory device, so that the data sharing between the plurality of hosts can be realized through the scheme.
[0064] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS
[0065] In order to make the technical solutions of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim to some embodiments of the present application, and other embodiments can be obtained by those skilled in the art based on these drawings.
[0066] Figure 1 The first flowchart of the data writing method provided by the embodiments of the present application;
[0067] Figure 2 The principle diagram of the data writing method provided by the embodiments of the present application;
[0068] Figure 3 The second flowchart of the data writing method provided by the embodiments of the present application;
[0069] Figure 4 The third flowchart of the data writing method provided by the embodiments of the present application;
[0070] Figure 5 The first flowchart of the data reading method provided by the embodiments of the present application;
[0071] Figure 6 The structure diagram of the data writing device provided by the embodiments of the present application;
[0072] Figure 7 The structure diagram of the data reading device provided by the embodiments of the present application;
[0073] Figure 8 The structure diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION
[0074] The technical solutions of the embodiments of the present application will be described clearly and completely in combination with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art based on the present application belong to the protection scope of the present application.
[0075] Firstly, some concepts involved in the embodiments of the present application will be introduced.
[0076] CXL memory: CXL memory is a high-speed interconnection technology based on the PCIe (peripheral component interconnect express) physical layer, which realizes memory sharing and pooling between CPU and accelerator, memory expansion device by building cache consistency protocol. Its core is to break the physical isolation of memory resources in traditional architecture, allowing direct access to unified memory space across devices without software intervention to achieve low-latency, high-bandwidth data interaction. This technology solves the problem of resource waste caused by memory capacity fragmentation (such as stranded memory) in AI (Artificial Intelligence), HPC (High Performance Computing) and other fields, and significantly improves the overall throughput efficiency of the system by dynamically allocating memory pools, which is a key infrastructure to support the next generation of data center computing power needs.
[0077] Memory pooling: Memory pooling is a memory resource collaborative management paradigm for distributed systems, which integrates the physical memory space of multiple independent hosts into a logically unified shared memory pool through global virtualization technology (such as distributed shared memory or remote direct memory access). This technology, with the help of consistency protocols, access control strategies and low-latency communication mechanisms (such as InfiniBand or RoCE (RDMA over Converged Ethernet), realizes dynamic allocation, transparent access and collaborative recovery of memory blocks between multiple nodes, thereby breaking through the single-machine memory capacity limit, improving cross-node data interaction efficiency (such as reducing serialization / deserialization overhead) and guaranteeing strong consistency or eventual consistency. Its core features include cross-node memory virtualization, distributed lock and synchronization mechanism, fault tolerance design (such as replica redundancy or erasure code) and load balancing strategy, widely used in high-performance computing, distributed databases, large-scale machine learning training and cloud computing platforms, etc.
[0078] FS-DAX (Filesystem DAX): FS-DAX is an access mode for persistent memory and similar high-speed media supported by the Linux file system. After FS-DAX is enabled, the file system that supports this mode (such as ext4 and xfs) can enable an application to directly access data on the storage medium with load / store instructions through mmap, bypassing the traditional page cache, thereby greatly improving I / O performance and reducing latency. FS-DAX is mainly used in single-host environments for high-performance and low-latency read-write scenarios for persistent memory, and does not have multi-host sharing and distributed consistency management capabilities.
[0079] In the era of artificial intelligence, big data analysis and machine learning as mainstream research topics and application scenarios, computers have an increasingly important demand for large-capacity and high-bandwidth memory, and traditional DDR (Double Data Rate SDRAM) memory is expensive and difficult to meet the growing demand for large-capacity memory. The emergence of CXL memory provides the possibility of more economical high-capacity memory expansion. After CXL 2.0 provides the functions of MHSLD (Multi-Headed-Single Logical Device) and MHMLD (Multi-Headed-Multi-Logical Device), combined with the pooling and sharing capabilities of CXL 2.0 memory devices, CXL 2.0 can provide memory pooling capabilities to the host. Multiple hosts can allocate or recycle memory from the CXL 2.0 memory pool as needed, and can exclusively or share the memory of the memory pool. Since the Type 3 device supported by CXL 2.0 can be composed of DDR particles or PMem (Persistent Memory) particles, and the CXL device composed of PMem particles can provide memory capabilities and storage capabilities, therefore, by referring to the implementation of memory pooling, CXL 2.0 can also support storage pooling.
[0080] CXL storage pooling can provide storage capabilities to multiple hosts and can provide them in an exclusive or shared manner. Due to these basic capabilities, for business scenarios, the same storage device can be shared on multiple hosts, or a host can exclusively use a storage device. Related technologies use a partitioned memory pool mode, and each partition is only used independently by one host, which cannot meet the data sharing needs between multiple hosts.
[0081] To solve the above problems and realize data sharing among multiple hosts, an embodiment of the present application provides a data writing method, a reading method, a device and an electronic equipment.
[0082] Firstly, a data writing method provided by an embodiment of the present application is introduced below. The method is applied to a first host, which can be any host connected with a memory device. The first host runs a file system, the file system is mounted with the memory device, the memory device stores metadata of the file system, the memory device provides shared memory for multiple hosts, and the multiple hosts all run the file system.
[0083] As shown in Figure 1 , the method comprises the following steps:
[0084] S101, creating a shared file in the file system;
[0085] S102, writing address information of the shared file in the metadata stored in the memory device based on a mapping relationship between a virtual address of the first host and a physical address of the memory device; wherein the address information of the shared file comprises: an offset of the virtual address of the shared file in the first host compared to a starting mapping address; and the starting mapping address is a starting address of continuous virtual addresses in the first host which have a mapping relationship with the memory device.
[0086] S103, writing shared data in the memory device based on the mapping relationship and the address information of the shared file.
[0087] In this embodiment, the host can be a host running a Linux operating system, and the memory device can be a CXL memory device accessed to the host in the form of DAX (Direct Access). The memory device can include a persistent memory and a general DAX memory device, i.e. a non-persistent memory device. The persistent memory can be, for example, NVDIMM (non-volatile dual-in-line memory module), CXL Type3, PMem, etc.
[0088] The above file system can be VFS (Virtual File Systems), which provides a conventional file system interface and enables data in the memory device to be managed and presented in the form of a file in the operating system of the host.
[0089] As shown in Figure 2 , the multiple hosts in this embodiment (such as Figure 2The user state process, the file system, the first driver and the second driver can be run in the host 1 and the host 2 in the figure. The first driver can be a driver provided by FS-DAX, which can establish a mapping relationship between a physical address of a PMem or other persistent memory device and a virtual address of a host kernel.
[0090] The second driver is used to establish a mapping relationship between a physical address of a non-persistent memory device and a virtual address of a host kernel. Specifically, a dev_dax_iomap interface in the driver provided by FS-DAX only supports a persistent memory device, i.e. a non-volatile memory. When the dev_dax_iomap interface is used to operate a non-persistent memory device, i.e. a volatile memory, it will refuse to respond. Therefore, the code of the dev_dax_iomap interface can be improved so as not to refuse the operation on the non-persistent memory device. Thus, a dev_dax_direct_access function of the dev_dax_iomap interface can be used to obtain a physical location of the non-persistent memory device and an access entry (i.e. a virtual address) in the host kernel, and to establish a mapping relationship between a continuous virtual address currently available in the host kernel and the physical location of the non-persistent memory device. In this way, an upper-layer application can directly read and write data at a certain address in the non-persistent memory device according to needs. The present scheme is based on FS-DAX and is extended to a general DAX device. The FS-DAX is retained to have the advantage of low latency achieved by bypassing a page cache, and the application range of the FS-DAX is expanded.
[0091] The dev_dax_iomap interface needs to implement the following functions: add a fs_dax_get function, and define operations of a device holder through holder_operations to ensure that exclusive access operations on the DAX device are correctly acquired and released; obtain and save a kva (Kernel Virtual Address) from a memremap interface in the Linux kernel to adapt to read and write operations of an iomap; and implement three key functions for DAX operations on non-persistent memory, including direct_access (to implement a conversion between a physical address and a page frame number), zero_page_range (to efficiently clear a page through a persistent memory interface), and recovery_write (to implement safe data writing by using a cache flush technology).
[0092] Through the driver, multiple hosts can correctly identify the same memory device and map the memory device into an accessible node, such as / dev / dax0.0, in the file system for use by the file system. It can be seen that the present solution can realize efficient and direct access to volatile memory and non-volatile memory between multiple physical hosts, greatly improving resource utilization and system collaboration capability.
[0093] Further, in the kernel mounting the file system, the fs_dax_get() function can be called to establish an association between the / dev / dax0.0 device managed by the above driver and the core structure of the file system, forming the skeleton of the kernel file system. For example, a superblock representing the association between the above memory device and the file system can be generated, and at this time, the virtual file system layer in the host is running. After mounting is completed, this virtual file system layer is ready to receive instructions from the user state. The file system can be flexibly extended, that is, the file system can mount multiple memory devices, and the mounted memory devices can be flexibly adjusted according to the changes of the host and storage resources, to improve the scalability and flexibility of the system.
[0094] Although the present embodiment is based on CXL memory pool, it is not designed only for CXL memory. In theory, the present embodiment can be used for sharing any form of DAX memory device, and has relatively wide applicability.
[0095] In order to register the file system, the following functions need to be configured in the host:
[0096] Since the non-persistent memory is a character device, a superblock uninstallation function kill_char_super() for the character device needs to be configured;
[0097] The core logic of the DAX-based shared memory file system includes: mounting process, superblock initialization, and inode creation;
[0098] The file system context operation is implemented, and the mounting parameter parsing and resource release are processed;
[0099] The necessary system resources are registered for the file system, and the access permission to check the device file is obtained;
[0100] The memory-based file system basic functions are provided to support directory creation and simple file operation, and the memory page management is realized through the ram_aops function.
[0101] The file system needs to be configured with the following functions:
[0102] The operation function is set for the index node of the file, and the basic attribute setting and acquisition are supported;
[0103] Define operation functions for the inode of the directory, including basic directory operations such as creation, lookup, link, deletion, etc.
[0104] Implement the mknod (a command for manually creating device files) method, allocate an inode and bind to dentry (entry in the directory), check the memory device error state, and maintain the timestamp;
[0105] Implement the mkdir (make directory) method, create the directory's inode through mknod, and increment the hard link count of the parent directory;
[0106] Implement the create (create file) method, which reuses the mknod logic to create a file represented by the S_IFREG flag, i.e., a normal file;
[0107] Define superblock operations that can be passed to the user space, support statistical file system information, delete inodes, and display mount options, so that multiple physical hosts can share file system directory and file structure information;
[0108] Associate s_op (superblock operation) when the file system is mounted to enable the above functions;
[0109] Implement mount option display, output the mode (mode) parameter, for example, output to the directory / proc / mounts;
[0110] Implement file read and write operations based on memory devices;
[0111] Provide a page fault handling function and a memory mapping system call processing function for the file form of the memory device.
[0112] For the above step S101, the host can receive a request delivered based on an input / output control interface; if an identifier representing the creation of a file is transmitted in the received request, and the type information carried by the request indicates a shared file, a shared file is created in the file system.
[0113] Specifically, when creating a file, a user first creates a virtual file in the file system of a host through a conventional file creation manner, and obtains a corresponding file identifier. Then, the user transmits, based on an ioctl (input output control interface) operation, an identifier of the virtual file, an identifier indicating creation of a file (for example, "MHSFSIOC_MAP_CREATE"), type information of the file, a size of the file, and address information of the file, and the host kernel can check whether a location and a size of a memory region occupied by the created file meet alignment and other rules, associate the address information of the file with an i_private field of an inode of the file, so that the file system of the host can respond to subsequent file operations of the user, such as open, close, read, and write, in a standard file operation manner based on the address information of the file.
[0114] Meanwhile, the user needs to write the address information of the file and the directory information into the metadata on the memory device synchronously, so as to enable other hosts to read the metadata and synchronize the metadata to the file systems of the other hosts, to reconstruct the directory of the file.
[0115] In addition to the shared file, metadata of the file system can also be written into the memory device in the form of a file in the file system. The file can also be presented in the file system. System-level metadata can also be written into the memory device in a similar manner, for example, can specifically include the following steps:
[0116] Step A1, receiving a request transmitted based on an input output control interface;
[0117] Step A2, if an identifier indicating creation of a file is transmitted in the received request, obtaining type information carried in the request;
[0118] Step A3, if the type information indicates metadata, in response to the request, writing, based on a mapping relationship between a virtual address of the first host and a physical address of the memory device, metadata of a file system running in the first host into the memory device.
[0119] In order to write the metadata of the file system into the memory device, the metadata corresponding file can also be created in the above-mentioned file creation manner. For the metadata corresponding file that has been created in the file system, the metadata can be read and written as a general file, so as to be able to write the metadata into the memory device. Further, when writing the metadata, data writing can be fixedly started from a starting address of the corresponding file, so that when other hosts need to synchronize the file, the metadata can be read from the starting address of the file.
[0120] By writing the metadata into the memory device, multiple hosts can read the metadata from the memory device to synchronize the metadata, and thus synchronize the state of the file system.
[0121] The file system in this embodiment can add ioctl interface support for the user state to perfect the DAX mapping management capability, expand the metadata management function, and complete the data interaction mechanism between the kernel module and the user space. Specifically, the user state interface of the file system needs to configure the following functions:
[0122] Include the user state ioctl header file in the maintenance scope to ensure tracking of ioctl interface changes;
[0123] Implement a dynamic allocation / release mechanism for the metadata structure to support multiple extended block storage;
[0124] Add MAP_CREATE / MAP_GET (mapping creation / mapping query) command processing logic to implement user state creation and query of file mapping;
[0125] Map the file offset to the PMD (Page Middle Directory) aligned physical address through the dev_dax_iomap interface to ensure direct memory access compliance;
[0126] Declare ioctl command numbers, data structures, and enumeration types to establish a communication protocol between the user and kernel spaces;
[0127] Strengthen special offset verification for superblock files to prevent non-privileged files from using reserved address space;
[0128] Increase metadata consistency checks to detect file size tampering and uninitialized access abnormal scenarios;
[0129] Use inode locks to protect the metadata writing process and prevent concurrent modifications from causing inconsistent states.
[0130] This embodiment receives a request transmitted based on an input / output control interface, and an identifier representing the creation of a file is transmitted in the request, and then the kernel can respond to the request to create a file according to the type. It can be seen that this embodiment can create a file based on the standard POSIX (Portable Operating System Interface), which can reduce the development complexity and threshold of the application implementing this embodiment.
[0131] Regarding step S102 above, the file system metadata may include: first metadata representing the memory device on which the file system is mounted, and second metadata representing the directory structure of the file system. The first metadata can be called superblock metadata, which records the overall usage status of the file system; the second metadata can be called log metadata. Both file system metadata can be stored within the capacity of the memory device.
[0132] Step S102 may specifically include:
[0133] Write the directory information and address information of the shared file into the second metadata stored in the memory device; where the directory information of the shared file represents the directory path of the shared file in the file system.
[0134] For example, if the shared file is named "bar" and created in the ". / dir" directory, then the directory path for the shared file could be ". / dir / bar". Figure 2 As shown, the area occupied by a shared file on a memory device can be dispersed. For example, "data0: {off0, len0}, data1: {off1, len1}" means that the starting address of data0 in the shared file is off0 and the data length is len0; the starting address of data1 is off1 and the data length is len1.
[0135] In this embodiment, for ease of reading and writing, the address of the file corresponding to the file system metadata can be set at a specified location on the memory device, such as a continuous range of addresses starting from the beginning address of the memory device. When it is necessary to write the address information of the shared file to the file system metadata, the host can first read the metadata from this specified location into local memory, then add the directory path and address information of the shared file to the read metadata to obtain the updated metadata, and finally write the updated metadata into the file corresponding to the metadata. Since user-mode processes directly access virtual addresses, in the above process, it is necessary to write the updated metadata to the memory device based on the mapping relationship between the virtual address of the first host and the physical address of the memory device.
[0136] The first host writes the directory and address information of the shared file into the metadata stored in the memory device. Then, other hosts connected to the memory device can create the shared file in their own file system after reading the metadata from the memory device, thus realizing file sharing among multiple hosts.
[0137] Regarding step S103 above, since a file has already been created in the file system, when a process on the first host generates shared data that needs to be shared with other hosts, it can open the file and then write the shared data into the file.
[0138] In an implementation, the virtual address of the shared file in the first host can be determined based on the address information of the shared file, and the shared data is written in the memory device based on the mapping relationship and the to-be-written virtual address.
[0139] Further, in an implementation, as shown in Figure 3 writing the shared data in the memory device based on the mapping relationship and the address information of the shared file can include:
[0140] S1021, determining the virtual address of the shared file in the first host based on the address information of the shared file;
[0141] Since the kernel usually allocates continuous addresses when allocating virtual addresses to processes, and the address information of the shared file includes the offset of the virtual address of the shared file in the first host relative to the start mapping address, the start mapping address is the start address of the continuous virtual addresses in the first host that have the mapping relationship with the memory device, therefore, the start address in this segment of virtual addresses plus the offset can obtain the virtual address of the shared file in the first host.
[0142] In an implementation, all physical addresses or a segment of continuous addresses of the memory device can be mapped to the virtual addresses of the host from the start address of the memory device, so that the address information of the shared file can also be the offset of the address of the shared file in the memory device relative to the start address of the memory device.
[0143] S1022, obtaining the address offset of the shared data to be written in the shared file;
[0144] The address offset is the offset relative to the start address of the shared file.
[0145] S1023, determining the to-be-written virtual address of the shared data according to the obtained address offset and the virtual address of the shared file in the first host;
[0146] Specifically, the sum of the obtained address offset and the virtual address of the shared file in the first host can be calculated to obtain the to-be-written virtual address of the shared data.
[0147] S1024, writing the shared data in the memory device based on the mapping relationship and the to-be-written virtual address.
[0148] That is, the physical address mapped by the to-be-written virtual address in the memory device is determined based on the mapping relationship, and then the shared data is written in the memory device according to the mapped physical address.
[0149] The embodiment determines the to-be-written virtual address of the shared data according to the address offset and the virtual address of the shared file in the first host, and then writes the shared data in the memory device based on the mapping relationship and the to-be-written virtual address, so that the shared data can be written at a specified position in the shared file according to the address offset.
[0150] In an embodiment of the present application, as shown in Figure 4 The method provided by the embodiment of the present application can further include:
[0151] S401, generating an index node of the shared file; wherein the address information of the shared file is recorded in the index node;
[0152] The index node is a core data structure in the file system, and is used to store the meta information of the file system object (such as a file or a directory), including the attributes such as permission, time stamp, and owner. The index node of the shared file can be generated in the local memory of the first host, so that the first host can manage the file system based on the index node. The address information of the shared file can be recorded in the i_private field of the index node.
[0153] The above writing of the shared data in the memory device based on the mapping relationship and the address information of the shared file includes:
[0154] S402, writing the shared data in the memory device based on the mapping relationship and the address information of the shared file recorded in the index node;
[0155] The method can further include:
[0156] S403, when receiving an unloading request for the file system of the first host, recursively deleting the index nodes of the files contained in the virtual file system of the first host based on the directory structure of the file system.
[0157] Specifically, the improved kill_char_super() function can be called, and the improved kill_char_super() function only recursively deletes all the index nodes under the file system. The address information stored in the i_private field of the index node is also naturally cleaned, so that all the information in the file system is completely deleted, and all the mapping relationships in the kernel are also cleaned.
[0158] Since the address information of shared files is stored in the host's inode, all subsequent read, write, or memory mapping operations on that file can utilize this address information, combined with the mapping relationship established by the aforementioned driver, to convert the address offset within the file into a physical address on the memory device, enabling direct access to the memory device without going through a cache. When unmounting the file system, the inodes of the files contained in the virtual file system of the first host can be recursively deleted, thereby completely erasing all information in the file system.
[0159] Furthermore, neither the file system nor the shared memory devices it relies on have built-in cross-host synchronization capabilities (such as memory cache consistency). When multiple hosts mount the same memory device, conflicts may occur if multiple hosts modify metadata simultaneously. To avoid this race condition, a master-slave management model can be adopted: one host is responsible for creating and updating superblock metadata, log metadata, and file data, while other slave hosts only read and use this data without modifying it. This reduces the risk of conflicts and ensures that the structure and content of the file system are consistent and predictable across multiple hosts. In this way, data consistency and reliability can be guaranteed in a relatively simple way during concurrent access from multiple hosts.
[0160] Alternatively, consistency protocols such as distributed locks and cache synchronization can be used across multiple hosts to achieve cache consistency.
[0161] In this embodiment, the file systems of multiple hosts are mounted to the same memory device, and the memory device contains the file system metadata and the address information of the shared files. Thus, any host can obtain the file system metadata and the address information of the shared files from the memory device to create shared files within its own file system. When reading or writing data to this shared file based on the file system, the shared data can be written or read from the memory device using the mapping relationship between the host's virtual address and the physical address of the memory device. Therefore, this solution enables data sharing among multiple hosts. When multiple hosts share memory through the file system, they do not need to read or write to the DAX memory device via local memory, thus achieving zero-copy data sharing and effectively improving data read / write performance.
[0162] The file system realizes unified, efficient and direct access to heterogeneous shared memory among multiple physical hosts through deep integration with new memory interconnection technologies such as CXL. At the file system level, the file system in the application can provide a standard POSIX interface and rich metadata management, and provide a concurrency control mechanism to ensure data consistency and reliability to a certain extent in a multi-host environment. Since the file system is compatible with the standard POSIX semantics, the application in the host can seamlessly integrate the file system without making significant modifications to utilize distributed shared memory, without modifying the operating system level, and without relying on hardware such as CXL Fabric Switch (CXL architecture switch) for multi-host and memory device interconnection. Therefore, the scheme can significantly simplify development, deployment and operation and maintenance, and reduce the threshold for multi-host sharing of heterogeneous memory. At the same time, the file system has elastic expansion capability, so that the shared memory resources can be efficiently and flexibly shared and managed by multiple hosts in the form of a file system.
[0163] Based on the above data writing method, the embodiment of the application further provides a data reading method, which can be applied to a second host, the second host running a file system, the file system mounting a memory device, the memory device providing shared memory for multiple hosts, and the multiple hosts each running a file system; the second host can be a host other than the first host in the multiple hosts.
[0164] The memory device stores metadata of the file system, and the metadata contains address information of a shared file, the address information of the shared file including an offset of a virtual address of the shared file in a host relative to a starting mapping address, and the starting mapping address being a starting address of a continuous virtual address in the host having a mapping relationship with the memory device.
[0165] The second host can also adopt the architecture in the host as shown in Figure 2 , so as to pre-establish a mapping relationship between the virtual address of the second host and the physical address of the memory device, and mount the memory device to the file system of the second host. As shown in Figure 5 , the method comprises:
[0166] S501, reading metadata from the memory device based on the mapping relationship between the virtual address of the second host and the physical address of the memory device;
[0167] S502, creating a shared file in the virtual file system of the second host based on the metadata;
[0168] After reading the metadata, the same shared file can be created in the local file system of the second host, and the address information of the shared file can also be saved in the local inode through the ioctl() interface.
[0169] S503, reading the shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file.
[0170] Corresponding to the above shared data writing process, in an embodiment of the present application, the reading of the shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file can include:
[0171] S B1, determining a virtual address of the shared file in the second host based on the address information of the shared file;
[0172] S B2, obtaining an address offset of the shared data to be read in the shared file;
[0173] S B3, determining a virtual address to be read of the shared data according to the obtained address offset and the virtual address of the shared file in the second host;
[0174] S B4, reading the shared data from the memory device based on the mapping relationship and the virtual address to be read.
[0175] In an embodiment of the present application, the metadata of the file system includes: a first metadata indicating the memory device mounted by the file system, and a second metadata, the second metadata recording a directory path of the shared file in the file system and address information of the shared file;
[0176] The creating of the shared file in the file system of the second host based on the metadata includes:
[0177] According to the directory path recorded by the second metadata, the shared file is created in the file system of the second host.
[0178] That is, according to the directory path recorded by the second metadata, the shared file is created under the corresponding directory path of the file system.
[0179] In an embodiment of the present application, after reading the metadata from the memory device, the method further includes:
[0180] Generating an index node of the shared file; wherein the index node records the address information of the shared file;
[0181] Correspondingly, the reading of the shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file includes:
[0182] Reading the shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file recorded in the index node;
[0183] The method can further include:
[0184] When receiving the unloading request for the file system of the second host, the index nodes of the files contained in the virtual file system of the second host are recursively deleted based on the directory structure of the file system.
[0185] In the embodiment, the file systems of the plurality of hosts are mounted to the same memory device, and the metadata of the file system and the address information of the shared file are written in the memory device, so that any host can obtain the metadata of the file system and the address information of the shared file from the memory device to create the shared file in the file system of the host, and when data of the shared file is read and written based on the file system, the shared data can be written or read in the memory device in the memory read and write mode based on the mapping relationship between the virtual address of the host and the physical address of the memory device, so that the data sharing between the plurality of hosts can be realized through the scheme.
[0186] Based on the same inventive concept, the embodiment of the application further provides a data writing device applied to a first host, wherein the first host runs a file system, the file system is mounted with a memory device, the memory device stores metadata of the file system, the memory device provides shared memory for a plurality of hosts, and the plurality of hosts all run the file system. Figure 6 As shown in the figure, the device comprises:
[0187] a file creation module 601 configured to create a shared file in the file system;
[0188] an address information writing module 602 configured to write address information of the shared file in the metadata stored in the memory device based on a mapping relationship between a virtual address of the first host and a physical address of the memory device; wherein the address information of the shared file comprises an offset of the virtual address of the shared file in the first host relative to a starting mapping address; and the starting mapping address is a starting address of continuous virtual addresses in the first host which have the mapping relationship with the memory device.
[0189] a shared data writing module 603 configured to write shared data in the memory device based on the mapping relationship and the address information of the shared file.
[0190] In an embodiment of the application, the shared data writing module 603 comprises:
[0191] a first virtual address determining sub-module configured to determine the virtual address of the shared file in the first host based on the address information of the shared file;
[0192] a first address offset obtaining sub-module configured to obtain an address offset of the shared data to be written in the shared file;
[0193] The to-be-written virtual address determination submodule is configured to determine a to-be-written virtual address of the shared data according to the obtained address offset and a virtual address of the shared file in the first host.
[0194] The shared data writing submodule is configured to write the shared data in the memory device based on the mapping relationship and the to-be-written virtual address.
[0195] In an embodiment of the present application, the metadata of the file system comprises: first metadata representing a memory device mounted by the file system, and second metadata representing a directory structure of the file system.
[0196] The address information writing module 602 is specifically configured to:
[0197] write, in the second metadata stored in the memory device, directory information of the shared file and address information of the shared file based on a mapping relationship between a virtual address of the first host and a physical address of the memory device; wherein the directory information of the shared file represents a directory path of the shared file in the file system.
[0198] In an embodiment of the present application, the apparatus further comprises: a request receiving module configured to receive a request transmitted based on an input / output control interface;
[0199] The type obtaining module is configured to obtain type information carried by the request if an identifier representing creation of a file is transmitted in the received request.
[0200] The request responding module is configured to, if the type information represents metadata, write metadata of the file system running in the first host into the memory device based on a mapping relationship between a virtual address of the first host and a physical address of the memory device in response to the request.
[0201] The file creating module 601 is specifically configured to create a shared file in the file system if an identifier representing creation of a file is transmitted in the received request and type information carried by the request represents a shared file.
[0202] In an embodiment of the present application, the apparatus further comprises:
[0203] The first inode generating module is configured to generate an inode of the shared file; wherein the inode records address information of the shared file.
[0204] The shared data writing module 603 is specifically configured to:
[0205] writing, in the memory device, shared data based on the mapping relationship and the address information of the shared file recorded in the index node;
[0206] The apparatus further includes:
[0207] The first index node deleting module is configured to, when receiving an unloading request for the file system of the first host, delete the index nodes of the files contained in the virtual file system of the first host recursively based on the directory structure of the file system.
[0208] The embodiment of the present application further provides a data reading apparatus, which is applied to a second host, wherein the second host runs a file system, the file system is mounted with a memory device, the memory device provides shared memory for a plurality of hosts, and the plurality of hosts all run the file system; the memory device stores metadata of the file system, the metadata contains address information of a shared file, and the address information of the shared file comprises an offset of a virtual address of the shared file in a host compared to a starting mapping address, and the starting mapping address is a starting address of continuous virtual addresses in a host which has a mapping relationship with the memory device; as shown in the figure, the apparatus includes: Figure 7
[0209] The metadata reading module 701 is configured to read the metadata from the memory device based on a mapping relationship between a virtual address of the second host and a physical address of the memory device;
[0210] The file synchronizing module 702 is configured to create the shared file in the file system of the second host based on the metadata;
[0211] The shared data reading module 703 is configured to read shared data in the shared file from the memory device based on the mapping relationship and the address information of the shared file.
[0212] In an embodiment of the present application, the shared data reading module 703 includes:
[0213] The second virtual address determining submodule is configured to determine a virtual address of the shared file in the second host based on the address information of the shared file;
[0214] The second address offset obtaining submodule is configured to obtain an address offset of the shared data to be read in the shared file;
[0215] The to-be-read virtual address determining submodule is configured to determine a to-be-read virtual address of the shared data according to the obtained address offset and the virtual address of the shared file in the second host;
[0216] A shared data reading submodule is used to read the shared data from the memory device based on the mapping relationship and the virtual address to be read.
[0217] In one embodiment of the present invention, the metadata of the file system includes: first metadata representing the memory device mounted on the file system, and second metadata, wherein the second metadata records the directory path of the shared file in the file system and the address information of the shared file;
[0218] File synchronization module 702 is specifically used for:
[0219] Based on the directory path recorded in the second metadata record, the shared file is created in the file system of the second host.
[0220] In one embodiment of the present invention, the apparatus further includes: a second index node creation module, configured to generate an index node for the shared file after reading the metadata from the memory device; wherein the index node records the address information of the shared file;
[0221] Shared data reading module 703 is specifically used for:
[0222] Based on the mapping relationship and the address information of the shared file recorded in the index node, the shared data in the shared file is read from the memory device;
[0223] The device further includes:
[0224] The second inode deletion module is used to recursively delete the inodes of files contained in the virtual file system of the second host based on the directory structure of the file system when an unload request for the file system of the second host is received.
[0225] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.
[0226] Memory 803 is used to store computer programs;
[0227] The processor 801, when executing the program stored in the memory 803, implements the steps of any of the data reading methods or data writing methods described above.
[0228] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0229] The communication interface is used for communication between the above electronic device and other devices.
[0230] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.
[0231] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.
[0232] In still another embodiment provided by the application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the data reading method or the data writing method described in any of the above embodiments are implemented.
[0233] In still another embodiment provided by the application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to execute the steps of the data reading method or the data writing method described in any of the above embodiments.
[0234] In the embodiments described above, all or some of the steps can be implemented by software, hardware or firmware, or any combination thereof. When implemented by software, all or some of the steps can be implemented in the form of one or more computer programs. The computer program can be stored in any computer readable medium, and loaded into the computer for execution. The computer readable medium includes computer storage media and communication media. The computer storage media includes any tangible or physical medium for storing or transmitting the program. The computer storage media can be a volatile (e.g., RAM) or non-volatile (e.g., ROM, disk, or CD) storage medium. The communication media typically include computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The computer readable media does not include carrier waves or other transient signals.
[0235] It should be noted that, in the present document, the terms such as first and second are used only to differentiate one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not necessarily include those elements only, but can include other elements not expressly listed, or other elements inherent in such process, method, article or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.
[0236] Each of the embodiments in the present document is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device, electronic device, and computer readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.
[0237] The above merely describes the preferred embodiments of the present application, but is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A data writing method, characterized in that, Applied to a first host, the first host runs a file system, the file system mounts a memory device, the memory device stores the metadata of the file system, the memory device provides shared memory for multiple hosts, all of which run the file system, the method includes: Create a shared file in the file system; Based on the mapping relationship between the virtual address of the first host and the physical address of the memory device, the address information of the shared file is written into the metadata stored in the memory device; wherein, the address information of the shared file includes: the offset of the virtual address of the shared file in the first host relative to the starting mapping address; the starting mapping address is: the starting address of the consecutive virtual addresses in the first host that have a mapping relationship with the memory device; Based on the mapping relationship and the address information of the shared file, shared data is written into the memory device.
2. The method according to claim 1, characterized in that, The step of writing shared data into the memory device based on the mapping relationship and the address information of the shared file includes: The virtual address of the shared file in the first host is determined based on the address information of the shared file. Obtain the address offset of the shared data to be written in the shared file; Based on the obtained address offset and the virtual address of the shared file in the first host, determine the virtual address to be written to the shared data; Based on the mapping relationship and the virtual address to be written, the shared data is written to the memory device.
3. The method according to claim 1, characterized in that, The metadata of the file system includes: first metadata representing the memory device mounted on the file system, and second metadata representing the directory structure of the file system; The step of writing the address information of the shared file into the metadata stored in the memory device includes: The directory information of the shared file and the address information of the shared file are written into the second metadata stored in the memory device; wherein, the directory information of the shared file represents the directory path of the shared file in the file system.
4. The method according to claim 1 or 3, characterized in that, The metadata of the file system is written to the memory device in the following manner: Receive requests transmitted via the input / output control interface; If the received request contains an identifier indicating the creation of a file, then obtain the type information carried by the request; If the type information represents metadata, then in response to the request, based on the mapping relationship between the virtual address of the first host and the physical address of the memory device, the metadata of the file system running in the first host is written to the memory device; Creating a shared file in the file system includes: Receive requests transmitted via the input / output control interface; If the received request contains an identifier indicating the creation of a file, and the type information carried by the request indicates that it is a shared file, then a shared file is created in the file system.
5. The method according to claim 1, characterized in that, The method further includes: Generate an index node for the shared file; wherein the index node records the address information of the shared file; The step of writing shared data into the memory device based on the mapping relationship and the address information of the shared file includes: Based on the mapping relationship and the address information of the shared file recorded in the index node, shared data is written into the memory device; The method further includes: When an unload request is received for the file system of the first host, the inodes of the files contained in the virtual file system of the first host are recursively deleted based on the directory structure of the file system.
6. A data reading method, characterized in that, The method is applied to a second host, which runs a file system and mounts a memory device. The memory device provides shared memory for multiple hosts, all of which run the file system. The memory device stores metadata of the file system, which includes address information of shared files. This address information includes the offset of the shared file's virtual address on the host relative to the starting mapping address, where the starting mapping address is the starting address of a contiguous set of virtual addresses on the host that has a mapping relationship with the memory device. The method includes: Based on the mapping relationship between the virtual address of the second host and the physical address of the memory device, the metadata is read from the memory device; The shared file is created in the file system of the second host based on the metadata; Based on the mapping relationship and the address information of the shared file, the shared data in the shared file is read from the memory device.
7. The method according to claim 6, characterized in that, The step of reading shared data from the shared file from the memory device based on the mapping relationship and the address information of the shared file includes: The virtual address of the shared file in the second host is determined based on the address information of the shared file; Obtain the address offset of the shared data to be read in the shared file; Based on the obtained address offset and the virtual address of the shared file in the second host, determine the virtual address to be read of the shared data; The shared data is read from the memory device based on the mapping relationship and the virtual address to be read.
8. The method according to claim 6, characterized in that, The metadata of the file system includes: first metadata representing the memory device mounted on the file system, and second metadata, wherein the second metadata records the directory path of the shared file in the file system and the address information of the shared file; Creating the shared file in the file system of the second host based on the metadata includes: Based on the directory path recorded in the second metadata record, the shared file is created in the file system of the second host.
9. The method according to claim 6, characterized in that, After reading the metadata from the memory device, the method further includes: Generate an index node for the shared file; wherein the index node records the address information of the shared file; The step of reading shared data from the shared file from the memory device based on the mapping relationship and the address information of the shared file includes: Based on the mapping relationship and the address information of the shared file recorded in the index node, the shared data in the shared file is read from the memory device; The method further includes: When an unload request is received for the file system of the second host, the inodes of the files contained in the virtual file system of the second host are recursively deleted based on the directory structure of the file system.
10. A data writing device, characterized in that, Applied to a first host, the first host running a file system, the file system mounting a memory device, the memory device storing the metadata of the file system, the memory device providing shared memory for multiple hosts, all of which are running the file system, the device includes: The file creation module is used to create shared files in the file system. The address information writing module is used to write the address information of the shared file into the metadata stored in the memory device based on the mapping relationship between the virtual address of the first host and the physical address of the memory device; wherein, the address information of the shared file includes: the offset of the virtual address of the shared file in the first host relative to the starting mapping address; the starting mapping address is: the starting address of the consecutive virtual addresses in the first host that have a mapping relationship with the memory device; A shared data writing module is used to write shared data into the memory device based on the mapping relationship and the address information of the shared file.
11. A data reading device, characterized in that, The method is applied to a second host, which runs a file system and mounts a memory device. The memory device provides shared memory for multiple hosts, all of which run the file system. The memory device stores metadata of the file system, which includes address information of shared files. This address information includes the offset of the shared file's virtual address on the host relative to the starting mapping address, where the starting mapping address is the starting address of a contiguous set of virtual addresses on the host that has a mapping relationship with the memory device. The apparatus includes: The metadata reading module is used to read the metadata from the memory device based on the mapping relationship between the virtual address of the second host and the physical address of the memory device; The file synchronization module is used to create the shared file in the virtual file system of the second host based on the metadata; A shared data reading module is used to read shared data from the shared file from the memory device based on the mapping relationship and the address information of the shared file.
12. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-5 or any one of claims 6-9.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5 or 6-9.
Citation Information
Patent Citations
Method for sharing file system by multiple hosts, product, equipment and storage medium
CN120123305A
Low-latency shared memory channel across address spaces without system call overhead in a computing system
US20220019362A1