Method and system for optimizing persistent cache performance of Lustre client based on GPU direct access

Abstract

The invention discloses a GPU direct access-based Lustre client persistent cache performance optimization method and system, relates to the technical field of high-performance computing storage, and discloses a GPU direct access-based Lustre client persistent cache performance optimization method and system which directly access a local SSD (Solid State Disk) and optimize a metadata management mechanism through a GPU. The CPU is prevented from serving as the bottleneck of data transfer, the copy operation of redundant memory is reduced, the problem of performance limitation caused by the fact that the GPU cannot directly access the storage device in the prior art is effectively solved, data transmission delay can be reduced, the system throughput is improved, CPU resource consumption is reduced, and the parallel processing capacity of the GPU is fully exerted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-performance computing storage, in particular to a Lustre client persistent cache performance optimization method and system based on GPU direct access. BACKGROUND

[0002] As a representative of high-performance parallel file systems, the Lustre file system plays a key role in high-performance computing and large-scale data processing. Its client cache technology (PCC) reduces server access requirements by caching data locally, but it exposes significant limitations in GPU computing environments. Under the current architecture, GPUs cannot directly access local solid-state drives (SSDs) and must rely on CPUs as data transits. This design leads to three core problems: first, the CPU becomes a performance bottleneck on the I / O path, not only increasing data transfer latency but also severely restricting the system's overall throughput; second, the data transfer between GPU and CPU requires additional memory copy operations, which not only consume valuable CPU computing resources but also increase memory bandwidth pressure; finally, the powerful parallel processing capabilities of GPUs cannot be effectively utilized in I / O operations as they must passively wait for the CPU to complete data processing.

[0003] In the prior art, the SNVMe solution enables GPU direct access to NVMe devices, but lacks deep integration with distributed file systems. The pure GPU file system solution is complex in architecture and has poor compatibility with existing CPU file systems, making it difficult to deploy in practice. The traditional Lustre PCC mechanism does not consider GPU computing characteristics at all and still requires data to be routed through the CPU. None of these technical solutions can build a complete solution, as they cannot achieve GPU direct access to local SSDs or guarantee metadata consistency synchronization with the Lustre file system, ultimately preventing the full release of GPU computing potential. Especially in scenarios that require frequent access to large-scale scientific data, this architectural flaw significantly reduces overall computing efficiency.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent an acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a Lustre client persistent cache performance optimization method and system based on GPU direct access, aiming to reduce data transfer latency, improve system throughput, reduce CPU resource consumption, and fully utilize the parallel processing capabilities of GPUs.

[0006] To achieve the above purpose, the present application provides a Lustre client persistent cache performance optimization method based on GPU direct access, which comprises: creating a file through a Lustre client and obtaining system information to initialize file metadata; wherein the file metadata comprises a file identifier, a physical storage descriptor, access control information, a status flag, and check information; performing embedding processing on the initialized file metadata to generate a main metadata area embedded in a file header and a simplified metadata area embedded in a file tail, and registering file information into a persistent cache table through the Lustre client; initiating a file opening request through a Lustre client POSIX interface to obtain a file identifier and access permissions, and directly accessing the simplified metadata area based on the file identifier and the access permissions to extract a physical storage descriptor, a status flag, and check information; allocating and managing an NVMe I / O queue in GPU memory based on the physical storage descriptor and the status flag, and directly reading and writing file data on a local SSD through an NVMe protocol; wherein the reading and writing operations comprise updating the status flag or updating the file metadata, and synchronizing the updated status flag or file metadata to the corresponding metadata area; when the GPU closes the file, notifying the CPU through shared memory; the CPU reads the updated file metadata in the main metadata area to verify consistency and trigger a dirty data migration mechanism; based on an aging mechanism or an AI prediction mechanism, migrating data of the local SSD to the Lustre file system or migrating data from the Lustre file system back to the local SSD to optimize data access performance.

[0007] In an embodiment, the step of creating a file through a Lustre client and obtaining system information to initialize file metadata comprises: allocating a file unique identifier in a Lustre file system through a CPU to obtain a file identifier; pre-allocating physical space on a local SSD and recording a logical block starting address to generate a physical storage descriptor; based on the file identifier and the physical storage descriptor, combining access control information and a status flag to construct initial file metadata; performing write processing on the initial file metadata to generate a main metadata area embedded in a file header and a simplified metadata area embedded in a file tail.

[0008] In an embodiment, the step of initiating a file opening request through a Lustre client POSIX interface to obtain a file identifier and access permissions, and directly accessing the simplified metadata area based on the file identifier and the access permissions to extract a physical storage descriptor, a status flag, and check information comprises: GPU sends a file open request through the Lustre client POSIX interface; CPU detects the file open request to obtain file information, and writes a file identifier and initial access permission into shared memory; GPU reads the file identifier and initial access permission in the shared memory; GPU directly reads the thin metadata area based on the file identifier through the NVMe protocol; GPU parses the content of the thin metadata area to extract a physical storage descriptor, a state tag, and check information; GPU verifies the validity of the metadata using the check information, and notifies the CPU to re-synchronize the file metadata if the verification fails.

[0009] In an embodiment, directly reading and writing file data on a local SSD through the NVMe protocol includes: GPU reads file data based on a logical block start address in the physical storage descriptor through the NVMe protocol; GPU updates an access timestamp in the state tag to generate an updated state tag, and synchronizes the updated state tag to the thin metadata area.

[0010] In an embodiment, directly reading and writing file data on a local SSD through the NVMe protocol includes: GPU checks write permission in the access control information; If the file space needs to be expanded, the GPU sends an expansion request to the CPU; the CPU allocates additional space and updates file metadata to generate a new physical storage descriptor and updated file metadata; the GPU obtains the new physical storage descriptor; GPU writes file data based on the new physical storage descriptor through the NVMe protocol; GPU updates a dirty data tag, a modification timestamp, and a file size in the state tag to generate an updated state tag; GPU uses atomic operations to synchronize the updated state tag and the updated file metadata to the thin metadata area and the main metadata area, respectively.

[0011] In an embodiment, allocating and managing NVMe I / O queues in GPU memory includes: GPU allocates an NVMe submission queue and a completion queue in memory; GPU threads construct NVMe I / O commands, and write the commands to the allocated submission queue; GPU notifies the NVMe controller to process the written I / O commands through a doorbell mechanism; The NVMe controller writes back the completion information to the allocated completion queue; The GPU thread polls the allocated completion queue to get the operation result.

[0012] In an embodiment, when the GPU closes the file, the CPU is notified through shared memory; the step of reading the updated file metadata in the master metadata area by the CPU to verify consistency and trigger the dirty data migration mechanism includes: After the GPU completes the file operation, the updated file metadata is written to the master metadata area; The GPU sets a synchronization flag and notifies the CPU through shared memory; The CPU reads the file metadata in the master metadata area based on the notification to verify consistency; The CPU updates the verified file metadata to the Lustre persistent cache table and triggers dirty data migration according to the dirty data flag.

[0013] In an embodiment, migrating data from the local SSD to the Lustre file system includes migration based on an aging mechanism or migration based on an AI prediction mechanism, wherein the migration based on the aging mechanism includes: Monitoring the residence time of the file data in the local SSD; When the residence time exceeds a preset threshold, automatically migrating the file data to the Lustre file system; Updating the file attributes in the Lustre global namespace to synchronize the migrated file data; The migration based on the AI prediction mechanism includes: The AI application predicts that the file data will not be accessed in the future; Based on the result of the prediction operation, migrating the file or directory to the Lustre file system; Updating the migration state in the state flag after the migration is completed.

[0014] In an embodiment, migrating back from the Lustre file system to the local SSD includes: When reading data in the Lustre file system, automatically migrating the data back to the local SSD; Alternatively, the AI application predicts that the file will be read, and the data is migrated back to the local SSD in advance, and the file metadata is updated based on the migrated data in the local SSD and registered to the persistent cache table to complete synchronization.

[0015] In addition, to achieve the above object, the application further provides a GPU direct access-based Lustre client persistent cache performance optimization system, which comprises a memory, a processor and a GPU direct access-based Lustre client persistent cache performance optimization program stored in the memory and capable of running on the processor.

[0016] The GPU direct access-based Lustre client persistent cache performance optimization method and system provided by the application can reduce data transmission delay, improve system throughput, reduce CPU resource consumption and fully exert the parallel processing capability of GPU by directly accessing local SSD through GPU and optimizing metadata management mechanism, thereby avoiding the bottleneck of CPU as data transfer and reducing redundant memory copy operation, effectively solving the performance limitation problem caused by the fact that GPU cannot directly access storage devices in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative labor.

[0019] Figure 1 A flowchart of an embodiment of the GPU direct access-based Lustre client persistent cache performance optimization method of the application is provided. Figure 2 A structural diagram of an embodiment of the GPU direct access-based Lustre client persistent cache performance optimization system of the application is provided.

[0020] Explanation of the reference signs: 10, memory; 20, processor.

[0021] The object implementation, functional characteristics and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0022] The technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0023] It should be understood that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0024] In the prior art, although the SNVMe scheme realizes the direct access of the GPU to the NVMe device, it lacks deep integration with the distributed file system; the pure GPU file system scheme is difficult to be actually deployed due to the complex architecture and poor compatibility with the existing CPU file system; and the traditional Lustre PCC mechanism does not consider the GPU computing characteristics at all, and still requires data to be transferred through the CPU. None of these technical solutions can build a complete solution, neither realizing the direct access of the GPU to the local SSD nor guaranteeing the synchronization of the metadata consistency with the Lustre file system, which finally leads to the failure to fully release the GPU computing potential. Especially in the scene of frequently accessing large-scale scientific data, such architectural defects will significantly reduce the overall computing efficiency.

[0025] Based on this, the embodiment of the present application provides a performance optimization method for a Lustre client persistent cache based on GPU direct access, referring to Figure 1 The performance optimization method for the Lustre client persistent cache based on GPU direct access includes steps S100-S600, in which: Step S100, a file is created through a Lustre client, and system information is obtained to initialize file metadata; wherein the file metadata includes a file identifier, a physical storage descriptor, access control information, a status flag, and check information; Step S200, the initialized file metadata is embedded to generate a main metadata area embedded in a file header and a simplified metadata area embedded in a file tail, and file information is registered into a persistent cache table through the Lustre client; Step S300, the GPU initiates a file opening request through a Lustre client POSIX interface to obtain a file identifier and access permission, and directly accesses the reduced metadata area based on the file identifier and access permission to extract a physical storage descriptor, a state tag, and check information; Step S400, the GPU allocates and manages an NVMe I / O queue in GPU memory based on the physical storage descriptor and the state tag, and directly reads and writes file data on a local SSD through an NVMe protocol; wherein the read and write operations include updating the state tag or updating file metadata, and synchronizing the updated state tag or file metadata to the corresponding metadata area; Step S500, when the GPU closes the file, the CPU is notified through shared memory; the CPU reads the updated file metadata in the main metadata area to verify consistency and trigger a dirty data migration mechanism; Step S600, based on an aging mechanism or an AI prediction mechanism, data of the local SSD is migrated to or from the Lustre file system to optimize data access performance.

[0026] In this embodiment, the file metadata is a set of information used to describe file attributes and states, which can be implemented in various ways. For example, the file identifier can be generated using a globally unique identifier (GUID) generation algorithm, or by calculating the file path through a hash function; the physical storage descriptor can record a logical block address range, or can be implemented in the form of a file mapping table; the access control information can be defined based on an access control list (ACL), or implemented through a simple permission bit mask; the state tag can represent the file state using a bitmap structure, such as whether it has been modified or is being accessed; the check information can be generated using a cyclic redundancy check (CRC) algorithm, or through a hash algorithm such as SHA-256. These implementation methods are mainly used to ensure that the file metadata can accurately describe the attributes and states of the file, thereby supporting subsequent file operations.

[0027] In this embodiment, the embedding of the main metadata area and the reduced metadata area can be implemented through various technical means. For example, the main metadata area can be embedded in the file header using a fixed-length header structure, or associated with the data file through an independent metadata file; the reduced metadata area can only retain key fields and reduce storage overhead through compression algorithms, or use sparse storage to record only non-default value fields. The main purpose of this partition design is to enable the GPU to quickly access the reduced metadata area, while providing complete metadata backup for the main metadata area for verification. In this embodiment, the process in which the GPU initiates a file opening request through a POSIX interface can be implemented in multiple ways. For example, the file opening request can be implemented through a standard POSIX API call or through a custom lightweight interface; the file identifier and access rights can be delivered through shared memory or through a message queue or other inter-process communication mechanism. The main purpose of this design is to enable the GPU to independently complete the file opening operation without relying on the full participation of the CPU. In addition, the allocation and management of the NVMe I / O queue can also be implemented in multiple ways. For example, the submission queue and the completion queue can be implemented in a ring buffer structure or dynamically expanded through a linked list structure; the doorbell mechanism can be triggered through register writing or implemented through an interrupt mechanism; the polling operation can be implemented through a timer driver or through an event-driven approach. The main purpose of this design is to enable the GPU to efficiently manage I / O operations, thereby fully exploiting its parallel processing capabilities. In this embodiment, the present application enables the GPU to bypass the CPU and directly interact with the local SSD by constructing a metadata structure and an I / O path that can be independently operated by the GPU, thereby eliminating the performance bottleneck in traditional solutions. Compared to the limitation in the prior art that the GPU must transmit data through the CPU, the present application reconstructs the I / O path through metadata partitioning, GPU direct access, and intelligent migration, etc., thereby solving the I / O performance bottleneck problem caused by the GPU's inability to directly access the local SSD in the Lustre file system.

[0028] The embodiments of the present application create a file and initialize file metadata through a Lustre client, wherein the file identifier ensures the uniqueness of the file, the physical storage descriptor is used to specify the physical storage location of the local SSD, the access control information manages the permission boundary, the status marker tracks the dynamic state of the file, and the verification information guarantees data integrity. The initialization of these metadata enables the GPU to directly locate and verify data, avoiding the repeated involvement of the CPU in the metadata acquisition link. Further, the initialized file metadata is embedded to generate a main metadata area embedded in the file header and a simplified metadata area embedded in the file tail. The main metadata area stores complete information for CPU verification consistency, while the simplified metadata area only retains key information for easy access by the GPU. In this way, the metadata processing overhead is significantly reduced, and unified management is achieved by registering the file information in the persistent cache table.

[0029] In this embodiment, the GPU initiates a file opening request through the Lustre client POSIX interface to obtain a file identifier and access rights. Based on the file identifier and access rights, the GPU directly accesses the reduced metadata area to extract the physical storage descriptor, status tag, and verification information. Specifically, this identifier-based direct access mechanism enables the GPU to independently complete metadata analysis without CPU involvement in each I / O operation, thereby significantly reducing latency.

[0030] In this embodiment, after obtaining the physical storage descriptor and status tag, the GPU allocates and manages the NVMe I / O queue in the video memory. According to the logical block address in the physical storage descriptor, the GPU constructs an NVMe I / O command and notifies the NVMe controller to process the written I / O command through the doorbell mechanism. The NVMe controller writes the completion information back to the completion queue, and the GPU thread polls the completion queue to obtain the operation result. In this way, the GPU can directly read and write file data on the local SSD through the NVMe protocol. During the read and write operations, the status tag or file metadata is updated and synchronized to the corresponding metadata area. For example, when reading file data, the access timestamp in the status tag is updated; when writing file data, the dirty data tag, modification timestamp, and file size in the status tag are updated. In this way, the GPU realizes direct access to the local SSD, eliminating the bottleneck problem of CPU as a data transfer intermediary.

[0031] In this embodiment, when the GPU completes the file operation, the CPU is notified through shared memory. The CPU reads the updated file metadata in the main metadata area to verify consistency and trigger the dirty data migration mechanism. Specifically, the CPU updates the verified file metadata to the Lustre persistent cache table and triggers dirty data migration based on the dirty data tag. As a preferred implementation, based on the aging mechanism or AI prediction mechanism, data on the local SSD is migrated to or from the Lustre file system. For example, based on the aging mechanism, the residence time of file data in the local SSD is monitored, and when the residence time exceeds a preset threshold, the file data is automatically migrated to the Lustre file system; based on the AI prediction mechanism, it is predicted that the file data will not be accessed in the future, and the file or directory is migrated to the Lustre file system. After migration is complete, the migration status in the status tag is updated.

[0032] In this embodiment, through the above technical solutions, the I / O performance bottleneck problem caused by the GPU's inability to directly access the local SSD is solved. This method, as a whole, through metadata partitioning, GPU direct access, and intelligent migration, completely restructures the I / O path, eliminates the performance bottleneck in traditional solutions, and optimizes data access performance.

[0033] In a feasible implementation, the step of creating a file by the Lustre client and obtaining system information to initialize file metadata includes: allocating a file unique identifier in the Lustre file system by a CPU to obtain a file identifier; pre-allocating physical space on a local SSD and recording a logical block start address to generate a physical storage descriptor; based on the file identifier and the physical storage descriptor, combining access control information and a state marker to construct initial file metadata; and writing the initial file metadata to generate a main metadata area embedded in a file header and a compact metadata area embedded in a file tail.

[0034] In this embodiment, the file identifier refers to a string or a numerical value used to uniquely identify a file, which can be implemented by UUID, hash value or other unique coding methods, aiming to ensure that each file has an independent and unique identity and avoid naming conflicts. The physical storage descriptor can be understood as an information structure describing the storage location of the file on the local SSD, which can be implemented by recording logical block address ranges, partition numbers or file system block mapping tables, aiming to provide accurate location guidance for subsequent data access. The main metadata area and the compact metadata area respectively refer to the metadata storage areas embedded in the file header and the file tail, which are designed to optimize the access efficiency of metadata and enable the GPU to quickly parse key information.

[0035] In this embodiment, the above technical solution achieves the goal of efficient initialization of file metadata through the coordinated operation of multiple steps. The CPU allocates a file unique identifier in the Lustre file system, which ensures the uniqueness of the file in the entire system and lays the foundation for subsequent metadata management. Then, the physical space is pre-allocated on the local SSD and the logical block start address is recorded, which reduces the delay caused by dynamic allocation of space and improves the initialization efficiency. Then, based on the file identifier and the physical storage descriptor, the initial file metadata is constructed by combining the access control information and the state marker, which ensures the integrity and consistency of the metadata. Finally, the initial file metadata is written into the main metadata area and the compact metadata area. This design of separate area storage not only facilitates direct access by the GPU, but also optimizes the update and synchronization process of metadata, solving the problems of low efficiency and insufficient accuracy in the initialization step, and providing efficient foundation support for GPU direct access to the local SSD. At the same time, this scheme is closely integrated with the overall architecture of the Lustre client, and by optimizing the generation and storage method of metadata, the overall performance of the system is significantly improved, especially in high-performance computing scenarios, which exhibits outstanding technical advantages.

[0036] In one possible implementation, the step that the GPU initiates a file open request through a Lustre client POSIX interface to obtain a file identifier and access permission, and directly accesses the reduced metadata area based on the file identifier and access permission to extract a physical storage descriptor, a state tag and check information includes: the GPU sends a file open request through a Lustre client POSIX interface; the CPU detects the file open request to obtain file information, and writes a file identifier and initial access permission into shared memory; the GPU reads the file identifier and initial access permission in the shared memory; the GPU directly reads the reduced metadata area based on the file identifier through an NVMe protocol; the GPU parses content of the reduced metadata area to extract the physical storage descriptor, the state tag and the check information; and the GPU verifies validity of the metadata using the check information, and notifies the CPU to resynchronize file metadata if the verification fails.

[0037] In this embodiment, the shared memory refers to a memory area that allows efficient data transmission between multiple processors (such as a CPU and a GPU), which can be implemented by using a unified memory architecture or a specific shared buffer, and the purpose is to reduce communication delay between the CPU and the GPU and improve data transmission efficiency. In actual application, the file open request can be understood as a standard file operation instruction, which can be generated through a POSIX compatible interface, and the purpose is to ensure seamless integration with existing file systems. In addition, the reduced metadata area is an optimized metadata storage structure, which can be implemented by using a fixed length field or a compression coding mode, and the purpose is to reduce the complexity of metadata access and improve the reading speed.

[0038] In this embodiment, the GPU of the above technical solution sends a file open request through a POSIX interface, which triggers detection of the request by the CPU and writes key information (a file identifier and initial access permission) into shared memory, thereby avoiding frequent CPU intervention. The GPU quickly obtains these information from the shared memory, and directly accesses the reduced metadata area based on the file identifier through an NVMe protocol, which fully utilizes the low delay characteristics of the NVMe protocol and significantly improves the metadata reading efficiency. After parsing the content of the metadata area, the GPU extracts a physical storage descriptor, a state tag and check information, which provides necessary support for subsequent file operations. Finally, the validity of the metadata is verified through the check information, and if the metadata is found to be inconsistent, the CPU is notified to resynchronize, thereby ensuring the accuracy and consistency of the data.

[0039] On this basis, the above technical solutions form a close technical association with the foregoing. For example, by combining the process of creating a file and initializing file metadata by the Lustre client, the integrity and consistency of the metadata are ensured; at the same time, by introducing a shared memory mechanism, the direct interaction between the CPU and the GPU is reduced, thereby effectively solving the problem that the GPU cannot efficiently obtain a file identifier and access permissions. In addition, the introduction of the check information further enhances the data reliability and reduces the risk of system failure caused by metadata errors. Through the above technical solutions, not only is the efficiency of the GPU directly accessing the local SSD significantly improved, but also a solid foundation is laid for subsequent file operations, thereby comprehensively optimizing the overall performance.

[0040] In a feasible implementation, directly reading and writing file data on the local SSD through the NVMe protocol includes: the GPU reads file data through the NVMe protocol based on the logical block start address in the physical storage descriptor; the GPU updates the access timestamp in the state marker to generate an updated state marker, and synchronizes the updated state marker to the reduced metadata area.

[0041] In this embodiment, the physical storage descriptor refers to an information structure for describing the storage location of a file on the local SSD, which can include a logical block start address, an allocated space size, and the like. The purpose is to provide the GPU with accurate physical positioning capability, thereby avoiding additional CPU query operations. In actual applications, the logical block start address can be a mapping form of a logical block address (LBA) in the SSD device, which can be directly parsed and used by the GPU through the NVMe protocol. The state marker can be understood as a lightweight metadata structure, which is used to record the state information of the file, such as the access timestamp, the dirty data marker, and the like. The purpose is to support efficient cache management and consistency maintenance. The update of the access timestamp can be implemented by a time counter inside the GPU, or can be completed with the aid of an external clock source. The purpose is to provide an accurate basis for subsequent cache management mechanisms based on the timestamp.

[0042] In this embodiment, the above technical solution solves the key problems of metadata update and synchronization by explicitly specifying the operation flow of GPU when directly reading and writing local SSD. First, GPU uses the logical block start address in the physical storage descriptor to directly locate and read file data through the NVMe protocol. This process reduces the involvement of CPU and significantly reduces access delay. Second, after reading is completed, GPU updates the access timestamp in the state marker. This operation ensures that the latest access time of the file is accurately recorded, providing reliable data support for subsequent aging mechanisms or cache eviction strategies. Finally, the updated state marker is synchronized to the reduced metadata area. This synchronization operation ensures the consistency of metadata and avoids cache management failure or performance bottlenecks caused by delayed updates. Through the organic combination of the above steps, not only efficient data access is achieved, but also real-time and accuracy of metadata management are ensured, thereby improving the reliability and overall performance of GPU direct access to local SSD.

[0043] In this embodiment, the above technical solution also closely cooperates with basic steps such as initializing file metadata, embedding processing main metadata area and reduced metadata area. By pre-allocating physical storage descriptors during file creation and embedding reduced metadata area at the end of the file, necessary metadata support is provided for GPU direct access. This design enables GPU to complete data reading and metadata updating without CPU intervention, fully utilizes the parallel processing capability of GPU, reduces the burden of CPU, and further optimizes the overall performance of the system.

[0044] In a feasible implementation, directly reading and writing file data on local SSD through NVMe protocol includes: GPU checks the write permission in the access control information; if file space needs to be expanded, GPU sends an expansion request to CPU; CPU allocates additional space and updates file metadata to generate new physical storage descriptors and updated file metadata; GPU obtains new physical storage descriptors; GPU writes file data through NVMe protocol based on the new physical storage descriptors; GPU updates the dirty data marker, modification timestamp and file size in the state marker to generate updated state marker; GPU uses atomic operation to synchronize the updated state marker and updated file metadata to the reduced metadata area and the main metadata area, respectively.

[0045] In this embodiment, the access control information refers to a data structure or mechanism used to define and limit access rights to resources, which can be implemented in the form of an access control list (ACL), role-based access control (RBAC), etc., with the purpose of ensuring that only subjects with appropriate permissions can perform specific operations, thereby ensuring data security and compliance. The physical storage descriptor refers to a data structure that describes the allocation of physical space on a storage device, which can be implemented in the form of a logical block address (LBA) range, a storage segment mapping table, etc., with the purpose of providing accurate spatial positioning information for data storage. The status marker refers to identification information used to record the current state of a file, which can be implemented in the form of a bitmap, status code, etc., with the purpose of reflecting the operational state and attribute changes of the file in real time.

[0046] In this embodiment, when the GPU needs to perform a write operation, the write permission in the access control information is first checked to verify the legality of the operation. If insufficient file space is detected, an expansion request is sent to the CPU, which is responsible for allocating additional space and updating the file metadata, generating a new physical storage descriptor and updated file metadata. This process allows the GPU to efficiently utilize the resources of the CPU to handle complex allocation tasks when encountering space limitations, avoiding operation interruptions due to insufficient space. Subsequently, the GPU directly writes file data based on the new physical storage descriptor through the NVMe protocol, reducing CPU intervention and data duplication, and significantly improving write operation efficiency. After completing the write, the GPU updates the dirty data marker, modification timestamp, and file size in the status marker to reflect data state changes in real time, ensuring that the metadata and file content remain synchronized. Finally, the updated status marker and file metadata are synchronized to the reduced metadata area and main metadata area through atomic operations, ensuring the atomicity and consistency of metadata updates, preventing data conflicts or loss in concurrent operations. Not only does this solve the space expansion requirements and metadata consistency challenges that may arise in direct GPU write operations, but it also provides an efficient collaborative processing mechanism. This scheme achieves a balance between performance optimization and data security through reasonable division of labor between the GPU and CPU, fully utilizing the parallel processing capabilities of the GPU and the advantages of the CPU in handling complex allocation tasks. At the same time, the introduction of atomic operations and real-time update mechanisms for status markers further enhances the reliability and stability of the system, providing strong support for high-performance data access in GPU computing environments.

[0047] In an implementable embodiment, the allocation and management of the NVMe I / O queue in the GPU memory includes: the GPU allocating a NVMe submission queue and a completion queue in the memory; a GPU thread constructing a NVMe I / O command and writing the command into the allocated submission queue; the GPU notifying the NVMe controller to process the written I / O command through a doorbell mechanism; the NVMe controller writing completion information back to the allocated completion queue; and a GPU thread polling the allocated completion queue to obtain operation results.

[0048] In the embodiment, the NVMe submission queue refers to a data structure for storing I / O commands to be processed, which can be implemented in a ring buffer or a linked list structure, aiming to provide efficient command storage and management capabilities. The NVMe completion queue refers to a data structure for storing the results of completed I / O operations, which can also be implemented in a ring buffer or a linked list structure, aiming to ensure that operation results can be quickly fed back to GPU threads. The doorbell mechanism is a hardware notification mechanism, which can be implemented through register writing or memory-mapped I / O, aiming to reduce CPU intervention and improve notification efficiency.

[0049] In the above scheme, the GPU allocates a NVMe submission queue and a completion queue in the memory, avoiding the overhead of data transmission between the CPU and the GPU, thereby significantly reducing the memory copy delay. The process of the GPU thread constructing a NVMe I / O command and writing the command into the submission queue fully utilizes the parallel processing capability of the GPU, greatly improving the efficiency of command generation. Through the doorbell mechanism, the GPU can directly communicate with the NVMe controller, eliminating the need for CPU intervention in the notification process in traditional schemes, thereby effectively reducing the notification delay. In addition, after the NVMe controller writes the completion information back to the completion queue, the GPU thread obtains the operation status in real time through active polling, avoiding the additional burden brought by the interrupt mechanism or CPU polling, further improving the overall response speed and performance of I / O operations. Based on the above technical scheme, the GPU can efficiently complete I / O operations without CPU intervention, solving the problems of increased I / O operation delay, insufficient utilization of GPU parallel capabilities, and potential CPU intervention risks caused by the lack of specific mechanisms. At the same time, the scheme is combined with the technology of directly reading and writing local SSD file data through physical storage descriptors and status markers, further optimizing the overall performance and ensuring that the advantages of the GPU in high-performance computing scenarios can be fully utilized.

[0050] In a feasible implementation, when the GPU closes the file, the CPU is notified through shared memory; the step of reading the updated file metadata in the main metadata area by the CPU to verify consistency and trigger the dirty data migration mechanism includes: after the GPU completes the file operation, the updated file metadata is written to the main metadata area; the GPU sets a synchronization flag and notifies the CPU through shared memory; the CPU reads the file metadata in the main metadata area based on the notification to verify consistency; the CPU updates the verified file metadata to the Lustre persistent cache table and triggers dirty data migration according to the dirty data flag.

[0051] In this embodiment, the main metadata area refers to the metadata storage area embedded in the file header, which can be implemented in a specific format of data structure, such as key-value pairs or fixed-length field arrangement, aiming to centrally store the core metadata of the file for efficient access and management. The synchronization flag is a signal mechanism used to indicate that the GPU has completed the file operation and is ready to interact with the CPU, which can be implemented through a specific variable or register state in shared memory, aiming to reduce communication delay and improve notification efficiency. The dirty data flag is used to identify the part of the file metadata that needs to be synchronized or migrated, which can be implemented in the form of bitmap or Boolean value, aiming to accurately trigger necessary data migration operations.

[0052] In this embodiment, the overall operation logic of the above scheme is as follows: first, after completing the file operation, the GPU writes the updated file metadata to the main metadata area, which ensures that the latest state of the metadata is fixedly stored, avoiding data loss or conflict caused by asynchronous operation. Subsequently, the GPU sets the synchronization flag and notifies the CPU through shared memory, taking advantage of the low-latency characteristics of shared memory to achieve efficient signal transmission, thereby avoiding the resource waste caused by traditional polling method. Then, after receiving the notification, the CPU reads the updated file metadata from the main metadata area and verifies its consistency, which ensures the accuracy and integrity of the data. Finally, the CPU updates the verified file metadata to the Lustre persistent cache table and triggers dirty data migration according to the dirty data flag, thereby achieving the goal of synchronizing the system state and optimizing the data storage performance. The whole process effectively improves the system efficiency and reliability during file closing through step-by-step coordination and verification mechanism. And the above scheme forms a close technical association with the foregoing content. For example, in the process of GPU directly accessing local SSD and finally synchronizing with the Lustre file system, this scheme solves the efficient coordination problem between GPU and CPU through explicit operation sequence and verification mechanism, while ensuring the accuracy and timeliness of metadata update. This design not only reduces communication delay, but also significantly reduces the risk of data consistency, thereby providing guarantee for the overall performance and reliability of the system.

[0053] In a feasible implementation, migrating data of the local SSD to the Lustre file system includes migration based on an aging mechanism or migration based on an AI prediction mechanism. The migration based on the aging mechanism includes: monitoring a residence time of file data of an operation in the local SSD; when the residence time exceeds a preset threshold, automatically migrating the file data to the Lustre file system; and updating file attributes in a Lustre global namespace to synchronize the migrated file data. The migration based on the AI prediction mechanism includes: an AI application predicting that file data of an operation will not be accessed in the future; based on a result of the prediction operation, migrating a file or a directory to the Lustre file system; and updating a migration state in a state flag after the migration is completed.

[0054] In this embodiment, the aging mechanism refers to a strategy of determining whether migration is needed by monitoring a residence time of file data in the local SSD, which can be implemented by setting a timer or recording a time stamp, and the purpose is to ensure that the local SSD does not affect efficient use of storage resources due to long-term occupation of data. The AI prediction mechanism can be understood as an intelligent decision tool based on a machine learning algorithm, which can generate a prediction result according to historical access patterns, user behaviors and other data, so as to determine whether to migrate a file or a directory, and the purpose is to reduce unnecessary migration operations and optimize resource allocation.

[0055] In this embodiment, the migration based on the aging mechanism is to continuously monitor the residence time of the file data in the local SSD, and when it is detected that the residence time of certain file data exceeds a preset threshold, a migration process is triggered to migrate the file data from the local SSD to the Lustre file system, and file attributes in the Lustre global namespace are updated synchronously to ensure data consistency. The migration based on the AI prediction mechanism relies on the prediction ability of the AI application on future access demand of the file, and when the prediction result shows that certain file or directory will not be accessed in the future for a period of time, the system actively migrates it to the Lustre file system, and updates the migration state in the state flag after the migration is completed to reflect the latest system state. The two mechanisms work together to intelligently decide the migration operation according to the actual data usage mode, thereby reducing CPU intervention and improving data access efficiency.

[0056] In this embodiment, by introducing the aging mechanism and AI prediction mechanism, not only the problem of insufficient intelligence in migration decision is solved, but also the cache management is optimized, and the performance bottleneck is reduced. For example, in the GPU computing environment, the aging mechanism can effectively avoid excessive residence of data in the local SSD, while the AI prediction mechanism can remove data that may no longer be accessed in advance, thereby achieving more efficient resource allocation and lower access latency. The application of this intelligent migration strategy enables the system to better meet the demand of GPU direct access to local SSD in high-performance computing scenarios, while maintaining efficient synchronization with the Lustre file system.

[0057] In a feasible implementation, migrating back to the local SSD from the Lustre file system includes: automatically migrating data in the Lustre file system back to the local SSD when reading the data; or, AI application prediction will read the file, and pre-migrate the data back to the local SSD, and update the file metadata based on the migrated data in the local SSD and register to the persistent cache table to complete synchronization.

[0058] In this embodiment, the passive triggering mechanism refers to triggering the data migration process only when the actual access demand occurs, which can be realized by monitoring file access requests, and the purpose is to avoid unnecessary data migration operations, thereby saving storage resources and computing overhead. Among them, the active prediction mechanism refers to using artificial intelligence technology to predict the future access mode, which can be realized by machine learning model or deep learning algorithm, and the purpose is to migrate the data that may be accessed back to the local SSD in advance, thereby eliminating the delay risk caused by waiting for migration. In addition, the process of updating file metadata and registering to the persistent cache table refers to the operation of ensuring file consistency and traceability after data migration, which can be realized by atomic operation or transaction management mechanism, and the purpose is to prevent data conflict and support subsequent efficient access.

[0059] In this embodiment, the above scheme realizes intelligent migration of data through two different triggering modes. When a user or an application initiates a read request to the Lustre file system, the system detects the request and automatically migrates the corresponding data back to the local SSD, which does not require additional manual intervention and can effectively reduce access delay. At the same time, the AI application predicts the files that may be accessed in the future by learning and analyzing historical access records, and migrates these files back to the local SSD in advance, thereby further optimizing data access performance. On this basis, the migrated data updates the file metadata in the local SSD and completes system synchronization by registering to the persistent cache table, which not only guarantees data consistency but also provides support for subsequent efficient access. Through the above technical scheme, the potential inefficiency in the migration mechanism is solved, the data access delay is reduced, and the system resource utilization is improved. At the same time, the passive triggering and active prediction mechanisms introduced can flexibly adjust the data migration strategy according to actual needs, thereby better adapting to complex computing environments. In addition, the synchronization mechanism of the persistent cache table further enhances the stability and reliability of the system, enabling the full play of GPU computing performance.

[0060] In the embodiments of the present application, the GPU direct access based Lustre client persistent cache performance optimization method avoids the bottleneck of CPU as a data transfer by directly accessing the local SSD through GPU and optimizing the metadata management mechanism, reduces redundant memory copy operations, effectively solves the performance limitation problem caused by the fact that GPU cannot directly access the storage device in the prior art, and can reduce data transmission delay, improve system throughput, reduce CPU resource consumption and fully exert the parallel processing capability of GPU.

[0061] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the GPU direct access based Lustre client persistent cache performance optimization method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0062] The present application also provides a GPU direct access based Lustre client persistent cache performance optimization system, which refers to Figure 2 The GPU direct access based Lustre client persistent cache performance optimization system comprises a memory 10, a processor 20 and a GPU direct access based Lustre client persistent cache performance optimization program stored on the memory 10 and executable on the processor 20, wherein the GPU direct access based Lustre client persistent cache performance optimization program is configured to implement the steps of the GPU direct access based Lustre client persistent cache performance optimization method.

[0063] The GPU direct access based Lustre client persistent cache performance optimization system provided in the application adopts the GPU direct access based Lustre client persistent cache performance optimization method in the above embodiment, can reduce data transmission delay, improve system throughput, reduce CPU resource consumption and fully exert the parallel processing capability of the GPU. Compared with the prior art, the GPU direct access based Lustre client persistent cache performance optimization system provided in the application has the same beneficial effects as the GPU direct access based Lustre client persistent cache performance optimization method provided in the above embodiment, and other technical features in the GPU direct access based Lustre client persistent cache performance optimization system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0064] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0065] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A method for performance optimization of a GPU direct access based Lustre client persistent cache, characterized in that, The method comprises: Creating a file through a Lustre client and obtaining system information to initialize file metadata; wherein the file metadata comprises a file identifier, a physical storage descriptor, access control information, a status flag and check information; Embedding the initialized file metadata to generate a main metadata area embedded in a file header and a simplified metadata area embedded in a file tail, and registering the file information in a persistent cache table through the Lustre client; The GPU initiates a file opening request through a Lustre client POSIX interface to obtain a file identifier and access permissions, and directly accesses the simplified metadata area based on the file identifier and access permissions to extract a physical storage descriptor, a status flag and check information; The GPU allocates and manages an NVMe I / O queue in GPU memory based on the physical storage descriptor and the status flag, and directly reads and writes file data on a local SSD through an NVMe protocol; wherein the read and write operations comprise updating the status flag or updating the file metadata, and synchronizing the updated status flag or file metadata to the corresponding metadata area; When the GPU closes the file, the CPU is notified through shared memory; the CPU reads the updated file metadata in the main metadata area to verify consistency and trigger a dirty data migration mechanism; Based on an aging mechanism or an AI prediction mechanism, data of the local SSD is migrated to or from the Lustre file system to optimize data access performance.

2. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, The step of creating a file through a Lustre client and obtaining system information to initialize file metadata comprises: Allocating a file unique identifier in the Lustre file system through a CPU to obtain a file identifier; Pre-allocating physical space on a local SSD and recording a logical block starting address to generate a physical storage descriptor; Based on the file identifier and the physical storage descriptor, combining access control information and a status flag to construct initial file metadata; Writing the initial file metadata to generate a main metadata area embedded in a file header and a simplified metadata area embedded in a file tail.

3. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, The step of the GPU initiating a file opening request through a Lustre client POSIX interface to obtain a file identifier and access permissions, and directly accessing the simplified metadata area based on the file identifier and access permissions to extract a physical storage descriptor, a status flag and check information comprises: The GPU sends a file opening request through a Lustre client POSIX interface; The CPU detects the file opening request to obtain file information, and writes a file identifier and initial access permissions to shared memory; The GPU reads the file identifier and initial access permissions in the shared memory; The GPU directly reads the simplified metadata area through an NVMe protocol based on the file identifier; The GPU parses the content of the simplified metadata area to extract a physical storage descriptor, a status flag and check information; The GPU uses the check information to verify the validity of the metadata, and if the verification fails, the CPU is notified to re-synchronize the file metadata.

4. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, Directly reading and writing file data on the local SSD through the NVMe protocol includes: The GPU reads file data through the NVMe protocol based on the logical block start address in the physical storage descriptor; The GPU updates the access timestamp in the state marker to generate an updated state marker, and synchronizes the updated state marker to the reduced metadata area.

5. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, Directly reading and writing file data on the local SSD through the NVMe protocol includes: The GPU checks the write permissions in the access control information; If the file space needs to be expanded, the GPU sends an expansion request to the CPU; the CPU allocates additional space and updates the file metadata to generate new physical storage descriptors and updated file metadata; the GPU obtains the new physical storage descriptors; The GPU writes file data through the NVMe protocol based on the new physical storage descriptors; The GPU updates the dirty data marker, modification timestamp, and file size in the state marker to generate an updated state marker; The GPU uses atomic operations to synchronize the updated state marker and updated file metadata to the reduced metadata area and the main metadata area, respectively.

6. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, Allocating and managing NVMe I / O queues in GPU memory includes: The GPU allocates NVMe submission queues and completion queues in memory; GPU threads construct NVMe I / O commands and write the commands to the allocated submission queues; The GPU notifies the NVMe controller to process the written I / O commands through the doorbell mechanism; The NVMe controller writes completion information back to the allocated completion queue; GPU threads poll the allocated completion queue to obtain operation results.

7. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, When the GPU closes the file, it notifies the CPU through shared memory; The steps of the CPU reading the updated file metadata in the main metadata area to verify consistency and trigger the dirty data migration mechanism include: After the GPU completes the file operation, it writes the updated file metadata to the main metadata area; The GPU sets a synchronization flag and notifies the CPU through shared memory; The CPU reads the file metadata in the main metadata area based on the notification to verify consistency; The CPU updates the verified file metadata to the Lustre persistent cache table and triggers dirty data migration based on the dirty data marker.

8. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, Migrating data from the local SSD to the Lustre file system includes migration based on the aging mechanism or migration based on the AI prediction mechanism, where the migration based on the aging mechanism includes: Monitoring the residence time of the operated file data in the local SSD; When the residence time exceeds a preset threshold, automatically migrating the file data to the Lustre file system; Updating the file attributes in the Lustre global namespace to synchronize the migrated file data; The migration based on the AI prediction mechanism includes: The AI predicts that the operated file data will not be accessed in the future; Based on the prediction operation result, migrating the file or directory to the Lustre file system; After the migration is completed, the migration state in the state flag is updated.

9. The method for performance optimization of GPU direct access based Lustre client persistent cache according to claim 1, wherein, Migrating back to local SSD from Lustre file system includes: When reading data in the Lustre file system, automatically migrating the data back to the local SSD; Or, the AI application predicts that a file will be read, and pre-migrates the data back to the local SSD, and updates the file metadata based on the migrated data in the local SSD and registers to the persistent cache table to complete synchronization.

10. A system for performance optimization of a GPU direct access based Lustre client persistent cache, comprising: The GPU direct access-based Lustre client persistent cache performance optimization system includes a memory, a processor, and a GPU direct access-based Lustre client persistent cache performance optimization program stored on the memory and executable on the processor, and the GPU direct access-based Lustre client persistent cache performance optimization program is configured to implement the steps of the GPU direct access-based Lustre client persistent cache performance optimization method according to any one of claims 1 to 9.

Images