Lustre文件系统写处理方法与装置
By setting a preset buffer in the Lustre client kernel mode and asynchronously processing distributed lock requests, the problem of slow Checkpoint file writing speed is solved, achieving efficient data writing and system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNICOM DIGITAL TECNOLOGY CO LTD
- Filing Date
- 2026-04-29
- Publication Date
- 2026-07-17
AI Technical Summary
In AI model training scenarios, the slow writing speed of checkpoint files leads to low training efficiency. The existing data writing mechanism of the Lustre file system is time-consuming and cannot meet the low latency requirements of hundreds of microseconds.
In the Lustre client kernel mode, a preset buffer is set up. When the conditions for accelerated writing are met, the data to be written is copied to the preset buffer and a write success is returned immediately. The distributed lock requests are processed asynchronously in batches using a background thread, and the process is downgraded to a page-by-page write process to ensure system stability.
By reducing network round-trip latency for lock requests, data writing efficiency is significantly improved, low latency requirements are met, and the system maintains stable operation under memory pressure.
Smart Images
Figure CN122111964B_ABST