Lightweight data storage access method, apparatus and server

Abstract

The application discloses a lightweight data storage access method and device and a server, and relates to the field of data storage. According to the scheme, IPMI packet data is written into SPI NOR Flash and marked as a temporary state, a commit control frame is written after complete reception and verification, and transaction-level atomicity commit is realized; a BMC maintains a flattened index table in SRAM, points to the physical offset address of the latest commit control frame through an atomic pointer, and realizes logical visibility switching. A background lock-free asynchronous garbage collection mechanism is introduced, effective data frames are effectively moved through rolling, and the atomic pointer is switched, so that front and back GCs are decoupled. When an abnormal power failure occurs, the BMC loads a snapshot frame to recover a basic index, sequentially scans a subsequent physical interval, and identifies effective transactions, so that automatic rollback is realized by filtering incomplete packets. The scheme solves the problem of power failure tearing under packet transmission, eliminates the blocking delay caused by Flash erasing and writing, and has the advantages of high atomicity, good real-time performance and low memory overhead.

Description

Technical Field

[0001] This application relates to the field of data storage, and in particular to a lightweight data storage access method, device, and server based on an intelligent platform management interface-based file system. Background Technology

[0002] In modern server architectures, the BIOS or operating system often needs to exchange non-volatile data (such as custom SMBIOS strings, crash logs, OEM configuration information, etc.) with the BMC. The BMC chip (such as the ASPEED series) is usually connected to a limited capacity (such as 32MB~64MB) SPI NOR Flash as its non-volatile storage medium via the SPI bus.

[0003] However, when using the traditional IPMI specification or out-of-band channel to read and write data to SPI NOR Flash, the following serious pain points exist: 1. Protocol Packet Size Limitations and Power-Loss Tear: The IPMI protocol only supports a maximum of 256 bytes per command. A large file (such as a 4KB crash log) must be split into dozens of packets. Due to the extremely small size of the SRAM (memory) inside the BMC (Browser Control Center) (typically only a few hundred KB), the firmware cannot allocate a large buffer in the SRAM to assemble packets; it can only append data to the SPI NOR Flash as it receives packets. If network packet loss, host restart, or abnormal power failure occurs during transmission, half of the "malformed and incomplete data" will be left in the Flash, making the file system extremely vulnerable to corruption.

[0004] 2. Read / Write Mutual Exclusion and IPMI Interface Timeout Risks: SPI NOR Flash has the physical characteristic of being able to erase only by block / sector (Erase-before-Write) with high erase latency. Erasing a 4KB sector or 64KB block typically takes hundreds of milliseconds. If a traditional in-place erase / write scheme is used directly in the firmware, or when old blocks are erased during garbage collection (GC), once a high-priority IPMI read / write request is sent from the host, the foreground thread will be forced to wait for the Flash hardware to unload from its busy state. This can easily lead to IPMI interface protocol timeouts (usually with a maximum of 1-3 seconds), which can cause the system to hang during POST or the system daemon process to malfunction.

[0005] 3. Traditional FTL or journaling file systems are not portable: The Flash Translation Layer (FTL) or general journaling file systems (such as JFFS2 / YAFFS) in standard firmware require a large amount of memory (RAM) resources and CPU computing power to maintain complex physical-logical address mapping tables (L2P). This is completely unacceptable for the BMC firmware layer, which has extremely limited memory resources, only a small amount of SRAM, and no large-capacity DRAM mapping space. Summary of the Invention

[0006] This application provides a lightweight data storage access method, device, and server, which has transaction-level power-loss safety capabilities, effectively prevents file system corruption caused by data residue, and significantly reduces memory resource consumption.

[0007] On one hand, this application provides a lightweight data storage access method applied in a Baseboard Management Controller (BMC), wherein the physical storage medium of the BMC is a Serial Peripheral Interface (SPI) NOR Flash memory, and the method includes: The system receives multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; without going through the memory packet buffer of the BMC, the multiple data packets are directly appended to the real-time write block of the SPI NOR Flash in the order of receipt, and the physical state of the written data packets in the flash memory is marked as temporary. Verify the received data packets. If and only if it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, write a commit control frame at the current append position of the real-time write block and mark the physical state of the commit control frame in the flash memory as valid. A flattened index table is maintained in the static random access memory (SRAM) of the BMC. The flattened index table records the mapping relationship between file number and the latest valid physical offset address. After the commit control frame is successfully written to disk, the pointer corresponding to the file number in the flattened index table is switched to the physical offset address of the commit control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

[0008] Specifically, maintaining a flattened index table in the BMC's SRAM includes: A fixed-length flat hash table or dynamic array is maintained in the SRAM; In the flattened index table, the key is the file number, and the value is an atomic pointer that points to a lightweight structure. The structure records at least the physical offset address of the data corresponding to the file number and the file size.

[0009] The payload header of the data packet encapsulates transaction control information, which includes at least one of the following: a globally unique transaction serial number, the total number of packets, the current packet sequence number, and the file number. The global verification passing means that: the current packet sequence number of the currently received data packet is equal to the total number of packets, and the cyclic redundancy check (CRC) code of the entire batch of data packets is correctly matched; the global unique transaction serial number is recorded in the submission control frame.

[0010] Specifically, the method further includes a snapshot control frame saving step: When the space of the current real-time write block of the SPI NOR Flash is full, or when an explicit synchronization command is received from the host, the BMC packages the current flat index table as a single snapshot control frame. Write the snapshot control frame to the end of the dedicated reserved area of ​​the SPI NOR Flash or the current real-time write block.

[0011] Specifically, the method further includes a rapid recovery step for abnormal power loss based on the snapshot control frame, including: After a system abnormal reset or power-on, the SPI NOR Flash is scanned in reverse to find the latest written snapshot control frame, and the snapshot control frame is loaded into the SRAM to restore the base file mapping index. Starting from the cutoff physical address recorded in the snapshot control frame, the remaining data frames are scanned sequentially backward. If a data packet with a physical state identifier of temporary state is scanned, and no matching commit control frame is scanned in the subsequent physical interval, it is determined to be an incomplete transaction. The temporary state data packet is ignored when reconstructing the memory index, thereby achieving transaction-level power-loss safe rollback.

[0012] Specifically, the method also includes a lock-free asynchronous garbage collection step that decouples the front-end and back-end, including: When the available space of the SPI NOR Flash is lower than a set threshold, the background garbage collection thread allocates new physical blocks that have been erased in advance; The old physical blocks to be reclaimed are scanned sequentially, valid data frames are identified and asynchronously rolled and copied into the new physical blocks, and the pointers in the flattened index table remain unchanged during this physical copying. If a high-priority IPMI read request is received in the foreground during the physical replication, the foreground read thread does not wait to acquire the mutex lock, but directly reads the required data frame from the old physical block based on the existing pointers in the flattened index table.

[0013] Specifically, in the lock-free asynchronous garbage collection steps that decouple the front-end and back-end: After all the valid data frames are copied to the new physical block, the background garbage collection thread issues a single atomic instruction to update the corresponding pointer in the flattened index table to the physical offset address pointing to the new physical block; After confirming that all valid file pointers in the old physical block have been safely switched, an asynchronous erase command is sent to the underlying controller to erase the old physical block.

[0014] Specifically, the method further includes foreground write conflict redirection and lock-free self-healing steps for the asynchronous garbage collection step, including: If, during the background garbage collection thread's copying of valid data frames with the same file number, the foreground thread suddenly receives a write request for the same file number, the foreground thread will not block and wait, but will directly redirect and append the newly received data packet to the end of the current foreground real-time write block. After the commit control frame corresponding to the newly received data packet is written, the foreground thread preemptively rewrites the atomic pointers in the flattened index table. Before completing the physical copy and preparing to rewrite the flattened index table, the background garbage collection thread compares the address actually pointed to by the current flattened index table with the physical address of the old data source. If the two are found to be inconsistent, a conflict comparison inconsistency mechanism is triggered, the pointer switching operation for the same file number is actively abandoned, and the copy to the new physical block is marked as invalid.

[0015] On the other hand, this application provides a lightweight data storage access method apparatus applied in a baseboard management controller (BMC), wherein the physical storage medium of the BMC is a serial peripheral interface non-volatile flash memory (SPI NOR Flash), and the apparatus includes: Packet streaming append module: used to receive multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; to directly append the multiple data packets into the real-time write block of the SPI NOR Flash in the order of receipt without going through the memory packet buffer of the BMC; and to mark the physical state of the written data packets in the flash memory as temporary state. Transaction control commit module: used to verify the received data packets. When it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, a commit control frame is written at the current append position of the real-time write block, and the physical state of the commit control frame in the flash memory is marked as valid. Index mapping update module: used to maintain a flattened index table in the static random access memory (SRAM) of the BMC. The flattened index table records the mapping relationship between file number and the latest valid physical offset address. After the commit control frame is successfully written to disk, the pointer corresponding to the file number in the flattened index table is switched to the physical offset address of the commit control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

[0016] In another aspect, this application also provides a server, including a host, a baseboard management controller (BMC), and a physical storage medium SPI NOR Flash connected to the BMC via an SPI bus, wherein the BMC integrates a static random access memory (SRAM). The host is used to split variable-length files belonging to the same file number into multiple data packets and send them to the BMC based on the Intelligent Platform Management Interface (IPMI) protocol. The BMC is equipped with a processor and a firmware memory. The firmware memory stores a computer program, which, when executed by the processor, implements the aforementioned lightweight data storage and access method.

[0017] The beneficial effects of the technical solution provided in this application include at least the following: by using a packet-based streaming appending step, data packets are directly appended to the real-time write block of the SPI NOR Flash without going through the BMC memory packet caching, and the data is marked as temporary. This design avoids dependence on the limited SRAM of the BMC, fundamentally solves the packet caching problem caused by limited memory resources, and effectively prevents the problem of invalid data polluting the file system due to transmission interruptions; A transaction control commit step is introduced, whereby a commit control frame is written and marked as valid only after the last data packet has been received and global validation has passed. This mechanism uses the commit control frame as the atomic boundary of the transaction, ensuring that only complete and correctly validated data is recognized as valid by the system. This effectively prevents malformed data residue caused by transmission interruptions and significantly improves the reliability of data storage.

[0018] By maintaining a flat index table in SRAM and utilizing low-level atomic operations to switch the file number pointer to the physical offset address of the commit control frame, extremely low memory overhead and efficient address mapping are achieved. This flat index structure and atomic pointer switching mechanism not only solve the problem of limited BMC memory resources but also ensure the real-time performance and consistency of file state transitions, avoiding the IPMI interface timeout risk that may be caused by complex locking mechanisms or long-running index update operations.

[0019] This application constructs a lightweight, highly reliable, and low-overhead data storage and access method through the synergistic effect of three core steps: packet-based streaming appending, transaction-controlled commit, and index mapping update. This method effectively solves key technical problems such as data integrity, power-loss safety, and performance efficiency in large file transfers, especially in application scenarios where BMC memory resources are extremely limited. Attached Figure Description

[0020] Figure 1 This is a flowchart of a lightweight data storage access method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the overall architecture of the server provided in the embodiments of this application; Figure 3 A schematic diagram illustrating the mechanism of transactional packet append writing and Commit control frames is shown; Figure 4 This is a schematic diagram of a flattened memory index and a fast power-off recovery mechanism; Figure 5 The diagram shows a lock-free asynchronous garbage collection mechanism that decouples the front-end and back-end. Figure 6 This diagram illustrates the foreground write preemption conflict redirection mechanism. Figure 7 This is a schematic diagram of a lightweight data storage access device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0022] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0023] For ease of understanding, the following explains some key terms in this embodiment: The Intelligent Platform Management Interface (IPMI) protocol is a standardized hardware management interface specification for out-of-band management of server hardware. This protocol allows the host to communicate with the Baseboard Management Controller (BMC) to monitor hardware status, configure system parameters, or transfer data.

[0024] The Baseboard Management Controller (BMC) is a dedicated microcontroller on the server motherboard responsible for monitoring the server's physical status, such as temperature, fan speed, and power supply voltage, and providing remote management capabilities. The BMC typically has its own dedicated processor and memory.

[0025] Serial Peripheral Interface (SPI) NOR Flash is a non-volatile storage medium connected via the Serial Peripheral Interface (SPI) bus. It is characterized by relatively fast read and write speeds, but erase operations are typically performed in blocks or sectors, and erase latency is relatively high.

[0026] Static Random Access Memory (SRAM) is a high-speed volatile memory integrated within the BMC. It is characterized by extremely fast read and write speeds, but its capacity is usually very limited, and data is lost when power is lost.

[0027] The purpose of this solution is to provide a lightweight out-of-band data storage solution that is file system-like, non-blocking, wear-resistant, and specifically designed for deep coupling between SPI NOR Flash and IPMI short packet streams. It addresses the blocking latency caused by millisecond-level erasure of SPI NOR Flash in situations where BMC SRAM resources are extremely limited, ensuring the real-time performance of the out-of-band channel. Simultaneously, it achieves transaction-level power-loss safe rollback under packetized transmission with extremely low memory overhead, making it highly suitable for resource-constrained BMC chips.

[0028] Therefore, this application provides a lightweight data storage and access method based on an intelligent platform management interface class file system, such as... Figure 1 As shown, it is applied in the Baseboard Management Controller (BMC), where the physical storage medium of the BMC is a Serial Peripheral Interface (SPI) NOR Flash. The method may include the following steps: Step 110, Packet Streaming Append Stage: Receive multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; without going through the BMC's memory packet buffer, directly append the multiple data packets to the real-time write block of the SPI NOR Flash in the order of receipt, and mark the physical state of the written data packets in the flash memory as temporary. This step receives multiple data packets belonging to the same file number sent by the host based on the IPMI protocol. After receiving these data packets, the multiple data packets are directly appended to the real-time write block of the SPI NOR Flash in the order of receipt, without going through the BMC's memory packet buffer.

[0029] For example, when the host needs to transfer a 4KB file, due to the packet size limitation of the IPMI protocol, the file will be split into multiple 256-byte data packets. After receiving the first data packet, the BMC does not cache it in SRAM for assembly; instead, it directly writes it to the beginning of the real-time write block of the SPI NOR Flash. The second data packet received subsequently is appended to the first data packet. This method avoids the problem of the BMC being unable to assemble large files due to limited SRAM capacity. After the data packet is written to flash memory, its physical state is marked as pendant. For example, this can be achieved by appending a specific status flag to the header or tail of the data packet. Furthermore, a separate metadata area can be maintained in flash memory to record the write status of each data packet.

[0030] Step 120, Transaction control commit phase: Verify the received data packets. If and only if it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, write a commit control frame at the current append position of the real-time write block and mark the physical state of the commit control frame in the flash memory as valid. During this phase, the received data packets are validated. A commit control frame (COMMIT frame) is written to the current append position of the real-time write block only if it is confirmed that the last data packet corresponding to the file number has been received and the global validation has passed. This COMMIT frame is a special data frame used to mark the successful commit of a complete data transaction. Its write position and status identifier play a crucial role in determining data validity.

[0031] For example, after receiving all data packets, the BMC can perform integrity checks on all received data packets, such as calculating the checksum of all data packets and comparing it with the expected value. If the checksum is correct and it is confirmed that the currently received data packet is the last data packet of that file number, a COMMIT control frame is written to the real-time write block immediately following the last data packet. This COMMIT control frame can be a fixed-size structure containing transaction identification information. After the COMMIT control frame is written to flash memory, its physical state is marked as valid (VALID). For example, this can be achieved by setting a valid flag in a specific field of the COMMIT control frame; or a transaction status table can be maintained in a specific area of ​​flash memory to record the validity of each COMMIT control frame.

[0032] Step 130 Index Mapping Update Phase: A flattened index table is maintained in the static random access memory (SRAM) of the BMC. The flattened index table records the mapping relationship between file numbers and the latest valid physical offset address. After the control frame is successfully written to disk, the pointer of the corresponding file number in the flattened index table is switched to the physical offset address of the control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

[0033] A flattened index table is a data structure maintained in the SRAM of the BMC (Block Memory Management System) to record the mapping between file numbers and the latest valid physical offset address of that file. This index table provides a mechanism for quickly finding the actual storage location of a file in flash memory. For example, the flattened index table could be a simple array, with each array item corresponding to a file number and storing the physical offset address of the data corresponding to that file number. Alternatively, a hash table could be used, using the file number as the key to quickly look up the corresponding physical offset address.

[0034] In some embodiments, a flattened index table is maintained in the SRAM of the BMC. Specifically, this can be a fixed-length flattened hash table or a dynamic array maintained in the SRAM. The key of the flattened index table is the file number, and the value is an atomic pointer that points to a lightweight structure. The structure records at least the physical offset address of the data corresponding to the file number and the file size.

[0035] With these two key pieces of information, the system can directly locate and read file data in the SPI NOR Flash without performing complex logical address to physical address conversion. This is distinct from the complex FTL mapping tree of general operating systems, thereby simplifying the file system design and reducing the consumption of memory and CPU resources, thus improving the stability and response speed of the entire server system.

[0036] Atomic operations are operations that, in a multi-threaded or concurrent environment, either succeed completely or fail completely, without any intermediate states. This ensures data consistency and integrity, especially when updating shared resources.

[0037] After the COMMIT control frame is successfully written to disk, a low-level atomic operation switches the pointer to the corresponding file number in the flattened index table to the physical offset address of the COMMIT control frame. For example, when the data of a file and its COMMIT control frame have been successfully written to flash memory, the BMC processor executes an atomic instruction to update the pointer corresponding to the file number in the flattened index table in SRAM from an old or invalid address to the physical offset address of the newly written COMMIT control frame. This atomic operation ensures the instantaneous switching of pointers, avoiding inconsistent states in a multi-tasking environment. As a result, the data corresponding to the file number enters a readable state visible to the foreground. This means that foreground applications or services can query the flattened index table to obtain the latest valid physical offset address of the file and read the complete and committed data starting from that address.

[0038] To more clearly illustrate the aforementioned lightweight data storage and access method, this application provides a server. Figure 2 The diagram shows the overall architecture of the server, including the host, the baseboard management controller (BMC), and the physical storage medium SPI NOR Flash 7 connected to the BMC via the SPI bus. The BMC integrates a static random access memory (BMC SRAM6). On the host side, based on the Intelligent Platform Management Interface (IPMI) protocol, variable-length files belonging to the same file number are split into multiple data packets and sent to the BMC. The BMC is equipped with a processor and firmware memory. The firmware memory stores computer programs, which are executed by the processor to implement lightweight data storage and access methods.

[0039] During runtime, when the underlying firmware UEFI BIOS 1 or the upper-level memory testing tool software 2 on the host side needs to write non-volatile data (such as memory error topology logs) to the BMC, the specific execution steps are as follows: The host-side IPMI packet driver layer 3 segments the large, variable-length file to be transmitted into packets according to the IPMI protocol's hardware limit of a maximum 256-byte single packet, forming a continuous IPMI OEM write command stream 5 which is then sent to the BMC. Upon receiving the write command stream 5, the BMC's physical layer receive buffer directly receives each packet, without needing to allocate a large packet assembly buffer in the BMC SRAM 6. Instead, it directly appends each data packet to the underlying SPI NOR Flash 7. At this time, the physical status flag of this data in transit in the Flash is set to "PENDING" by default. Before the data is completely transmitted successfully, the flat memory index table in the BMC SRAM 6 remains unchanged, preventing the host-initiated IPMI read command 4 from reading this incomplete temporary data, thus logically isolating dirty data.

[0040] In some embodiments, the payload header of the split data packets encapsulates transaction control information, which includes at least one of the following: a globally unique transaction sequence number, the total number of packets, the current packet sequence number, and the file number. In the transaction control commit step, the following indicators are used to verify that the global validation has passed: The current packet sequence number of the received data packet is equal to the total number of packets, and the CRC checksum of the entire batch of data packets is correct; the Commit control frame records a globally unique transaction serial number.

[0041] This solution encapsulates a globally unique transaction sequence number, total number of packets, current packet sequence number, and file number, enabling the BMC to accurately identify the ownership and timing of each packet. This allows for accurate judgment of data stream integrity without relying on large-capacity memory for packet assembly. During the transaction control commit step, comparing the current packet sequence number with the total number of packets ensures that all packets have arrived as expected, preventing transaction incompleteness due to missing packets. Simultaneously, introducing a Cyclic Redundancy Check (CRC) check of the entire batch of data packets effectively detects potential data bit errors during transmission, ensuring the accuracy of the data written to disk. The globally unique transaction sequence number recorded in the commit control frame provides a crucial identifier for subsequent transaction tracking and status confirmation, enabling the system to map commit actions one-to-one with specific transaction requests, thus building a reliable transaction confirmation mechanism at the underlying storage layer.

[0042] Figure 3This diagram illustrates the mechanism of transactional packet appending and the COMMIT control frame, using a memory testing tool sending a large log file as an example: The log file is divided into data packet 1, data packet 2, and a COMMIT control frame as an end marker. In the early stages of transmission, the BMC receives data packet 1 and data packet 2 sequentially and appends them directly to the SPIFlash physical storage area 4 in that order. At this time, their physical state is marked as "PENDING". During this period, the mapping pointer for the log file in the right-hand memory flat index table 5 remains unchanged, still pointing to the physical address of the historical old data, ensuring that the read thread will never read an incomplete file. The BMC writes the frame to the end of the SPIFlash physical storage area 4 and sets its physical state to VALID only when the BMC has completely received the last packet, i.e., the COMMIT control frame 3, and the global CRC check of the entire batch of data is completely correct. The instant the control frame is successfully written to disk, the BMC firmware is triggered to call the underlying atomic operation instruction, which instantly switches the atomic mapping pointer in the memory flat index table 5 to the physical offset address of the latest submitted VALID control frame (i.e., the COMMIT control frame that has become VALID). At this moment, the entire batch of streaming appended packets becomes logically visible to the foreground instantly, achieving atomic modification with high tear protection against power failure.

[0043] Thus, the entire process ensures the integrity and atomicity of data transmission through streaming writes and transaction control, even under conditions of limited BMC memory resources, and provides a fast and efficient indexing mechanism, enabling large files to be reliably stored and accessed.

[0044] In summary, traditional solutions for handling large file transfers typically require data packet caching within the BMC's memory. When a file is split into multiple 256-byte packets, if the BMC's SRAM cannot provide sufficient buffer space for packet caching, data must be written to flash memory as it is received. This approach is highly susceptible to leaving fragmented, incomplete data in flash memory during transmission interruptions or power outages, leading to file system corruption. This application addresses this issue by using a packet-based streaming appending step, directly appending data packets to the real-time write block of the SPI NOR Flash without BMC memory caching, and marking the data as PENDING. This design avoids reliance on the limited SRAM of the BMC, fundamentally solving the packet caching problem caused by limited memory resources, and effectively preventing invalid data from polluting the file system due to transmission interruptions.

[0045] Furthermore, traditional solutions may lack an effective mechanism to ensure transaction integrity after data is written to flash memory. If transmission is interrupted, partially written data may be mistakenly considered complete data, leading to logical errors. This application introduces a transaction control commit step, writing a commit control frame and marking it as valid only after confirming receipt of the last data packet and passing global verification. This mechanism uses the commit control frame as the atomic boundary of the transaction, ensuring that only complete and correctly verified data is recognized as valid by the system. This effectively prevents malformed data residue caused by transmission interruptions and significantly improves the reliability of data storage.

[0046] Furthermore, traditional file systems or flash translation layers (FTLs) require significant memory and CPU resources to maintain complex physical-logical address mapping tables, which is unsustainable for the BMC firmware layer with its limited SRAM capacity. This application addresses this by maintaining a flattened index table in SRAM and using low-level atomic operations to switch the file number pointer to the physical offset address of the commit control frame, achieving extremely low memory overhead and efficient address mapping. This flattened index structure and atomic pointer switching mechanism not only solve the problem of limited BMC memory resources but also ensure the real-time performance and consistency of file state transitions, avoiding the IPMI interface timeout risk that may result from complex locking mechanisms or lengthy index update operations.

[0047] In summary, this application constructs a lightweight, highly reliable, and low-overhead data storage and access method through the synergistic effect of three core steps: packet-based streaming appending, transaction-controlled commit, and index mapping update. This method effectively solves key technical problems such as data integrity, power-loss safety, and performance efficiency in large file transfers under specific application scenarios where BMC memory resources are extremely limited, representing a significant advancement over existing technologies.

[0048] In some embodiments, if the server system experiences a sudden power outage or forced hard reboot during the aforementioned packet transmission, the flattened index table, existing only in volatile static random access memory (SRAM), will be lost upon power failure. Consequently, some malformed and incomplete data will remain in the flash memory. Without a global index backup, the system recovery process requires traversing the entire flash memory space, which is not only time-consuming but may also lead to data recovery failure or integrity issues due to the inability to accurately distinguish between temporary and valid data. This embodiment achieves millisecond-level power-on self-healing recovery through a two-level retrieval mechanism, specifically by setting snapshot control frames (i.e., checkpoint control frames).

[0049] Figure 4This is a schematic diagram of a flattened memory index and a fast power-down recovery mechanism. For ease of description, the tail region of SPI NOR Flash can contain the following types of data frames: Checkpoint control frame: This frame is a "snapshot" of the memory flattened index table at a specific point in time.

[0050] COMMIT completion control frame: This frame represents the integrity of a transaction and serves as verification proof that file data has been physically written to disk. Only when this frame exists and the CRC check passes can the corresponding packet data be upgraded from PENDING to a readable state. It is the legal boundary for the atomicity of a transaction.

[0051] PENDING orphan data frames: These frames refer to data packets that have been written to the SPI NOR Flash but lack a matching COMMIT frame. They embody the "power-down tear" phenomenon caused by IPMI transmission interruptions and are "incomplete transactions" that need to be logically filtered during power-down recovery.

[0052] The steps for saving a checkpoint frame include the following: A. When the space of the current real-time write block of the SPI NOR Flash is full, or when an explicit synchronization command is received from the host, the BMC packages the current flat index table as a single Checkpoint control frame. This Checkpoint control frame may include information such as file number, physical offset address, and file size, and is serialized according to a predefined format to form a complete data structure. It is a self-contained, parsable unit that represents the global state of the file system at a certain moment. B. Write the Checkpoint control frame to the end of the dedicated reserved area of ​​the SPI NOR Flash or the current real-time write block.

[0053] The Checkpoint control frame is designed to persistently store the flattened index table maintained in volatile SRAM to handle scenarios such as abnormal system power loss or the need for state synchronization. It provides a non-volatile backup of the index data in memory, ensuring that the system can quickly rebuild the file mapping relationship during recovery. A snapshot is triggered when the space of the current real-time write block of the SPI NOR Flash is full. When the available space of the current real-time write block of the SPI NOR Flash reaches a preset low threshold or is completely exhausted, it indicates that the write block will soon be unable to continue carrying new data. At this time, to ensure the continuity of data writing and subsequent garbage collection operations, the current flattened index table needs to be backed up so that the valid data state of the current block is recorded before switching to a new write block.

[0054] Another implementation is to trigger snapshot saving when the effective data volume of the real-time write block reaches a certain percentage (e.g., 80% or 90%), reserving sufficient space for subsequent operations. Receiving an explicit synchronization command from the host is another condition for triggering snapshot saving. The host can proactively send a specific command to the Baseboard Management Controller (BMC) based on its own business logic or data consistency requirements, instructing the BMC to immediately perform a persistent backup of the current flattened index table. For example, this command can be sent when the host system is undergoing critical configuration updates or is about to be shut down to ensure that the latest file index state is saved. In addition, the index synchronization and saving can also be triggered periodically or automatically when specific system events (such as before firmware upgrades) occur through the BMC's internal timer or event triggering mechanism.

[0055] C. After a system abnormal reset or power-on, reverse scan the SPI NOR Flash to find the latest written snapshot control frame, and load the snapshot control frame into SRAM to restore the underlying file mapping index. After a system reset or power-on failure, the BMC can read and parse data in reverse order, starting from the end of the physical address of the SPI NOR Flash, using preset block sizes or frame structure characteristics (e.g., specific frame header identifiers, checksums, or length information), until a valid frame conforming to the Checkpoint control frame format is identified. Alternatively, the BMC firmware can maintain a list of physical addresses for Checkpoint control frames during normal operation. During recovery, the address of the latest snapshot is directly retrieved from this list and reverse-verified to ensure its integrity. The Checkpoint control frame is then loaded into SRAM to restore the underlying file mapping index. This step aims to load the most recent complete file system state into the BMC's memory as a baseline for subsequent data recovery.

[0056] D. Starting from the physical address recorded in the snapshot control frame, scan the remaining data frames sequentially. If a data packet with a physical state of temporary state is scanned, and no matching commit control frame is scanned in the subsequent physical interval, it is determined to be an incomplete transaction. When reconstructing the memory index, the temporary data packet is ignored to achieve transaction-level power-loss safe rollback.

[0057] The BMC reads the physical address of the last valid write to the SPI NOR Flash when the snapshot was generated, recorded in the snapshot control frame. Starting from this address, the BMC sequentially reads data from the SPI NOR Flash via the SPI bus interface, in units of data packets or COMMIT control frames.

[0058] During sequential scanning, when the BMC reads a data packet and finds its physical status marked as PENDING, it records the physical address and file number of the data packet. Subsequently, the BMC continues scanning a physical interval following the data packet (e.g., until the next block boundary, the next Checkpoint control frame, or the preset maximum transaction length) to check for the existence of a COMMIT control frame corresponding to that file number. If no matching COMMIT control frame is found within the defined scan range, it indicates that the transaction to which the "PENDING" data packet belongs has not been fully committed and should be considered an incomplete transaction (i.e., a PENDING orphan data frame). Furthermore, after loading the Checkpoint control frame into SRAM and performing subsequent scans, the BMC will not include the file numbers and physical addresses corresponding to these incomplete transactions in the index when building or updating the flattened index table in SRAM. In other words, these incomplete data packets will not be recognized as valid data by the file system.

[0059] exist Figure 4 In the illustrated diagram, after the BMC experiences an abnormal power outage and reboot, the firmware first performs a power-on self-test, then scans the Flash reserved area in reverse to find the latest snapshot checkpoint control frame. The firmware directly loads the entire index data from this snapshot into memory, corresponding to step 4 on the right (loading the checkpoint control frame), instantly restoring over 95% of the historical basic file mapping index within 1 millisecond. Subsequently, the firmware uses the cutoff address of this snapshot record as the starting point and performs a reverse tracing scan backward. When it encounters an incomplete PENDING orphan data frame 2, since its status is PENDING and no matching COMMIT control frame can be found in the subsequent physical range, the firmware performs step 5 sequential scan (filtering PENDING orphan frames and performing power-down transaction rollback), directly pruning and ignoring it during the reconstruction of the memory index, logically achieving a natural transaction-level power-down safe rollback. As the scan line continues to move forward, it scans the last data packet corresponding to the transaction (i.e., COMMIT completion data frame 3) and retrieves the subsequent COMMIT control frame. This proves that the file was completely written to disk before the power failure. The firmware then executes step 6 (parses the COMMIT control frame and updates the File_ID physical offset pointer), updates the corresponding file pointer in the memory index table to the address of the COMMIT frame, and completes the millisecond-level power-on state self-recovery.

[0060] Through the above series of steps, this solution ensures that the file system can automatically and safely recover to the most recent fully committed state after an abnormal power outage. All transactions that were not fully committed before the power outage (i.e., those with only "PENDING" data packets but no corresponding "COMMIT" control frames) will be automatically identified and ignored by the system and will not be included in the file system. This effectively avoids loading incomplete or corrupted data into memory, thus guaranteeing data consistency and integrity of the file system after restart.

[0061] In SPI NOR Flash storage media, due to the physical characteristics of SPI NOR Flash being erased by sector and having extremely high erase latency (up to hundreds of milliseconds), when there is insufficient available space and garbage collection is required, if the traditional synchronous erase and data migration method is used, the high-priority IPMI read and write requests in the foreground will be forced to wait for the Flash hardware to unload the busy state, which will then cause the interface protocol to time out and cause the system communication to hang.

[0062] Therefore, this application provides a lock-free asynchronous garbage collection scheme with decoupled front-end and back-end, such as... Figure 5 The diagram illustrates a lock-free asynchronous garbage collection (GC) mechanism with decoupled front-end and back-end, including the following steps: A. When the available space of SPI NOR Flash is lower than the set threshold, the background garbage collection thread allocates new physical blocks that have been erased in advance; This solution does not use traditional mutex locks for synchronization between foreground and background tasks; instead, it executes asynchronously. This avoids blocking high-priority foreground processes during garbage collection, thereby improving the system's real-time responsiveness and concurrency. The threshold can be a fixed value, such as remaining space less than 10% of the total capacity, or a dynamically adjusted value. Alternatively, garbage collection can be triggered when a write operation fails to allocate a new block, or after a write operation completes, by checking if the available space is below the threshold.

[0063] The background garbage collection thread allocates new physical blocks that have been pre-erased. Its purpose is to prepare new storage areas for receiving valid data before the background thread performs data migration. The background thread can maintain a list of free physical blocks. When a new physical block is needed, it retrieves an erased block from this list. If there are insufficient free blocks, the background thread can pre-erase some physical blocks and add them to the free list when the system load is low. Alternatively, when garbage collection starts, the background thread can immediately issue an erase command to the underlying Flash controller to erase one or more physical blocks and mark them as available for subsequent data copying.

[0064] B. Sequentially scan the old physical blocks to be reclaimed, identify valid data frames and asynchronously roll them to the new physical blocks. During this physical copying, the pointers in the flattened index table remain unchanged. A background thread iterates through the physical blocks containing old data, filtering out still valid data—that is, data that has not been logically deleted or overwritten—and copies it to a pre-prepared new physical block. The background thread can determine whether data frames in the old physical block are still valid based on records in the flattened index table. For valid data frames, data is read from the old physical block via the SPI interface and then written to the new physical block. This process can be performed in batches, copying a small amount of data each time to avoid prolonged occupation of the Flash bus.

[0065] During this physical copying process, the pointers in the flattened index table remain unchanged, which is crucial for achieving lock-free concurrent access. While data is copied in the background, the foreground still accesses data through the old index pointers, ensuring the continuity and correctness of foreground read operations and avoiding access interruptions or errors caused by data migration. The flattened index table is maintained in SRAM, and its pointers point to the data in the old physical blocks. When copying data in the background, only a temporary mapping of the new physical blocks is maintained within the background thread; the flattened index table in SRAM is not immediately updated.

[0066] by Figure 5 For example, when the total flash memory space reaches the erase / write threshold, the background GC thread starts and allocates a completely clean new physical block Active_Block3 in the Flash. The GC thread begins to sequentially scan the old physical block Old_Block1 to be reclaimed, directly ignoring the invalid data (INVALID) in it, and asynchronously rolling and copying the only valid data frame 2 to the free position of the new physical block 3.

[0067] C. If a high-priority IPMI read request is received in the foreground during physical replication, the foreground read thread does not wait to acquire the mutex lock, but directly reads the required data frame from the old physical block based on the existing pointers in the flattened index table.

[0068] This step directly embodies the lock-free design, ensuring real-time response to high-priority foreground requests. Foreground threads do not need to wait for background garbage collection to complete or locks to be released; they can directly access data based on the current index information, thus avoiding IPMI protocol timeouts. When the foreground read thread receives an IPMI read request, it directly queries the flattened index table in SRAM to obtain the physical offset address corresponding to the file number, and then reads data from the old physical block of the SPI NOR Flash via the SPI interface. Since the pointers in the flattened index table remain unchanged during copying, the read operation can proceed normally. In the firmware design, the foreground read path and the background garbage collection path can be completely independent, without sharing any resources requiring mutual exclusion protection. The foreground read operation relies only on the flattened index table in SRAM and read operations on the Flash hardware, while the background copy operation involves Flash read and write operations. Through reasonable scheduling and hardware arbitration, concurrent execution of both can be ensured.

[0069] In some embodiments, in the lock-free asynchronous garbage collection step with front-end and back-end decoupling, after all valid data frames are copied to the new physical block, the background garbage collection thread issues a single atomic instruction to update the corresponding pointer in the flattened index table to the physical offset address pointing to the new physical block; and after determining that all valid file pointers in the old physical block have been safely switched, it issues an asynchronous erase instruction to the underlying controller to erase the old physical block.

[0070] For example, when the background garbage collection thread starts, suppose the old physical block A contains valid data frames for files F1 and F2, while the data for file F3 is invalid. The background thread first allocates a brand new, erased physical block B. Then, the background thread asynchronously copies the valid data frames for files F1 and F2 from physical block A to physical block B. Once all valid data frames for files F1 and F2 have been successfully copied to physical block B, the background garbage collection thread performs an atomic operation to update the flattened index table. For example, for file F1, the atomic pointer in the flattened index table that originally pointed to the starting address of F1 data in physical block A can be instantly updated to point to the new starting address of F1 data in physical block B. Similarly, the pointer for file F2 is also updated atomically. During this process, even if the foreground thread attempts to read files F1 or F2, it can always obtain the correct physical address through the flattened index table—either an address in the old physical block A (before the atomic update) or an address in the new physical block B (after the atomic update), thus ensuring data access consistency. After all valid file pointers pointing to physical block A (i.e., pointers F1 and F2) have been confirmed to have safely switched to pointing to physical block B, the background garbage collection thread sends an asynchronous erase command to the underlying controller of the SPI NOR Flash, specifying the erasure of physical block A. This erase operation is performed independently in the background, without affecting foreground access to files F1 and F2 (which are now read from physical block B), and without blocking the processing of other IPMI commands.

[0071] by Figure 5 For example, during the extremely time-consuming physical relocation and sector erasure cycle performed by the background GC thread, the foreground concurrent read thread 5 suddenly initiates a high-priority IPMI read request. At this time, the foreground concurrent read thread 5 does not need to wait for any mutex locks, nor does it experience any blocking. Instead, it directly follows the existing, intact pointer in memory index table 4 to access the valid data frame 2 within the old physical block 1. The entire read response time is less than 1 millisecond, completely eliminating the risk of out-of-band protocol timeouts. After the new block 3 has received all the valid migrated data, the GC thread issues a single atomic instruction to memory index table 4 in the background to complete the pointer switch, and the original old physical block 1 is asynchronously erased in the background.

[0072] Through the above technical solution, this application effectively resolves the conflict between garbage collection operations and high-priority IPMI requests in SPI NOR Flash storage media. In environments with limited BMC resources, it significantly improves the reliability and efficiency of the flash memory garbage collection process.

[0073] Traditional lock-free asynchronous garbage collection steps, which decouple the front-end and back-end, are used to reclaim flash memory space without blocking front-end read and write requests. However, in its implementation, if the back-end garbage collection thread is performing physical copying and the front-end happens to receive a write request for the same file, the competition between the front-end and back-end for the same physical address or logical mapping may lead to data consistency conflicts or confusion in pointer update logic, thereby causing uncertainty in the file system state.

[0074] To this end, this application further proposes a front-end write conflict redirection and lock-free self-healing mechanism for asynchronous garbage collection steps.

[0075] "Foreground write conflict redirection" refers to a mechanism where, when a foreground operation (such as a write request initiated by the host) and a background operation (such as data copying by a garbage collection thread) simultaneously compete for the same data resource (same file number), the foreground operation is not blocked. Instead, the new write data is redirected to a different, independent storage area for processing. The purpose is to avoid delays or timeouts caused by foreground operations waiting for background operations to complete, ensuring high system responsiveness. This can be achieved by reserving one or more independent real-time write blocks for foreground write operations in the BMC firmware logic. When a conflict is detected, the foreground thread writes data to these reserved blocks.

[0076] "Lock-free self-healing steps" refer to a system that, when conflicts occur between the front-end and back-end, does not need to coordinate access through traditional mutex lock mechanisms. Instead, it automatically detects, processes, and resolves conflicts in an intelligent, non-blocking manner (such as atomic operations and state comparisons), thereby restoring the system to a consistent and correct state. Its core principle is to avoid the performance overhead and deadlock risk associated with locks, while ensuring data consistency.

[0077] This mechanism is implemented through the following scheme: A. If a write request for the same file number is suddenly received in the foreground while the background garbage collection thread is copying a valid data frame with the same file number, the foreground thread will not block and wait, but will directly redirect the newly received data packet and append it to the end of the current foreground real-time write block. If the foreground suddenly receives a write request for the same file number, the system will not block the foreground thread while waiting for the background operation to complete. Instead, the foreground thread will immediately redirect the newly received data packet and append it to the end of the current foreground real-time write block. This mechanism ensures the immediate responsiveness of foreground write requests, avoids the risk of IPMI interface timeouts due to waiting, and thus maintains the high availability of the system.

[0078] B. After the commit control frame corresponding to the newly received data packet is written, the foreground thread preemptively rewrites the atomic pointers in the flattened index table. Once the newly received data packet is written and its corresponding COMMIT control frame is successfully written to disk, the foreground thread immediately executes a preemptive operation, rewriting the atomic pointer of the corresponding file number in the flattened index table using atomic instructions. Because this operation is atomic, it guarantees the uninterrupted nature of the pointer update, ensuring that the flattened index table always points to the latest and most valid data version. This preemptive update mechanism allows data written by the foreground thread to quickly become visible to the system and become the latest valid version of the file.

[0079] C. Before the background garbage collection thread completes the physical copy and prepares to rewrite the flattened index table, it compares the address that the current flattened index table actually points to with the physical address of the old data source. If the two are found to be inconsistent, the conflict comparison inconsistency mechanism is triggered, the pointer switching operation for the same file number is actively abandoned, and the copy to the new physical block is marked as invalid.

[0080] After completing its physical copying task, the background garbage collection thread does not immediately update the flattened index table. Before performing a pointer switch, it performs a crucial comparison: checking if the address currently pointed to by the flattened index table is still the old data source physical address when it initially started copying. If the comparison shows a discrepancy, it means that the foreground thread has preemptively updated the file's index, pointing to the newer data version. In this case, the background thread triggers a conflict resolution mechanism, proactively abandoning the pointer switch operation for that file number and marking its painstakingly copied copy to the new physical block as invalid.

[0081] Through this ingenious design, this solution achieves seamless coordination of concurrent writes and garbage collection of the same file between the foreground and background without the need for traditional mutex locks. Foreground write conflict redirection ensures low latency, while the background self-healing mechanism ensures strong consistency of the flattened index table by intelligently comparing and discarding outdated operations, preventing data loss or file system corruption caused by background operations overwriting the latest data in the foreground. This collaborative approach enables the system to efficiently and stably handle high-concurrency read / write and garbage collection tasks in resource-constrained BMC environments, significantly improving the robustness and reliability of the file system.

[0082] Figure 6A schematic diagram of the foreground write preemption conflict redirection mechanism is shown. The lock-free resolution mechanism of write preemption conflict is further illustrated through the specific industrial application scenario of "dynamic data exchange between the high-frequency memory testing tool of the upper layer of the operating system and the underlying firmware". The specific scenario is: the memory testing tool on the host side is rewriting the memory topology error log file F located in the BMC storage. At this time, the background GC thread is just in the middle of rolling the old version of file F (2) from the old physical block 1 to the newly allocated Active_Block3.

[0083] Combination Figure 5 The steps for writing a lock-free conflict resolution code are as follows: 1. Write Redirection: During a break in GC migration, the foreground suddenly receives a high-priority write request from a memory testing tool. The foreground write thread does not compete with the background GC thread for any global mutex lock, nor does it wait for GC to finish. Instead, it directly redirects and appends the latest version of the data stream to the end of the foreground's dedicated current write block Current_Write_Block4, and successfully writes it to disk as the latest version of the file F+COMMIT control frame 5.

[0084] 2. Index preemption: The moment the latest version of data 5 is successfully appended to disk, the foreground write thread preemptively rewrites the memory index table 6, instantly updating the atomic mapping pointer of file F to point to the latest physical address 5.

[0085] 3. Conflict comparison and active abandonment: At this time, the background GC thread finally completed the physical copy of the old data and was ready to modify the memory index table 6. Before rewriting, the GC thread performed an atomic comparison operation: comparing the "physical address of the old data source in its own hands (2)" with the "latest visible address actually pointed to by the current memory index table 6 (5)".

[0086] 4. Lock-free self-healing: Since the foreground write thread has just completed its preemptive modification, the GC thread immediately detects that the versions are inconsistent (the source address has been shuffled). To prevent the old data from overwriting the latest data just written by the foreground thread, the GC thread triggers the conflict comparison inconsistency mechanism, actively aborting the mapping rewrite operation for file F (as shown in label 7), and directly marking the old copy copied to new block 3 as invalid.

[0087] This solution effectively addresses the data consistency conflicts and file system state uncertainty that can arise when handling concurrent writes to the same file in a decoupled, lock-free asynchronous garbage collection process. Without introducing traditional mutex locks, it achieves seamless coordination between front-end and back-end concurrent writes and garbage collection of the same file through a sophisticated write conflict redirection and lock-free self-healing mechanism. This ensures efficient, stable, and strong consistency of the file system in resource-constrained environments.

[0088] Based on the above technical solution, the following beneficial technical effects are achieved: 1. Completely resolves power-loss tearing issues in packet transmission: The general transaction log is moved down to the 256-byte IPMI protocol frame level. Incomplete packets that are not fully transmitted remain permanently penned in Flash memory and are not recognized by memory indexes, ensuring absolute data atomicity.

[0089] 2. Achieve "zero-blocking" IPMI foreground response: Through a lock-free design of "rolling and moving + atomic memory pointer switching," Flash erase and write operations during garbage collection are completely invisible to real-time foreground read requests. Write conflicts are resolved through the logic of "foreground preemptive update, background comparison and abandonment," abandoning traditional mutex locks and eliminating the risk of IPMI response timeouts caused by bus or resource contention.

[0090] 3. Natural wear leveling and extremely low firmware overhead: The sequential append write mechanism fundamentally eliminates repeated erasure and writing (perforation effect) of the same physical sector of the Spinor Flash, which greatly extends the lifespan of the BMC storage medium; and only file-level pointers are stored in memory, with extremely low code volume and SRAM usage, making it a perfect fit for resource-constrained BMC chips.

[0091] Figure 7 This is a schematic diagram of a lightweight data storage access device provided in an embodiment of this application. Its characteristic is that it is applied in a Baseboard Management Controller (BMC), where the physical storage medium of the BMC is a Serial Peripheral Interface (SPI) non-volatile flash memory. The device includes: Packet streaming append module 710: Used to receive multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; append the multiple data packets directly to the real-time write block of SPI NOR Flash in the order of receipt without going through the BMC memory packet buffer, and mark the physical state of the written data packets in the flash memory as temporary state. Transaction control commit module 720: Used to verify the received data packets. If and only if it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, a commit control frame is written at the current append position of the real-time write block, and the physical state of the commit control frame in the flash memory is marked as valid. Index mapping update module 730: It is used to maintain a flat index table in the static random access memory (SRAM) of the BMC. The flat index table records the mapping relationship between file number and the latest valid physical offset address. After the control frame is successfully written to disk, the pointer of the corresponding file number in the flat index table is switched to the physical offset address of the control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

[0092] Furthermore, this application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the lightweight data storage and access method described above.

[0093] This application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, which, when executed by a processor, implements the methods described in the above-described method embodiments. Those skilled in the art will understand that implementing all or part of the processes in the methods described in the above-described embodiments of this application can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0094] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of the present invention.

Claims

1. A lightweight data storage and access method, characterized in that, The method, applied in a Baseboard Management Controller (BMC), wherein the physical storage medium of the BMC is a Serial Peripheral Interface (SPI) NOR Flash, includes: The system receives multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; without going through the memory packet buffer of the BMC, the multiple data packets are directly appended to the real-time write block of the SPI NOR Flash in the order of receipt, and the physical state of the written data packets in the flash memory is marked as temporary. Verify the received data packets. If and only if it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, write a commit control frame at the current append position of the real-time write block and mark the physical state of the commit control frame in the flash memory as valid. A flattened index table is maintained in the static random access memory (SRAM) of the BMC. The flattened index table records the mapping relationship between file number and the latest valid physical offset address. After the commit control frame is successfully written to disk, the pointer corresponding to the file number in the flattened index table is switched to the physical offset address of the commit control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

2. The method according to claim 1, characterized in that, Maintaining a flattened index table in the SRAM of the BMC includes: A fixed-length flat hash table or dynamic array is maintained in the SRAM; In the flattened index table, the key is the file number, and the value is an atomic pointer that points to a lightweight structure. The structure records at least the physical offset address of the data corresponding to the file number and the file size.

3. The method according to claim 1, characterized in that, The payload header of the data packet encapsulates transaction control information, which includes at least one of the following: a globally unique transaction serial number, the total number of packets, the current packet sequence number, and the file number. The global verification passing means that: the current packet sequence number of the currently received data packet is equal to the total number of packets, and the cyclic redundancy check (CRC) code of the entire batch of data packets is correctly matched; the global unique transaction serial number is recorded in the submission control frame.

4. The method according to claim 1, characterized in that, The method also includes a snapshot control frame saving step: When the space of the current real-time write block of the SPI NOR Flash is full, or when an explicit synchronization command is received from the host, the BMC packages the current flat index table as a single snapshot control frame. Write the snapshot control frame to the end of the dedicated reserved area of ​​the SPI NOR Flash or the current real-time write block.

5. The method according to claim 4, characterized in that, The method further includes a rapid recovery step for abnormal power loss based on the snapshot control frame, including: After a system abnormal reset or power-on, the SPI NOR Flash is scanned in reverse to find the latest written snapshot control frame, and the snapshot control frame is loaded into the SRAM to restore the base file mapping index. Starting from the cutoff physical address recorded in the snapshot control frame, the remaining data frames are scanned sequentially backward. If a data packet with a physical state identifier of temporary state is scanned, and no matching commit control frame is scanned in the subsequent physical interval, it is determined to be an incomplete transaction. The temporary state data packet is ignored when reconstructing the memory index, thereby achieving transaction-level power-loss safe rollback.

6. The method according to claim 1, characterized in that, The method also includes a lock-free asynchronous garbage collection step that decouples the front-end and back-end, including: When the available space of the SPI NOR Flash is lower than a set threshold, the background garbage collection thread allocates new physical blocks that have been erased in advance; The old physical blocks to be reclaimed are scanned sequentially, valid data frames are identified and asynchronously rolled and copied into the new physical blocks, and the pointers in the flattened index table remain unchanged during this physical copying. If a high-priority IPMI read request is received in the foreground during the physical replication, the foreground read thread does not wait to acquire the mutex lock, but directly reads the required data frame from the old physical block based on the existing pointers in the flattened index table.

7. The method according to claim 6, characterized in that, In the aforementioned lock-free asynchronous garbage collection step with decoupled front-end and back-end: After all the valid data frames are copied to the new physical block, the background garbage collection thread issues a single atomic instruction to update the corresponding pointer in the flattened index table to the physical offset address pointing to the new physical block; After confirming that all valid file pointers in the old physical block have been safely switched, an asynchronous erase command is sent to the underlying controller to erase the old physical block.

8. The method according to claim 6, characterized in that, The method also includes foreground write conflict redirection and lock-free self-healing steps for the asynchronous garbage collection step, including: If, during the background garbage collection thread's copying of valid data frames with the same file number, the foreground thread suddenly receives a write request for the same file number, the foreground thread will not block and wait, but will directly redirect and append the newly received data packet to the end of the current foreground real-time write block. After the commit control frame corresponding to the newly received data packet is written, the foreground thread preemptively rewrites the atomic pointers in the flattened index table. Before completing the physical copy and preparing to rewrite the flattened index table, the background garbage collection thread compares the address actually pointed to by the current flattened index table with the physical address of the old data source. If the two are found to be inconsistent, a conflict comparison inconsistency mechanism is triggered, the pointer switching operation for the same file number is actively abandoned, and the copy to the new physical block is marked as invalid.

9. A lightweight data storage and access device, characterized in that, The device is used in a baseboard management controller (BMC), wherein the physical storage medium of the BMC is a serial peripheral interface non-volatile flash memory (SPI NOR Flash), and the device includes: Packet streaming append module: used to receive multiple data packets belonging to the same file number sent by the host based on the Intelligent Platform Management Interface (IPMI) protocol; to directly append the multiple data packets into the real-time write block of the SPI NOR Flash in the order of receipt without going through the memory packet buffer of the BMC; and to mark the physical state of the written data packets in the flash memory as temporary state. Transaction control commit module: used to verify the received data packets. When it is confirmed that the last data packet corresponding to the file number has been received and the global verification is passed, a commit control frame is written at the current append position of the real-time write block, and the physical state of the commit control frame in the flash memory is marked as valid. Index mapping update module: used to maintain a flattened index table in the static random access memory (SRAM) of the BMC. The flattened index table records the mapping relationship between file number and the latest valid physical offset address. After the commit control frame is successfully written to disk, the pointer corresponding to the file number in the flattened index table is switched to the physical offset address of the commit control frame through low-level atomic operations, so that the data corresponding to the file number enters a readable state that is visible to the foreground.

10. A server, characterized in that, It includes a host unit, a baseboard management controller (BMC), and a physical storage medium, SPI NOR Flash, which is connected to the BMC via an SPI bus. The BMC integrates a static random access memory (SRAM). The host is used to split variable-length files belonging to the same file number into multiple data packets and send them to the BMC based on the Intelligent Platform Management Interface (IPMI) protocol. The BMC is configured with a processor and a firmware memory. The firmware memory stores a computer program, which, when executed by the processor, implements the lightweight data storage and access method as described in any one of claims 1 to 8.

Images