CMB Doorbell Synchronization for Reliable NVMe SQ Fetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The race condition between write commands and doorbell operations in the controller memory buffer (CMB) leads to the potential retrieval of incorrect data due to the slow access time of CMB in DRAM, which is not addressed by the Relaxed Ordering bit in the NVMe standard.
Innovation Solution
Implementing a second, invisible doorbell within the controller to ensure data is fully written to the CMB before retrieval, using a double doorbell mechanism to confirm data availability before fetching commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the host device writes commands to CMB and rings doorbell, then command notification is sent to controller, but data may not be fully written to CMB before controller reads it due to slow DRAM access and separate paths
Solution Approach 1:
The controller performs preliminary actions by checking the CMB doorbell count before fetching commands. This ensures that the controller waits until the host has fully written commands to CMB before initiating read operations, preventing race conditions without significantly increasing latency
Solution Approach 2:
The system uses feedback from the CMB doorbell count to control the command fetching process. The controller continuously monitors the doorbell count and only fetches commands when the count indicates data is ready, ensuring data integrity through feedback-based synchronization
2Device complexity
If the controller uses a single doorbell mechanism, then device complexity is reduced, but race conditions occur between write commands and doorbell operations
Solution Approach 1:
The doorbell mechanism is segmented into two separate counters: CMB doorbell count for tracking command writes and SQ doorbell count for tracking notification sends. This segmentation allows independent tracking of each operation, enabling the controller to synchronize properly without complex locking mechanisms
3Productivity
If the controller fetches commands immediately after doorbell ringing, then productivity is improved, but outdated data may be retrieved due to race conditions
Solution Approach 1:
The controller performs preliminary verification by checking the CMB doorbell count before fetching commands. This preliminary action ensures data is fully written to CMB before retrieval, preventing outdated data from being processed while maintaining efficient command processing
Data Source
AI summary
Controller memory buffer (CMB) usage is on the rise. Host devices will oftentimes place a submission queue (SQ) in CMB to take advantage of data storage device provided memory. The host device will place an entry in the SQ and ring the doorbell associated with the SQ. The controller, upon the doorbell ringing, will check the SQ. Even though the doorbell was rung, the data may not be in the SQ due to the fact that accessing the CMB is slow. To ensure the data is in the SQ, the controller can utilize two doorbells, the one that the host device rings, and a second doorbell invisible to the host device. The second doorbell is rung once the data is in the SQ. The controller then will retrieve data from the SQ only if both doorbells have been rung.