Multi-controller NVMe SSD service quality monitoring system and method based on global heartbeat synchronization
Through the global heartbeat synchronization and sliding window algorithm, the performance bottleneck analysis error caused by resource competition in multi-controller NVMe SSD is solved, and the time consistency and accurate performance bottleneck analysis of QoS data across virtual machines is realized, improving the accuracy and efficiency of QoS monitoring.
Patent Information
- Application Number
- CN202510491720.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-07-25
AI Technical Summary
Under the SRIOV architecture, the service quality monitoring of multi-controller NVMe SSDs has performance bottleneck analysis errors caused by resource competition, and the cross-VM QoS data lacks a global view, and the clock deviation affects the aggregation accuracy of QoS indicators.
The global heartbeat synchronization mechanism and sliding window algorithm are adopted to achieve time synchronization and data alignment through the local service quality monitoring module, the global service quality monitoring module, the data frame alignment module and the resource competition analysis module, and the performance bottleneck is determined.
Ensure the time consistency of QoS data across virtual machines, the error control is in the microsecond level, accurately analyze performance bottlenecks, optimize resource allocation, and improve QoS monitoring accuracy.
Smart Images
Figure CN120371645A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of solid-state storage devices, and specifically to a multi-controller NVMe SSD quality of service monitoring system and method based on global heartbeat synchronization. Background Art
[0002] An NVMe SSD is a solid-state drive (Solid State Disk) that complies with the Non-Volatile Memory Host Controller Interface Specification (Non-Volatile Memory express). There may be multiple controllers inside an SSD, but the simultaneous operation of multiple controllers may cause resource competition and affect the quality of service (QoS). For example, uneven loads on different controllers may lead to increased request latency on some controllers, or improper priority handling, affecting the bandwidth on some controllers. Therefore, monitoring QoS is the basis for high-quality QoS and is very important for optimizing resource allocation and ensuring QoS among multiple controllers.
[0003] In the current SRIOV architecture, there are multiple controllers in a single physical NVMe SSD, and each controller corresponds to a virtual function (VF). Each VF is managed and used by an independent virtual machine at the host side, resulting in QoS data being scattered among multiple virtual machines and lacking a global resource view. Moreover, the hardware resource isolation feature of SRIOV makes it difficult to analyze the QoS correlation across VFs. It is necessary to record the QoS metrics of different VFs in multiple virtual machines and then aggregate the QoS metrics of multiple controllers. It is very sensitive to the time parameters of each QoS metric. If there is a deviation in the timestamps of the QoS metrics recorded by each controller, there will be errors in the overall QoS performance bottleneck analysis; when there is competition for flash bandwidth and PCIe bandwidth among multiple controllers, the QoS data at this time cannot effectively analyze where the performance bottleneck is. Summary of the Invention
[0004] Aiming at the defects of the prior art, the present invention discloses a multi-controller NVMe SSD quality of service monitoring system and method based on global heartbeat synchronization. By means of a global heartbeat synchronization mechanism and a dynamic sliding window algorithm, a resource competition intensity quantification model is established, which is especially suitable for the global performance analysis of multi-controller resource competition in a virtualized scenario under the SR-IOV architecture.
[0005] To solve the above technical problems, the technical solution adopted by the present invention is: A multi-controller NVMe SSD quality of service monitoring system based on global heartbeat synchronization, including a local quality of service monitoring module, a global quality of service monitoring module, a data frame alignment module, and a resource competition analysis module; Each NVMe SSD controller embeds a local quality of service monitoring module, which is used to collect the quality of service monitoring data of each NVMe SSD controller; The global quality of service monitoring module is used to collect the global resource occupancy status of the SSD in real time, generate a global heartbeat signal, and broadcast the global heartbeat signal to the local quality of service monitoring module regularly to achieve time synchronization; The data frame alignment module is used to align the quality of service data of multiple controllers along the global time axis; The resource competition analysis module is used to analyze the competition intensity of each controller for shared resources and determine the level of the performance bottleneck.
[0006] Furthermore, the global heartbeat signal includes a global heartbeat count value and a global heartbeat timestamp. The global quality of service monitoring module uses a hardware timer to generate the global heartbeat signal periodically, where the global heartbeat timestamp generation formula is: where , where represents the k-th global heartbeat timestamp, represents the initial time reference, is the clock period.
[0007] Furthermore, each local quality of service monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global quality of service monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ; where represents the local time calibrated when the i-th controller receives the k-th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time.
[0008] Furthermore, the data frame alignment module uses a sliding window algorithm to align the quality of service data of each controller. The alignment process is as follows: Let the time window length be W, and the data frame set of N controllers is included in the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowed time deviation, it is considered that the alignment is successful; If the alignment is not successful, the window boundary is adjusted through the weight function . The weight function calculation formula is: , where \(t\) represents the timestamp of the current data frame, is the center time of the current window, is the adjustment rate; The step size for sliding the window to the right The calculation formula is: , where represents the weight of the timestamp of the \(i\)-th data frame; Slide the window to the right according to the sliding step size .
[0009] Furthermore, the process of the resource competition module analyzing the competition intensity of each controller for shared resources is as follows: First, define variables. Let the controller set be , \(N\) represents the total number of controllers; the shared resource set is , \(M\) represents the total number of shared resources, represents the controller at time \(t\) for the shared resource utilization rate; Calculate the competition intensity of the shared resource within the time window : , where \(y\) represents the sensitivity of controller latency to resource utilization; If , and the latency of at least one controller , then judge as a resource bottleneck; where represents the allowed minimum competition intensity, represents the allowed minimum controller latency.
[0010] Furthermore, the local service quality parameters collected by the local service quality monitoring module include I / O request latency, iops, the number of flash media accesses, and the single command size distribution.
[0011] Furthermore, the SSD global resource occupancy status collected by the global service quality monitoring module includes the bandwidth allocation of the flash media parallel channels and the PCIe bus transport layer throughput.
[0012] Furthermore, the clock frequency compensation coefficient is obtained by calculating the clock deviation of the adjacent 10 heartbeats through linear regression. Let the local count of the \(i\)-th local heartbeat counter be , , the frequency compensation coefficient is .
[0013] Furthermore, the initial time reference is synchronized with the host time.
[0014] The present invention also discloses a method for monitoring the quality of service of a multi - controller NVMe SSD based on global heartbeat synchronization. This method is implemented based on the above - mentioned monitoring system and includes three parts: global heartbeat synchronization, data frame time alignment, and resource competition analysis; The implementation process of global heartbeat synchronization is as follows: The global quality - of - service monitoring module uses a hardware timer to generate a global heartbeat signal at a period , and the global heartbeat timestamp generation formula is: , where represents the k - th global heartbeat timestamp, represents the initial time reference, is the clock cycle; The global quality - of - service monitoring module sends the global heartbeat signal to each local quality - of - service monitoring module. Each local quality - of - service monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global quality - of - service monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ; where represents the local time generated after calibration when the i - th controller receives the k - th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time; The implementation process of data frame time alignment is as follows: Let the time window length be W, and the data frame set of N controllers is included in the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowable time deviation, it is considered that the alignment is successful; If the alignment is not successful, the window boundary is adjusted through the weight function . The calculation formula of the weight function is: , where t represents the timestamp of the current data frame, is the central time of the current window, is the adjustment rate; The right sliding step length within the window The calculation formula is as follows: , where represents the weight of the timestamp of the i-th data frame; Slide the window to the right according to the sliding step length .
[0015] The implementation process of resource competition analysis is as follows: Define variables. Let the controller set be , where N represents the total number of controllers; the shared resource set is , where M represents the total number of shared resources, represents the controller 's utilization rate of the shared resource at time t; Calculate the competition intensity of the shared resource within the time window : , where y represents the sensitivity of controller delay to resource utilization rate; If , and the delay of at least one controller , then judge as a resource bottleneck; where represents the minimum allowable competition intensity, represents the minimum allowable controller delay.
[0016] The beneficial effects of the present invention: Through the global heartbeat synchronization mechanism, ensure the time consistency of cross-virtual machine QoS data, and control the error within the microsecond level. Combine the global resource occupancy and local QoS metrics to distinguish the performance degradation caused by uneven controller load, medium access conflict or bus congestion. All related modules are implemented based on the controller firmware, making full use of the hard disk controller example and making the host computing power occupancy 0. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is the principle block diagram of the system described in Embodiment 1; Figure 2 is the flowchart of the method described in Embodiment 2. DETAILED DESCRIPTION OF THE INVENTION
[0018] The present invention will be further described below with reference to the drawings and specific embodiments.
[0019] Embodiment 1 This embodiment discloses a multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization, as Figure 1 shown, which includes a global quality - of - service monitoring module and multiple local quality - of - service monitoring modules. In each controller of the NVMe SSD, an independent local quality - of - service monitoring module is embedded. This module is used to collect service monitoring data (Qos data) of its respective controller. When the local quality - of - service monitoring module collects Qos monitoring data of the controller, for each frame of data collected, a local heartbeat count is added. Specifically, the local quality - of - service monitoring module is deployed at the firmware layer of each NVMe SSD controller to collect QoS raw data of this controller in real time, including: I / O request latency, iops, number of flash media accesses, and single - command size distribution. The global quality - of - service monitoring module is used to collect the global resource occupancy status of the SSD in real time, including the bandwidth allocation of the parallel channels of the flash media and the throughput of the PCIe bus transport layer, generate a global heartbeat signal, and broadcast it to all LQM modules to achieve time synchronization. And inside the global quality - of - service monitoring module, there are a data - frame alignment module and a resource - competition analysis module; the data - frame alignment module is used to align the quality - of - service data of multiple controllers along the global time axis; the resource - competition analysis module is used to analyze the competition intensity of each controller for shared resources and determine the level where the performance bottleneck belongs.
[0020] In this embodiment, the local quality - of - service monitoring module periodically receives the global heartbeat signal from the global quality - of - service monitoring module at a fixed period (such as 1ms) to complete global heartbeat synchronization, and periodically returns the local quality - of - service monitoring data (Qos data) it has collected to the global quality - of - service monitoring module. Each frame of QoS data is appended with the local heartbeat counter value and the global heartbeat timestamp to ensure the consistency of the cross - controller data time reference.
[0021] This embodiment eliminates the clock skew and drift across controllers based on the global heartbeat synchronization mechanism formed by the global quality - of - service monitoring module and the local quality - of - service monitoring modules. The implementation process of the global heartbeat synchronization mechanism is as follows: The global heartbeat signal includes a global heartbeat count value and a global heartbeat timestamp. The global quality - of - service monitoring module uses a hardware timer to generate the global heartbeat signal at a period The formula for generating the global heartbeat timestamp is: , where represents the k - th global heartbeat timestamp, represents the initial time reference, synchronized with the host time, is the clock period.
[0022] Each local quality of service monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global quality of service monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ; where represents the local time generated after calibration when the i-th controller receives the k-th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time.
[0023] In this embodiment, the clock frequency compensation coefficient is obtained by calculating the clock deviation of adjacent 10 heartbeats through linear regression. Let the local counts of the i-th local heartbeat counter at the (k - 1)-th and k-th heartbeats be 、 , then the frequency compensation coefficient is .
[0024] The data frame alignment module uses a sliding window algorithm to align the quality of service data of each controller and dynamically compensates for network transmission delays; when it detects that the heartbeat deviation of a certain controller exceeds the threshold, it triggers a data resynchronization mechanism.
[0025] Specifically, the alignment process is as follows: Let the time window length be W, and the data frame set of N controllers is included in the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowable time deviation, it is considered that the alignment is successful; If the alignment is not successful, the window boundary is adjusted through the weight function . The calculation formula of the weight function is: , where t represents the timestamp of the current data frame, is the center time of the current window, is the adjustment rate; The right sliding step size in the window is calculated as: , where represents the weight of the timestamp of the i-th data frame; The window is slid to the right according to the sliding step size .
[0026] In this embodiment, the resource competition module analyzes the competition intensity of each controller for shared resources, so as to quantify the competition intensity of multiple controllers for shared resources (flash channels, PCIe bandwidth). The specific implementation process is as follows: First, define variables. Let the controller set be , where N represents the total number of controllers; the shared resource set is , where M represents the total number of shared resources, represents the utilization rate of the controller for the shared resource at time t; Calculate the competition intensity of the shared resource within the time window : , where y represents the sensitivity of controller latency to resource utilization (fitted by historical data regression); If , and the latency of at least one controller , then determine that is a resource bottleneck; where represents the minimum allowable competition intensity, represents the minimum allowable controller latency.
[0027] In this embodiment, all involved modules are implemented based on controller firmware, making full use of the hard disk controller example and resulting in 0 host computing power occupancy.
[0028] Embodiment 2 The present invention also discloses a multi - controller NVMe SSD quality - of - service monitoring method based on global heartbeat synchronization. This method is implemented based on the monitoring system described in Embodiment 1. As Figure 2 shown, it includes three parts: global heartbeat synchronization, data frame time alignment, and resource competition analysis; The implementation process of global heartbeat synchronization is as follows: The global quality - of - service monitoring module uses a hardware timer to generate a global heartbeat signal at a period . The global heartbeat timestamp generation formula is: , where represents the k - th global heartbeat timestamp, represents the initial time reference, is the clock cycle; The global service quality monitoring module sends the global heartbeat signal to each local service quality monitoring module. Each local service quality monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global service quality monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ; Where represents the local time generated after calibration when the i-th controller receives the k-th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time.
[0029] The process of data frame time alignment is as follows: Let the time window length be W, and the data frame set of N controllers is included in the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowable time deviation, then it is considered that the alignment is successful; If the alignment is not successful, the window boundary is adjusted through the weight function . The calculation formula of the weight function is: , where t represents the timestamp of the current data frame, is the center time of the current window, is the adjustment rate; The calculation formula of the right sliding step size in the window is: , where represents the weight of the timestamp of the i-th data frame; Slide the window to the right according to the sliding step size .
[0030] The process of resource competition analysis is as follows: Define variables. Let the controller set be , N represents the total number of controllers; the shared resource set is , M represents the total number of shared resources, represents the utilization rate of the shared resource by the controller at time t; Calculate the competition intensity of the shared resource within the time window : , where y represents the sensitivity of controller latency to resource utilization; If , and the latency of at least one controller , then it is determined that is a resource bottleneck; where represents the minimum allowable competition intensity, represents the minimum allowable controller latency.
[0031] The foregoing description is only of the basic principles and preferred embodiments of the present invention. Modifications and substitutions made by those skilled in the art based on the present invention fall within the protection scope of the present invention.
Claims
1. A multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization, characterized in that: It includes a local quality of service monitoring module, a global quality of service monitoring module, a data frame alignment module, and a resource competition analysis module; A local quality of service monitoring module is embedded in each NVMe SSD controller, and the local quality of service monitoring module is used to collect the quality of service monitoring data of each NVMe SSD controller; The global quality of service monitoring module is used to collect the global resource occupancy status of the SSD in real time, generate a global heartbeat signal, and broadcast the global heartbeat signal to the local quality of service monitoring module regularly to achieve time synchronization; The data frame alignment module is used to align the quality of service data of multiple controllers along the global time axis; The resource competition analysis module is used to analyze the competition intensity of each controller for shared resources and determine the level of the performance bottleneck.
2. The multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization according to claim 1, wherein: The global heartbeat signal includes a global heartbeat count value and a global heartbeat timestamp. The global quality of service monitoring module uses a hardware timer to generate the global heartbeat signal periodically where the formula for generating the global heartbeat timestamp is: ， wherein represents the global heartbeat timestamp at the k-th time, represents the initial time reference, is the clock cycle.
3. The multi-controller NVMe SSD quality of service monitoring system based on global heartbeat synchronization according to claim 2, wherein: Each local quality of service monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global quality of service monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ； where represents the local time generated after calibration when the i-th controller receives the k-th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time.
4. The multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization according to claim 1, characterized in that: The data frame alignment module uses a sliding window algorithm to align the quality of service data of each controller. The alignment process is: Let the time window length be W, and the set of data frames of N controllers is included within the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowed time deviation, it is considered that the alignment is successful; If the alignment is not successful, the window boundary is adjusted through a weight function The weight function The calculation formula is as follows: ， where t represents the timestamp of the current data frame, is the center time of the current window, is the adjustment rate; Right sliding step length within the window The calculation formula is as follows: ， wherein represents the weight of the timestamp of the i-th data frame; According to the sliding step size Slide the window to the right.
5. The quality of service monitoring system for multi - controller NVMe SSD based on global heartbeat synchronization according to claim 1, characterized in that: The process of the resource competition module analyzing the competition intensity of each controller for shared resources is: First, define variables. Let the set of controllers be , where N represents the total number of controllers; the set of shared resources be , where M represents the total number of shared resources, represents the utilization rate of the shared resource by the controller at time t. Calculating shared resources within a time window for competition intensity : ， where y represents the sensitivity of controller latency to resource utilization; If and the latency of at least one controller then it is determined that is a resource bottleneck; wherein represents the minimum allowable competition intensity, represents the minimum allowable controller delay.
6. The multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization according to claim 1, characterized in that: The local quality of service parameters collected by the local quality of service monitoring module include I / O request latency, iops, the number of flash media accesses, and the single command size distribution.
7. The multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization according to claim 1, wherein: The global resource occupancy status of the SSD collected by the global quality of service monitoring module includes the bandwidth allocation of the parallel channels of the flash media and the throughput of the PCIe bus transport layer.
8. The multi - controller NVMe SSD quality - of - service monitoring system based on global heartbeat synchronization according to claim 3, characterized in that: Clock frequency compensation coefficient Calculated by linear regression of the clock deviation of adjacent 10 heartbeats. Let the local count of the i-th local heartbeat counter at the (k - 1)-th and k-th heartbeats be 、 , then the frequency compensation coefficient is .
9. The multi-controller NVMe SSD quality of service monitoring system based on global heartbeat synchronization according to claim 2, characterized in that: Initial time reference Synchronize with the host time.
10. A method for monitoring the quality of service of a multi - controller NVMe SSD based on global heartbeat synchronization, characterized in that: This method is implemented based on the monitoring system described in any one of claims 1-9, and includes three parts: global heartbeat synchronization, data frame time alignment, and resource competition analysis; The implementation process of global heartbeat synchronization is: The global service quality monitoring module uses a hardware timer to generate a global heartbeat signal periodically according to a cycle The global heartbeat timestamp generation formula is as follows: ， Among them represents the global heartbeat timestamp at the k-th time, represents the initial time reference, is the clock period; The global quality of service monitoring module sends the global heartbeat signal to each local quality of service monitoring module. Each local quality of service monitoring module maintains a local heartbeat counter. The local heartbeat counter receives the global heartbeat signal sent by the global quality of service monitoring module and performs local time calibration to achieve time synchronization. The local time calibration formula is: ； wherein represents the local time generated after calibration when the i-th controller receives the k-th global heartbeat signal, represents the clock frequency compensation coefficient, is the initial offset correction value, represents the previous calibration time; The implementation process of data frame time alignment is as follows: assume that the time window length is W, and the data frame set of N controllers is included in the window , and the timestamp of each frame is . If all data frames satisfy , where is the maximum allowed time deviation, it is regarded as successful alignment; If the alignment is not successful, then through the weight function adjust the window boundary, and the weight function The calculation formula is: ， where t represents the timestamp of the current data frame, is the center time of the current window, is the adjustment rate; Rightward sliding step length within the window The calculation formula is as follows: ， wherein represents the weight of the timestamp of the i-th data frame; According to the sliding step size Slide the window to the right; The implementation process of resource competition analysis is: Define variables. Let the set of controllers be , where N represents the total number of controllers; the set of shared resources be , where M represents the total number of shared resources, represents the utilization rate of the shared resource by the controller at time t. Calculating shared resources within a time window for the competition intensity : ， where y represents the sensitivity of controller latency to resource utilization; If and the latency of at least one controller , then it is determined that is a resource bottleneck; Among them represents the minimum allowable competition intensity represents the minimum allowable controller delay