Method, device and equipment for monitoring storage performance of server

Abstract

The invention provides a server storage performance monitoring method, device and equipment, and relates to the technical field of server operation and maintenance. The method comprises the following steps: acquiring a first index data sequence of storage equipment on a server on an equipment layer, a second index data sequence of an operating system layer and a third index data sequence of an application layer; according to the first index data sequence, the second index data sequence and the third index data sequence, constructing full-link performance portrait features of the storage device; and according to the full-link performance portrait features, performing performance baseline prediction by adopting a preset dynamic performance baseline model to obtain a first index data change curve of the storage device in a device layer, a second index data change curve of an operating system layer and a third index data change curve of an application layer. According to the method and the device, the performance bottleneck which does not occur can be predicted before the performance of the storage equipment is reduced, and the storage reliability and the service continuity of the server are improved.

Description

Technical Field

[0001] This application relates to the field of server operation and maintenance technology, and more specifically, to a method, apparatus, and device for monitoring the storage performance of a server. Background Technology

[0002] With the rapid development of emerging technologies such as cloud computing, big data, and artificial intelligence, data volume is experiencing explosive growth, placing higher demands on the performance, capacity, and response speed of storage systems. Non-Volatile Memory Express (NVMe), with its high-speed interface design based on the PCIe bus, can fully leverage the performance advantages of NAND flash memory. Compared to traditional SATA and SAS interface storage devices, it achieves orders of magnitude improvements in data transfer rate, IOPS (input / output operations per second), and latency. It has been widely applied in critical business scenarios sensitive to storage performance, such as financial transaction systems, high-performance computing clusters, and enterprise-level database services.

[0003] In the actual operation of NVMe storage devices, their performance stability directly determines the service quality of upper-layer applications. For example, in high-frequency trading scenarios in finance, millisecond-level latency fluctuations in NVMe storage can lead to delayed execution of trading instructions, resulting in economic losses. In large-scale AI model training scenarios, a decrease in NVMe storage IOPS can significantly prolong data retrieval time and reduce model training efficiency. Therefore, real-time monitoring of NVMe storage devices to promptly detect performance anomalies and ensure their stable operation has become one of the core requirements in the current storage management field.

[0004] Currently, existing NVMe storage monitoring technologies mainly revolve around real-time performance metric collection and anomaly alerts. However, these existing technologies only trigger alerts after performance degradation, missing the opportunity for intervention and failing to meet the forward-looking requirements for NVMe storage performance assurance in critical business scenarios. Due to the lack of predictive capabilities, the existing alert mechanisms are essentially "post-event responses"—alerts are only triggered when the performance metrics of the NVMe storage device have exceeded normal thresholds, performance has significantly degraded, and may affect upper-layer services. For some irreversible performance degradation (such as permanent flash block damage), post-event alerts may not even be able to recover the performance loss already caused, further reducing the reliability and service continuity of the NVMe storage system. Summary of the Invention

[0005] This application addresses the shortcomings of the prior art by providing a method, apparatus, and device for monitoring server storage performance, in order to solve the problems existing in the prior art.

[0006] The technical solution adopted in the embodiments of this application is as follows: In a first aspect, embodiments of this application provide a method for monitoring the storage performance of a server, comprising: Collect the first indicator data sequence of the storage device at the device layer, the second indicator data sequence at the operating system layer, and the third indicator data sequence at the application layer on the server; Based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence, construct the full-link performance profile of the storage device; Based on the full-link performance profile characteristics, a preset dynamic performance baseline model is used to predict the performance baseline, and the first indicator data change curve of the storage device at the device layer, the second indicator data change curve at the operating system layer, and the third indicator data change curve at the application layer are obtained. The first indicator data change curve is marked with at least the indicator data threshold of the device layer and the corresponding performance bottleneck probability; the second indicator data change curve is marked with at least the indicator data threshold of the operating system layer and the corresponding performance bottleneck probability; and the third indicator data change curve is marked with at least the indicator data threshold of the application layer and the corresponding performance bottleneck probability.

[0007] In one embodiment, constructing the end-to-end performance profile of the storage device based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence includes: The first indicator data sequence, the second indicator data sequence, and the third indicator data sequence are subjected to data standardization processing. The dynamic time warping algorithm is used to perform time-series alignment on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence after data standardization to obtain the full-link performance profile features.

[0008] In one embodiment, the data standardization process for the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence includes: Perform data transformation on the first indicator data sequence; Time alignment is performed on the second indicator data sequence; The third indicator data sequence is marked as abnormal data.

[0009] In one embodiment, the first indicator data sequence includes a hardware status data sequence and an indicator queue status sequence; the second indicator data sequence includes a kernel scheduling data sequence and a memory management data sequence; and the third indicator data sequence includes a service load data sequence and a resource contention data sequence.

[0010] In one embodiment, the method further includes: A preset hybrid filling algorithm is used to adjust the abnormal data in the third indicator data sequence.

[0011] In one embodiment, the method further includes: A preset root cause analysis algorithm is used to perform performance bottleneck root cause analysis on the first indicator data change curve, the second indicator data change curve, and the third indicator data change curve to obtain the performance bottleneck root cause of the storage device. The performance bottleneck root cause of the storage device includes at least one abnormal indicator type from the device layer, the operating system layer, and the application layer.

[0012] In one embodiment, the method further includes: A preset deep learning algorithm is used to generate adjustment strategies corresponding to the abnormal indicator types based on the root causes of the performance bottlenecks.

[0013] In one embodiment, the method further includes: If the abnormal indicator type includes the indicator type of the indicator queue status of the device layer, then the preset deep learning algorithm is used to generate a parameter optimization strategy for the indicator queue status of the device layer based on the root cause of the performance bottleneck of the device layer. And / or, If the abnormal indicator type also includes the kernel scheduling indicator type of the operating system layer, then the preset deep learning algorithm is used to generate a kernel scheduling optimization strategy for the operating system layer based on the root cause of the performance bottleneck of the operating system layer.

[0014] Secondly, embodiments of this application also provide a server storage performance monitoring device, comprising: The acquisition module is used to acquire the first indicator data sequence of the storage device on the server at the device layer, the second indicator data sequence at the operating system layer, and the third indicator data sequence at the application layer; The construction module is used to construct the full-link performance profile features of the storage device based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence. The prediction module is used to predict the performance baseline based on the full-link performance profile characteristics and a preset dynamic performance baseline model, so as to obtain the first indicator data change curve of the storage device at the device layer, the second indicator data change curve at the operating system layer, and the third indicator data change curve at the application layer. The first indicator data change curve is marked with at least the indicator data threshold of the device layer and the corresponding performance bottleneck probability; the second indicator data change curve is marked with at least the indicator data threshold of the operating system layer and the corresponding performance bottleneck probability; and the third indicator data change curve is marked with at least the indicator data threshold of the application layer and the corresponding performance bottleneck probability.

[0015] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the server storage performance monitoring method described in any of the above embodiments.

[0016] The beneficial effects of this application are as follows: This application provides a method for monitoring the storage performance of a server. It collects a first indicator data sequence at the device layer, a second indicator data sequence at the operating system layer, and a third indicator data sequence at the application layer of the storage device on the server. Based on the first, second, and third indicator data sequences, it constructs a full-link performance profile of the storage device. Finally, based on the full-link performance profile, it uses a preset dynamic performance baseline model to predict the performance baseline, obtaining the change curves of the first indicator data at the device layer, the second indicator data at the operating system layer, and the third indicator data at the application layer. This application can predict performance bottlenecks before they occur, improving the storage reliability and service continuity of the server. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 One of the flowcharts for a server storage performance monitoring method provided in this application embodiment; Figure 2 A second schematic flowchart of the server storage performance monitoring method provided in this application embodiment; Figure 3 The third flowchart illustrating the server storage performance monitoring method provided in this application embodiment; Figure 4 The fourth flowchart illustrating the server storage performance monitoring method provided in this application embodiment; Figure 5 A schematic diagram of the storage performance monitoring device for a server provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0023] This application provides a method for monitoring the storage performance of a server. This method can be generated by any electronic device with computing and processing capabilities. The electronic device can be, for example, a terminal-facing computer device or a backend server.

[0024] The following examples, in conjunction with the accompanying drawings, provide specific illustrations of the server storage performance monitoring method provided in this application.

[0025] Figure 1 This is one of the flowcharts illustrating the server storage performance monitoring method provided in the embodiments of this application, such as... Figure 1 As shown, the method includes: S101, The data acquisition server stores the first indicator data sequence at the device layer, the second indicator data sequence at the operating system layer, and the third indicator data sequence at the application layer.

[0026] The device layer corresponds to the physical hardware status and internal instruction scheduling logic of the storage device (the storage device in this application may be, for example, a solid-state drive that meets NVMe requirements), and mainly reflects the operational health, resource usage, and instruction processing status of the storage device; the operating system layer is the "bridge" between the storage device and the application layer, and is mainly responsible for scheduling IO requests, allocating resources (such as CPU and memory), and managing kernel-level parameters; the application layer is the "business target layer" for storage performance monitoring, which directly corresponds to the quality of service (QoS) of the upper-layer business. Its indicators reflect the actual impact of storage performance on the business and are the core basis for judging whether "performance meets business requirements".

[0027] In this embodiment, as shown in Table 1, the first indicator data sequence of the storage device at the device layer is obtained through the NVMe-MI 2.0 interface, including hardware status data sequences (e.g., temperature, wear leveling count, error type (read / write / erase)) and indicator queue status sequences (e.g., SQ / CQ depth, NCQ priority, number of incomplete commands); the second indicator data sequence of the storage device at the operating system layer is obtained using an eBPF hook, including kernel scheduling data sequences (e.g., io_uring queue saturation, CPU wait time, number of interrupts) and memory management data sequences (e.g., dirty page ratio, DMA cache hit rate); an NVMe data collection agent is deployed to obtain a third indicator data sequence, including service load data sequences (QPS, P99 latency, transaction abort rate, log throughput) and resource contention data sequences (lock contention time, number of thread blocking times). Here, "sequence" refers to data obtained in a time series (multiple consecutive time points).

[0028] Table 1 Indicator Data Sequence

[0029] After obtaining the first indicator data sequence of the device layer, the second indicator data sequence of the operating system layer, and the third indicator data sequence of the application layer, these data sequences can be displayed on a visualization interface. For example, a dynamic line chart can be used to represent the P99 latency trend, a heat map can be used to represent the io_uring queue saturation, and a pie chart can be used to represent the percentage of error types.

[0030] S102. Based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence, construct the full-link performance profile of the storage device.

[0031] Based on the first, second, and third indicator data sequences of S101, features are extracted from three dimensions: “device layer, operating system layer, and application layer”, to construct a full-link performance profile of the storage device.

[0032] S103. Based on the full-link performance profile characteristics, a preset dynamic performance baseline model is used to predict the performance baseline, and the first indicator data change curve of the storage device at the device layer, the second indicator data change curve of the operating system layer, and the third indicator data change curve of the application layer are obtained.

[0033] 1. Loading the preset dynamic performance baseline model In this embodiment, the preset dynamic performance baseline model is an LSTM-GRU hybrid network model, and the model structure and parameter configuration are as follows: Input layer: Receives the full-link performance profile features constructed by S102. The input data format is "time step × feature dimension" (in this embodiment, the time step is 60, that is, the input profile feature sequence of 60 collection cycles (6 seconds).

[0034] LSTM layer: number of hidden units = 32, activation function is ReLU, used to capture long-term dependencies of indicator data (such as the slow change trend of wear leveling count over time).

[0035] GRU layer: number of hidden units = 16, activation function is tanh, used to extract short-term fluctuation features of indicator data (such as a sudden increase in io_uring queue saturation caused by sudden I / O).

[0036] Attention layer: A 4-head attention mechanism (Multihead Attention) is adopted to assign weights to the output features of the GRU layer, giving priority to features that have a significant impact on performance (such as device layer temperature and application layer P99 latency).

[0037] Output layer: Outputs the predicted values ​​corresponding to the three data change curves (the first indicator data change curve of the device layer, the second indicator data change curve of the operating system layer, and the third indicator data change curve of the application layer), and also outputs the threshold of each curve and the performance bottleneck probability corresponding to the threshold point.

[0038] In addition, the model enables a dynamic sliding window mechanism, which automatically adjusts the window size to 60 based on the current business load type (i.e., makes predictions based on 6 seconds of historical data), and enables dynamic retraining every 5 minutes (updating model parameters to adapt to load changes).

[0039] 2. Model Prediction Execution Input data preparation: Extract feature data from the most recent 60 acquisition cycles (6 seconds) from the profile feature sequence of S102 to form the model input matrix X=[F_1,F_2,...,F_60] (where F_i is the 17-dimensional profile feature of the i-th acquisition cycle); input matrix X into the LSTM-GRU hybrid network, and obtain the prediction results through forward propagation, including: the predicted value of the device layer first indicator data sequence (denoted as D_pred), the predicted value of the operating system layer second indicator data sequence (denoted as O_pred), and the predicted value of the application layer third indicator data sequence for the next 15 minutes (900 acquisition cycles). Value (denoted as A_pred); dynamic thresholds for each level of indicators (calculated based on business load type, such as setting the device layer temperature threshold to 0.45 under mixed load (corresponding to ℃=(0.45×(100-0))×(1 / 1.8)-32 / 1.8≈53℃), setting the operating system layer io_uring queue saturation threshold to 0.65, and setting the application layer P99 latency threshold to 0.5); performance bottleneck probability for each indicator (calculated by the sigmoid function, probability=1 / (1+e^(-(predicted value-threshold) / threshold)), a probability>0.7 indicates a high risk of a performance bottleneck occurring within the next 15 minutes).

[0040] 3. Prediction Results Output: Curves showing the changes in three indicator data. The predicted D_pred, O_pred, and A_pred values ​​from the model are integrated with their corresponding thresholds and bottleneck probabilities to generate three visualized data change curves (the first indicator data change curve, the second indicator data change curve, and the third indicator data change curve). The first indicator data change curve is labeled with at least the device-level indicator data threshold and its corresponding performance bottleneck probability; the second indicator data change curve is labeled with at least the operating system-level indicator data threshold and its corresponding performance bottleneck probability; and the third indicator data change curve is labeled with at least the application-level indicator data threshold and its corresponding performance bottleneck probability. The annotations for each curve are as follows: The first indicator data change curve of the equipment layer: the horizontal axis is time (the next 15 minutes, unit: seconds), and the vertical axis is the value of the core indicator of the equipment layer (three key indicators are selected: temperature, SQ / CQ depth ratio, and total error count, and the values ​​have been standardized).

[0041] Optionally, the dynamic thresholds of each indicator can be marked with red dashed lines (e.g., temperature threshold = 0.45, SQ / CQ depth ratio threshold = 1.2, total error count threshold = 0.3); or, the performance bottleneck probability corresponding to each time point can be marked with numerical labels next to the curve (e.g., temperature prediction value at the 300th second = 0.42, bottleneck probability = 0.6; temperature prediction value at the 600th second = 0.48, bottleneck probability = 0.75); or regional shading can be used for marking. When a certain prediction value exceeds the threshold and the bottleneck probability > 0.7, the corresponding time interval is marked with red shading (e.g., temperature prediction value from the 600th to the 900th second > 0.45, marked as "high temperature bottleneck risk zone").

[0042] The second indicator data change curve of the operating system layer: the horizontal axis is time (the next 15 minutes, in seconds); the vertical axis is the value of the core indicators of the operating system layer (selecting three key indicators: io_uring queue saturation, CPU waiting time for IO, and DMA cache hit rate, the values ​​of which have been standardized).

[0043] Optionally, the dynamic thresholds of each indicator are marked with orange dashed lines (e.g., io_uring queue saturation threshold = 0.65, CPU wait time for IO threshold = 0.4, DMA cache hit rate threshold = 0.8); the performance bottleneck probability corresponding to each time point is marked with numerical labels next to the curves (e.g., predicted value of io_uring queue saturation at second 200 = 0.6, bottleneck probability = 0.5; predicted value at second 500 = 0.7, bottleneck probability = 0.82); risk areas are marked with shaded areas: when io_uring queue saturation > 0.65 and bottleneck probability > 0.7, the corresponding time interval is marked with orange shaded areas (e.g., second 500-800 is marked as "kernel scheduling bottleneck risk area").

[0044] Application layer third indicator data change curve: The horizontal axis is time (future 15 minutes, unit: seconds); the vertical axis is the application layer core indicator value (selecting three key indicators: MySQLP99 latency, transaction abort rate, and AI training log throughput, the values ​​have been standardized).

[0045] Optionally, the dynamic thresholds of each indicator are marked with blue dashed lines (e.g., P99 latency threshold = 0.5, transaction abort rate threshold = 0.2, log throughput threshold = 0.3); the performance bottleneck probability corresponding to each time point is marked with numerical labels next to the curve (e.g., predicted P99 latency at second 100 = 0.45, bottleneck probability = 0.4; predicted value at second 400 = 0.55, bottleneck probability = 0.78); risk areas are marked with shade: when P99 latency > 0.5 and bottleneck probability > 0.7, the corresponding time interval is marked with blue shade (e.g., second 400-700 is marked as "business response bottleneck risk area").

[0046] The accuracy of the prediction results for S103 was verified, and the trend of the indicators was verified: the predicted curve was compared with the actual acquisition curve (the D, O, A sequences actually acquired in the next 15 minutes). The results showed that the average error between the device layer temperature prediction curve and the actual curve was <5%, the prediction error of the io_uring queue saturation of the operating system layer was <8%, and the prediction error of the P99 latency of the application layer was <10%. Bottleneck probability verification: within the predicted "high temperature bottleneck risk zone" (600-900 seconds), the actual device layer temperature reached 54℃ (exceeding the threshold of 53℃), triggering performance frequency reduction, verifying that the accuracy of the bottleneck probability prediction reached 92%.

[0047] In summary, this embodiment provides a method for monitoring the storage performance of a server, which can predict performance bottlenecks before they occur and improve the storage reliability and service continuity of the server.

[0048] Figure 2 The second flowchart illustrating the server storage performance monitoring method provided in this application embodiment is as follows: Figure 2 As shown, step S102, which involves constructing a full-link performance profile of the storage device based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence, includes: S201. Perform data standardization processing on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence.

[0049] Specific implementation methods of data standardization are as follows: Figure 3 As shown, it includes S301-S303: S301, Perform data transformation on the first indicator data sequence.

[0050] Data conversion can be, for example, standardizing the units of the data. For instance, converting the temperature unit (°C) in the hardware status data sequence at the device layer to a unified unit (such as °F, the conversion formula is °F = °C × 1.8 + 32).

[0051] Data transformation can also involve scaling the values ​​of indicators at each level to the [0,1] range (to avoid the difference in numerical magnitude affecting subsequent model calculations).

[0052] S302. Time alignment of the second indicator data sequence.

[0053] Align the timestamps of the kernel scheduling data sequence (e.g., io_uring queue saturation, CPU wait time, interrupt count) and the memory management data sequence (e.g., dirty page ratio, DMA cache hit rate) in the second metric data sequence.

[0054] S303, Mark abnormal data in the third indicator data sequence.

[0055] Taking the QPS and P99 latency of the business load data sequence in the third indicator data sequence as an example, when the QPS or P99 latency exceeds the preset threshold (e.g., 1000 microseconds), the data exceeding 1000 microseconds is marked as abnormal data, and then the preset hybrid filling algorithm is used to adjust the abnormal data in the third indicator data sequence.

[0056] For example, take the value of the data preceding the current position of the abnormal data as the forward padding value; take the value of the data following the current position of the abnormal data as the backward padding value; then take the average of the forward padding value and the backward padding value, and replace the abnormal data with the average value.

[0057] S202. Using a dynamic time warping algorithm, the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence after data standardization are time-series aligned to obtain the full-link performance profile features.

[0058] The Dynamic Time Warping (DWT) algorithm is used to time-align the data sequences at the device layer, operating system layer, and application layer to correct the timestamp error of the data sequences at these layers to 1 millisecond. The specific implementation steps are as follows: 1. Configure key parameters of the DTW algorithm: Use the first index data sequence after data standardization as the time base sequence (because the hardware clock of the device layer has the highest accuracy and the strongest stability, it can be used as the base); take the timestamp of the first index data sequence as the target, adjust the timestamp and data distribution of the second and third index data sequences, and align the first, second, and third index data sequences in time.

[0059] 2. Based on the aligned first, second, and third indicator data sequences, extract five core features from the device layer (normalized temperature value, normalized wear leveling count value, normalized error type value, SQ / CQ depth ratio, and normalized NCQ priority value), five core features from the operating system layer (normalized io_uring queue saturation value, normalized CPU wait time value, normalized interrupt count value, normalized dirty page ratio value, and normalized DMA cache hit rate value), and five core features from the application layer (normalized QPS value, normalized P99 latency value, normalized transaction abort rate value, normalized log throughput value, and normalized thread blocking count value).

[0060] A total of 15 core features were extracted, resulting in a data format of timestamp + 15-dimensional features, which serves as the full-link performance profile feature.

[0061] Figure 4 The fourth flowchart illustrates the server storage performance monitoring method provided in this application embodiment. Figure 4 As shown, the method of this application further includes: S401. Using a preset root cause analysis algorithm, perform performance bottleneck root cause analysis on the first indicator data change curve, the second indicator data change curve, and the third indicator data change curve to obtain the performance bottleneck root cause of the storage device.

[0062] The preset root cause analysis algorithm could be, for example, an improved Apriori algorithm. Based on the three indicator data change curves output by S103, it extracts abnormal indicator data during the "bottleneck period (time interval with performance bottleneck probability > 0.7)" to form the root cause analysis input dataset (denoted as R_input). This dataset contains abnormal data for the first indicator at the device layer: during the bottleneck period (e.g., 10:00:00-10:15:00), the device layer temperature change curve shows that the temperature is consistently higher than the threshold of 53℃ (maximum 58℃, bottleneck probability 0.88), the SQ / CQ depth ratio is consistently > 1.2 (maximum 1.5, bottleneck probability 0.82), and the standardized value of the total error count is > 0.3 (maximum > 0.7). 0.45, bottleneck probability 0.75); Abnormal data for the second indicator of the operating system layer: The saturation change curve of the io_uring queue of the operating system layer during the same period shows saturation > 0.65 (maximum 0.92, bottleneck probability 0.91), and the normalized value of CPU waiting IO time > 0.4 (maximum 0.68, bottleneck probability 0.85); Abnormal data for the third indicator of the application layer: The latency change curve of the P99 of the application layer during the same period shows latency > 1000μs (maximum 1800μs, bottleneck probability 0.95), transaction abort rate > 0.2 (maximum 0.35, bottleneck probability 0.83), and AI training log throughput < 0.3 (minimum 0.22, bottleneck probability 0.78).

[0063] Based on the 8 transaction items transformed from R_input, frequent itemsets are filtered by "minimum support = 0.25 (currently a severe bottleneck, refer to the parameter table)". Then, based on the frequent itemsets, association rules are mined by "confidence threshold = 0.9 (severe bottleneck parameter)" (rule format is "bottom-level abnormal indicator combination" → "application layer abnormal indicator" or "root cause conclusion"), and sorted in descending order of confidence. Based on the logical chain of association rules, combined with the "root cause location confidence assessment" mechanism (excluding interference factors such as normal IO peaks, such as the current business QPS not exceeding the historical peak, excluding "business burst load" interference), the root cause of the performance bottleneck is finally located. The root cause includes at least one abnormal indicator type from the device layer, operating system layer, and application layer.

[0064] S402. Employ a pre-defined deep learning algorithm to generate adjustment strategies corresponding to abnormal indicator types based on the root causes of performance bottlenecks.

[0065] The default deep learning algorithm is the Deep Q-Network (DQN) algorithm, whose core design is tailored to NVMe storage performance optimization scenarios: 1. State Space: Includes the current values ​​of the root cause anomaly indicators located by S401 (device layer temperature 56℃, operating system layer io_uring saturation 0.92, CPU IO wait time 0.68, application layer P99 latency 1600μs), and the current resource utilization of the NVMe device (CPU utilization 75%, memory utilization 62%, NVMe bandwidth utilization 88%), for a total of 8 dimensions of state features.

[0066] 2. Action Space: Based on the "Storage Performance Optimization Strategy Library" (containing 23 preset strategies), six types of executable actions matching the current root cause are selected: NVMe queue depth adjustment - adjust SQ / CQ depth from 32 / 32 to 24 / 24; IO scheduling strategy switching - switch from the default mq-deadline scheduler to the Kyber scheduler (more suitable for mixed loads); CPU affinity configuration - bind NVMe device interrupts to CPU cores 1-4 (avoid interrupts occupying business cores); Device heat dissipation strategy optimization - enable active heat dissipation for NVMe devices (fan speed increased from 50% to 80%); MySQL transaction parameter adjustment - reduce InnoDB transaction isolation level from REPEATABLE READ to READCOMMITTED to reduce lock waits; AI training batch processing adjustment - reduce AI training batch size from 128 to 64 to reduce the number of single IO requests.

[0067] 3. Reward function: It is calculated by weighting "performance improvement + stability score" and the formula is Reward = 0.7 × (percentage reduction in P99 latency after optimization) + 0.3 × (1 - standard deviation of index fluctuation after optimization), where "standard deviation of index fluctuation" must be < 0.1 (to ensure stable performance after optimization). 4. Network Structure: Both PolicyNet and TargetNet adopt a 3-layer fully connected structure, with 8 neurons in the input layer (corresponding to the state space), 64 neurons in the hidden layer (with ReLU activation function), and 6 neurons in the output layer (corresponding to the action space, outputting the Q value of each action). The experience replay buffer capacity is set to 10000, the batch size is 64, and the initial value of ε in the ε-greedy policy is 0.9 (exploration probability), which decays by 0.01 every 100 training steps, down to a minimum of 0.1.

[0068] The performance bottleneck root cause of S401 is substituted into a preset deep learning algorithm to generate adjustment strategies corresponding to abnormal indicator types. For example, if the abnormal indicator type includes the indicator type of the device layer indicator queue status, then the preset deep learning algorithm is used to generate parameter optimization strategies for the device layer indicator queue status based on the performance bottleneck root cause of the device layer; and / or, if the abnormal indicator type includes the indicator type of the operating system layer kernel scheduling, then the preset deep learning algorithm is used to generate scheduling optimization strategies for the operating system layer kernel scheduling based on the performance bottleneck root cause of the operating system layer.

[0069] The following will continue to explain the apparatus, device and storage medium for implementing the server storage performance monitoring method provided in any of the above embodiments of this application. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, the parts not mentioned in the following embodiments can be referred to the corresponding content in the method embodiments.

[0070] like Figure 5 As shown, this application provides a server storage performance monitoring device, including: The acquisition module 10 is used to acquire the first indicator data sequence of the storage device on the server at the device layer, the second indicator data sequence at the operating system layer, and the third indicator data sequence at the application layer.

[0071] The construction module 20 is used to construct the full-link performance profile features of the storage device based on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence.

[0072] The prediction module 30 is used to predict the performance baseline based on the full-link performance profile characteristics using a preset dynamic performance baseline model, and to obtain the first indicator data change curve of the storage device at the device layer, the second indicator data change curve at the operating system layer, and the third indicator data change curve at the application layer. The first indicator data change curve is marked with at least the indicator data threshold and the corresponding performance bottleneck probability at the device layer; the second indicator data change curve is marked with at least the indicator data threshold and the corresponding performance bottleneck probability at the operating system layer; and the third indicator data change curve is marked with at least the indicator data threshold and the corresponding performance bottleneck probability at the application layer.

[0073] Optionally, the construction module 20 is further configured to perform data standardization processing on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence; and to use a dynamic time warping algorithm to perform time-series alignment on the first indicator data sequence, the second indicator data sequence, and the third indicator data sequence after data standardization processing to obtain the end-to-end performance profile features.

[0074] Optionally, the construction module 20 is further configured to perform data transformation on the first indicator data sequence; perform time alignment on the second indicator data sequence; and mark abnormal data on the third indicator data sequence.

[0075] Optionally, the device further includes an adjustment module for adjusting abnormal data in the third indicator data sequence using a preset hybrid filling algorithm.

[0076] Optionally, the device further includes an analysis module, used to perform performance bottleneck root cause analysis on the first indicator data change curve, the second indicator data change curve and the third indicator data change curve using a preset root cause analysis algorithm, to obtain the performance bottleneck root cause of the storage device, wherein the performance bottleneck root cause of the storage device includes at least one abnormal indicator type of the device layer, the operating system layer and the application layer.

[0077] Optionally, the device further includes a strategy generation module, used to generate an adjustment strategy corresponding to the abnormal indicator type based on the root cause of the performance bottleneck using a preset deep learning algorithm.

[0078] Optionally, if the abnormal indicator type includes: the indicator type of the indicator queue status of the device layer, the strategy generation module is further used to use the preset deep learning algorithm to generate a parameter optimization strategy for the indicator queue status of the device layer based on the root cause of the performance bottleneck of the device layer.

[0079] If the abnormal indicator type includes: the kernel scheduling indicator type of the operating system layer, the policy generation module is also used to use the preset deep learning algorithm to generate a scheduling optimization policy for the kernel scheduling of the operating system layer based on the root cause of the performance bottleneck of the operating system layer.

[0080] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0081] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0082] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, this application also provides an electronic device, including a processor 100, a storage medium 200 and a bus 300. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the server storage performance monitoring method described in any of the above embodiments.

[0083] This application also provides a readable storage medium storing program instructions, which, when executed by a processor, implement the server storage performance monitoring method described in any of the above embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0087] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of monitoring storage performance of a server, the method comprising: The method comprises the following steps: collecting first index data sequences of a device layer, second index data sequences of an operating system layer, and third index data sequences of an application layer of a storage device on a server; constructing a full-link performance portrait feature of the storage device according to the first index data sequences, the second index data sequences, and the third index data sequences; performing performance baseline prediction on the full-link performance portrait feature by using a preset dynamic performance baseline model to obtain a first index data change curve of the device layer, a second index data change curve of the operating system layer, and a third index data change curve of the application layer of the storage device; wherein the first index data change curve is marked with at least an index data threshold value of the device layer and a corresponding performance bottleneck probability, the second index data change curve is marked with at least an index data threshold value of the operating system layer and a corresponding performance bottleneck probability, and the third index data change curve is marked with at least an index data threshold value of the application layer and a corresponding performance bottleneck probability.

2. The method of claim 1, wherein, The method of constructing a full-link performance portrait feature of the storage device according to the first index data sequences, the second index data sequences, and the third index data sequences comprises the following steps: performing data standardization processing on the first index data sequences, the second index data sequences, and the third index data sequences; performing time series alignment on the first index data sequences, the second index data sequences, and the third index data sequences after data standardization processing by using a dynamic time warping algorithm to obtain the full-link performance portrait feature.

3. The method of claim 2, wherein, The method of performing data standardization processing on the first index data sequences, the second index data sequences, and the third index data sequences comprises the following steps: performing data conversion on the first index data sequences; performing time alignment on the second index data sequences; performing abnormal data marking on the third index data sequences.

4. The method of claim 3, wherein, The first index data sequences comprise hardware state data sequences and index queue state sequences, the second index data sequences comprise kernel scheduling data sequences and memory management data sequences, and the third index data sequences comprise business load data sequences and resource competition data sequences.

5. The method of claim 3, wherein, The method further comprises the following steps: performing data adjustment on abnormal data in the third index data sequences by using a preset mixed padding algorithm.

6. The method of claim 1, wherein, The method further comprises the following steps: performing performance bottleneck root cause analysis on the first index data change curve, the second index data change curve, and the third index data change curve by using a preset root cause analysis algorithm to obtain performance bottleneck root causes of the storage device, wherein the performance bottleneck root causes of the storage device at least include at least one abnormal index type of the device layer, the operating system layer, and the application layer.

7. The method of claim 6, wherein, The method further comprises the following steps: generating an adjustment strategy corresponding to the abnormal index type according to the performance bottleneck root causes by using a preset deep learning algorithm.

8. The method of claim 7, wherein, The method further comprises the following steps: If the abnormal index type includes: the index type of the index queue state of the device layer, the preset deep learning algorithm is adopted to generate the parameter optimization strategy of the index queue state of the device layer according to the performance bottleneck reason of the device layer. And / or, If the abnormal index type further includes: the index type of the kernel scheduling of the operating system layer, the preset deep learning algorithm is adopted to generate the scheduling optimization strategy of the kernel scheduling of the operating system layer according to the performance bottleneck reason of the operating system layer.

9. A storage performance monitoring apparatus of a server, characterized by comprising: Comprise: The acquisition module is used for acquiring the first index data sequence of the device layer, the second index data sequence of the operating system layer and the third index data sequence of the application layer of the storage device on the server; The construction module is used for constructing the full-link performance portrait feature of the storage device according to the first index data sequence, the second index data sequence and the third index data sequence; The prediction module is used for performing performance baseline prediction by adopting a preset dynamic performance baseline model according to the full-link performance portrait feature, to obtain the first index data change curve of the device layer, the second index data change curve of the operating system layer and the third index data change curve of the application layer of the storage device; Wherein, the first index data change curve is at least marked with: the index data threshold of the device layer and the corresponding performance bottleneck probability, the second index data change curve is at least marked with: the index data threshold of the operating system layer and the corresponding performance bottleneck probability, and the third index data change curve is at least marked with: the index data threshold of the application layer and the corresponding performance bottleneck probability.

10. An electronic device, comprising: Comprise: The processor, the storage medium and the bus, the storage medium stores the program instructions executable by the processor, when the electronic equipment runs, the processor and the storage medium communicate through the bus, the processor executes the program instructions, to realize the storage performance monitoring method of the server of any one of claims 1 to 8.

Images