Monitoring method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a monitoring method and device, equipment, a storage medium and a program product, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: determining a mapping relationship between a storage logic unit LUN and a virtual machine according to configuration data of a storage system; in the running process of the storage system, performance data of the LUN are obtained, and the performance data comprise at least one of input and output operation times IOPS, throughput, time delay and processor utilization rate. And under the condition that the target performance data located outside the preset performance interval exists, determining an abnormal virtual machine according to the LUN corresponding to the target performance data and the mapping relationship, and triggering a fault response strategy. According to the method provided by the invention, the efficiency and accuracy of detecting the abnormality of the virtual machine in the storage system are improved.

Description

Technical Field

[0001] This application relates to the field of financial technology or other related fields, and in particular to a monitoring method, device, equipment, storage medium and program product. Background Technology

[0002] Against the backdrop of accelerated information innovation and transformation in the financial industry, Remote Direct Memory Access (RDMA) over Converged Ethernet Storage Area Network (RoCE-SAN), with its advantages of low latency, high bandwidth, and Ethernet compatibility, is gradually replacing traditional Fibre Channel Storage Area Network (FC-SAN) solutions and becoming the mainstream storage network architecture in cloud data centers. In a typical cloud virtual machine scenario, the host machine connects to centralized storage devices through RoCE switches to carry core business data. However, in existing monitoring solutions, the data processing logic of each monitoring layer, such as network, storage, and host, is relatively isolated and lacks effective linkage, resulting in insufficient timeliness of emergency response.

[0003] Therefore, improving the efficiency and accuracy of virtual machine anomaly detection in storage systems is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a monitoring method, apparatus, device, storage medium, and program product to solve the technical problem of low efficiency and accuracy in detecting virtual machine anomalies in storage systems.

[0005] Firstly, this application provides a monitoring method, including:

[0006] Based on the storage system configuration data, determine the mapping relationship between storage logical units (LUNs) and virtual machines;

[0007] During the operation of the storage system, the performance data of the LUN is acquired, and the performance data includes at least one of the following: input / output operation count (IOPS), throughput, latency, and processor utilization.

[0008] If target performance data exists outside the preset performance range, the abnormal virtual machine is determined based on the LUN corresponding to the target performance data and the mapping relationship, and a fault response strategy is triggered.

[0009] Optionally, determining the mapping relationship between LUNs and virtual machines based on the storage system configuration data includes:

[0010] Based on the configuration data, the data processing flow chain of the host machine, the virtual machine, the storage device, and the LUN in the storage system is determined;

[0011] Based on the data processing flow chain, the host machine identifier, the virtual machine identifier, the storage device identifier, and the LUN identifier, a flow relationship mapping table is generated. The flow relationship mapping table is used to represent the mapping relationship between the LUN and the virtual machine.

[0012] Optional, also includes:

[0013] Obtain historical performance data of the LUN within the target time period;

[0014] Extract the periodic time-series features of the historical performance data, wherein the periodic time-series features are related to the business cycle of the storage system;

[0015] Based on the periodic time sequence characteristics, the preset performance range is determined, and the preset performance range is different for different business cycles.

[0016] Optionally, extracting the periodic time-series features of the historical performance data includes:

[0017] The periodic time-series features are extracted from the historical performance data using a pre-trained time-series feature extraction model.

[0018] Optionally, determining the preset performance range based on the periodic time series characteristics includes:

[0019] Based on the periodic time-series characteristics and threshold generation rules, a threshold change curve for the performance data is generated;

[0020] The preset performance range is determined based on the threshold change curve.

[0021] Optional, also includes:

[0022] Obtain incremental data of the historical performance data of the LUN;

[0023] The preset performance range is fine-tuned based on the incremental data to obtain the updated preset performance range.

[0024] Optional, also includes:

[0025] Based on the threshold change curve, at least two levels of fault response threshold curves are generated. The threshold change curve and the at least two levels of the fault response threshold curves are used to determine the abnormality level of the abnormal virtual machine.

[0026] Optionally, the fault response threshold curve includes a threshold alarm curve and a threshold isolation curve. The step of generating at least two levels of fault response threshold curves based on the threshold change curve includes:

[0027] The threshold alarm curve is determined based on the threshold change curve and the first preset value, wherein the first preset value is greater than zero;

[0028] The threshold isolation curve is determined based on the threshold change curve and the second preset value, wherein the second preset value is greater than the first preset value;

[0029] The fault response triggering strategy includes:

[0030] If the target performance data is located between the threshold change curve and the threshold alarm curve, an alarm message will be output.

[0031] If the target performance data is located between the threshold alarm curve and the threshold isolation curve, then the abnormal virtual machine is isolated and hot-migrated to another host machine.

[0032] Secondly, this application provides a monitoring device, comprising:

[0033] The processing module is used to determine the mapping relationship between logical storage units (LUNs) and virtual machines based on the configuration data of the storage system.

[0034] The acquisition module is used to acquire performance data of the LUN during the operation of the storage system. The performance data includes at least one of the following: input / output operation count (IOPS), throughput, latency, and processor utilization.

[0035] The control module is used to determine the abnormal virtual machine based on the LUN corresponding to the target performance data and the mapping relationship when target performance data exists outside the preset performance range, and to trigger a fault response strategy.

[0036] Thirdly, this application provides an electronic device, including: a processor and a memory; the processor and the memory are communicatively connected.

[0037] The memory stores computer-executed instructions;

[0038] The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0039] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the monitoring method as described in any one of the first aspects.

[0040] Fifthly, this application provides a computer program product, which, when executed by a processor, is used to implement the monitoring method as described in any one of the first aspects.

[0041] The monitoring method, apparatus, equipment, storage medium, and program products provided in this application determine the mapping relationship between LUNs and virtual machines by parsing the configuration data of the storage system, and acquire the performance data of LUNs during the operation of the storage system. When there is target performance data outside the preset performance range, the abnormal virtual machine is identified according to the LUN and mapping relationship corresponding to the target performance data and the fault response strategy is triggered. The association between various components in the storage system and the real-time monitoring performance data are first clarified, and then the abnormal virtual machine can be accurately located when performance is abnormal, thereby realizing the rapid discovery and handling of virtual machine anomalies in the storage system and ensuring the stable operation of the storage system. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 A flowchart illustrating a monitoring method provided in an embodiment of this application;

[0044] Figure 2 A flowchart illustrating another monitoring method provided in an embodiment of this application;

[0045] Figure 3 A flowchart illustrating yet another monitoring method provided in an embodiment of this application;

[0046] Figure 4 A flowchart illustrating another monitoring method provided in an embodiment of this application;

[0047] Figure 5 A flowchart illustrating another monitoring method provided in an embodiment of this application;

[0048] Figure 6 This application provides a schematic diagram of the structure of a monitoring system according to an embodiment of the present application.

[0049] Figure 7 This is a schematic diagram of the structure of a monitoring device provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0054] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0055] It should be noted that the monitoring methods, devices, equipment, storage media, and program products provided in this application can be used in the field of financial technology or other related fields, or in any field other than the field of financial technology or other related fields. The application fields of the monitoring methods, devices, equipment, storage media, and program products in this application are not limited.

[0056] Currently, in cloud virtual machine-based storage system scenarios, the host machine can connect to centralized storage devices via RoCE switches to carry core business data. Existing conventional RoCE-SAN storage network monitoring solutions, while possessing multi-level monitoring and alarm capabilities, and covering network, storage, and host aspects at each monitoring level, suffer from relatively isolated data processing logic and a lack of effective linkage between different levels during implementation.

[0057] Regarding monitoring granularity, since the RoCE protocol itself does not encapsulate virtual machine identification information, network traffic monitoring can only be refined to the host machine's physical network interface card level, making it impossible to accurately locate specific virtual machines. Simultaneously, performance data of Logical Unit Numbers (LUNs) on the storage side cannot be directly mapped to specific virtual machines. This forces operations personnel to manually establish the mapping relationship between virtual machines and LUNs when encountering anomalies, increasing the complexity and workload of operations and maintenance.

[0058] In terms of fault localization, existing monitoring methods struggle to capture virtual machine-level performance degradation, typically only discovering problems when business operations crash. Furthermore, when a fault occurs, troubleshooting requires tracing from the network layer to the host layer and then to the virtual machine layer. This process relies heavily on manual operations, resulting in low efficiency and difficulty in quickly locating the root cause of the fault.

[0059] In view of this, this application provides a monitoring method that determines the mapping relationship between LUNs and virtual machines by parsing the configuration data of the storage system, and obtains the performance data of LUNs during the operation of the storage system. When there is target performance data outside the preset performance range, the abnormal virtual machine is identified according to the LUN and mapping relationship corresponding to the target performance data and a fault response strategy is triggered. The method first clarifies the relationship between the components in the storage system and monitors the performance data in real time, and then accurately locates the abnormal virtual machine when performance is abnormal, thereby realizing the rapid discovery and handling of virtual machine anomalies in the storage system and ensuring the stable operation of the storage system.

[0060] The entity executing the monitoring method provided in this application embodiment may be an electronic device with monitoring capabilities, such as a mobile phone, computer, or server, or a monitoring system used to monitor the performance data of storage devices in a storage system.

[0061] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below through specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0062] Figure 1 This is a flowchart illustrating a monitoring method provided in an embodiment of this application. Figure 1 As shown, the method may include:

[0063] S101. Determine the mapping relationship between LUN and virtual machine based on the storage system configuration data.

[0064] In this embodiment, the storage system configuration data includes detailed information about the storage system architecture, device connections, and resource allocation. This configuration data may be stored, for example, in a dedicated configuration database or in the form of configuration files.

[0065] The configuration data can record the attribute information and interrelationship rules of the host machine, virtual machines, storage devices, and LUNs. For example, the configuration data may include the virtual machines included in each host machine, the connection between the host machine and the corresponding storage device through a specific storage channel, the multiple LUNs in the storage device, the correspondence between each virtual machine and the host machine, storage device, and LUN, and the read and write operations that virtual machines can perform on their corresponding LUNs.

[0066] In this step, the configuration data can be parsed. For example, a data parsing tool can be used to convert the configuration data from its original storage format into structured data that is easier to process. For instance, if the configuration data is stored in XML format, an XML parser can be used to parse it into a tree structure, where each node represents a configuration item. By traversing this tree structure, the required information can be extracted. Next, key information related to LUNs and virtual machines is extracted from the parsed data. This key information may include the LUN identifier, the virtual machine identifier, the host machine identifier, and the data transfer paths between the host machine, virtual machine, storage device, and LUN. For example, the configuration data may include virtual machine VM-A accessing LUN-1 on storage device Storage-1 through host machine Host-1. Based on this, the mapping relationship between VM-A and LUN-1 can be determined.

[0067] S102. During the operation of the storage system, obtain the performance data of the LUN.

[0068] The performance data includes at least one of the following: Input / Output Operations Per Second (IOPS), throughput, latency, and processor utilization.

[0069] When a storage system is running, appropriate monitoring mechanisms can be deployed to obtain LUN performance data. For example, performance monitoring programs can be installed on storage devices to periodically collect various performance metrics of the LUNs (e.g., once per minute).

[0070] For example, to obtain IOPS, the number of input / output operations performed by the LUN within a fixed time interval can be recorded. For instance, the number of read and write operations on the LUN can be counted every 10 seconds, and these statistics can be used as the IOPS measurement.

[0071] Throughput is obtained by monitoring the amount of data transmitted by the LUN per unit of time. For example, the inflow and outflow of data can be recorded, and the throughput can be obtained by calculating the total amount of data transmitted over a period of time. For example, if the LUN transmits a certain amount of data in one minute, this amount of data can be divided by the time (one minute) to obtain the throughput.

[0072] Latency refers to the time elapsed from initiating an operation to its completion. For example, the initiation and completion times of each operation can be recorded, and the difference between them can be calculated as the latency. For instance, when a read operation is initiated, the initiation time is recorded, and when the operation is completed, the completion time is recorded; the difference between these two times is the latency of that read operation.

[0073] Processor utilization is obtained by monitoring the percentage of processor resources used by a LUN during operation. For example, this can be achieved by interacting with the host operating system to obtain processor usage information and calculate the percentage used by LUNs. For instance, given a host machine's total processor capacity, the percentage of processing power used by LUNs over a given period is calculated as CPU utilization.

[0074] S103. If target performance data exists outside the preset performance range, determine the abnormal virtual machine based on the LUN and mapping relationship corresponding to the target performance data, and trigger the fault response strategy.

[0075] The preset performance range is a reasonable performance range determined based on factors such as the storage system's historical performance data, business needs, and system design requirements.

[0076] One possible approach is to use statistical analysis to determine the preset performance range. For example, performance data of LUNs over a period of time can be collected, and statistical measures such as the mean and standard deviation of these data can be calculated. Then, based on business requirements and system stability requirements, a reasonable fluctuation range can be determined, and the mean plus or minus a certain multiple of the standard deviation can be used as the upper and lower limits of the preset performance range.

[0077] Another possibility is the absorption approach, which involves extracting the temporal characteristics of historical performance data and determining the changes in data volume (such as periodic changes) based on these characteristics. For example, in a banking system, the extracted temporal characteristics can be used to determine the business cycle forming peak / trough periods, which corresponds to the periodic changes in data volume. Based on these temporal characteristics corresponding to the business cycle, a time-related dynamically preset performance range can be determined.

[0078] In this step, if there is performance data for a LUN that exceeds the preset performance range, then that performance data is the target performance data. Based on the LUN corresponding to the target performance data and the previously determined mapping relationship, it can be determined which virtual machine (i.e., the virtual machine corresponding to the LUN corresponding to the target performance data) is experiencing an anomaly. For example, assuming the IOPS of LUN-1 exceeds the preset range, by querying the mapping relationship, the virtual machine VM-A using LUN-1 can be found, and VM-A is the abnormal virtual machine.

[0079] Once the abnormal virtual machine is identified, an appropriate fault response strategy can be triggered. Optionally, the fault response strategy can be related to the category of performance data; for example, if different categories of performance data show abnormalities, corresponding fault response measures can be taken. Alternatively, the fault response strategy can also be tiered according to the severity of the anomaly. For example, for minor anomalies, an alarm message can be issued to remind the administrator to pay attention; for moderate anomalies, resource restrictions can be imposed on the abnormal virtual machine to prevent it from affecting the normal operation of other virtual machines; for severe anomalies, the abnormal virtual machine can be isolated and migrated to another host to ensure the overall stability of the storage system, etc.

[0080] The method provided in this application determines the mapping relationship between LUNs and virtual machines by parsing the configuration data of the storage system, and obtains the performance data of LUNs during the operation of the storage system. When there is target performance data outside the preset performance range, the abnormal virtual machine is identified according to the LUN and mapping relationship corresponding to the target performance data and the fault response strategy is triggered. The method first clarifies the association of each component in the storage system and monitors the performance data in real time, and then accurately locates the abnormal virtual machine when the performance is abnormal, thereby realizing the rapid discovery and handling of virtual machine anomalies in the storage system and ensuring the stable operation of the storage system.

[0081] The following section provides a detailed explanation of how the mapping relationship between LUNs and virtual machines is determined based on the storage system configuration data in step S101. Figure 2 This is a flowchart illustrating another monitoring method provided in an embodiment of this application. Figure 2 As shown, the method may include:

[0082] S201. Based on the configuration data, determine the data processing flow links of the host machine, virtual machine, storage device, and LUN in the storage system.

[0083] The storage system's configuration data details the data interaction rules and processes between the host machine, virtual machines, storage devices, and LUNs. In-depth analysis and understanding of this configuration data can help determine the data processing flow links between these components.

[0084] Specifically, configuration data can be categorized and organized according to different components. For example, configuration information related to the host machine can be grouped into one category, including the host machine's hardware configuration, network connection information, and operating system version; configuration information related to virtual machines can be grouped into another category, including virtual machine resource allocation, network settings, and storage mounting. Then, based on the connection relationships and interaction rules recorded in the configuration data, a data processing flow model can be constructed. For example, the configuration data may indicate that a virtual machine connects to the host machine's physical network interface through a specific virtual network interface, the host machine then communicates with storage devices through the storage area network, and the storage devices store data in different LUNs. Based on this information, a data processing flow chain from the virtual machine to the LUN can be constructed, including key information such as data flow direction, transmission protocol, and access permissions.

[0085] Optionally, during the model building process, graphical tools can be used to visualize the data processing flow. For example, flowchart tools can be used to draw the connections and data flows between virtual machines, host machines, storage devices, and LUNs to more intuitively demonstrate the entire data processing flow.

[0086] S202. Generate a process relationship mapping table based on the data processing flow link, the host machine identifier, the virtual machine identifier, the storage device identifier, and the LUN identifier.

[0087] The process relationship mapping table is used to represent the mapping relationship between LUNs and virtual machines.

[0088] In this step, after determining the data processing flow chain, a flow relationship mapping table can be generated by combining the identifiers of the host machine, virtual machine, storage device, and LUN. This identifier can be, for example, a device name, device code, or device ID, used to distinguish different components.

[0089] Associating each node and connection in the data processing workflow with its corresponding identification information. For example, in the data processing workflow, virtual machine VM-A connects to LUN-1 on storage device Storage-1 through host machine Host-1. The identification information of VM-A, Host-1, Storage-1, and LUN-1 is recorded, and the association between them is established.

[0090] For example, these relationships can be stored in the form of a table, which is called a process relationship mapping table. The columns of the process relationship mapping table can include information such as virtual machine identifier, host identifier, storage device identifier, and LUN identifier. For example:

[0091] Table 1

[0092]

[0093] The method provided in this application determines the data processing flow links of each component in the storage system based on configuration data, and generates a flow relationship mapping table by combining the identification information of each component, so as to accurately determine the mapping relationship between LUN and virtual machine, thereby providing an important basis for subsequent performance monitoring and fault handling.

[0094] Figure 3 This is a flowchart illustrating yet another monitoring method provided in an embodiment of this application. Figure 3 As shown, the method may further include:

[0095] S301. Obtain historical performance data of LUN within the target time period.

[0096] In this embodiment, the target time period can be determined according to specific business needs and analysis objectives. This application does not limit this. For example, it can be historical performance data of LUNs within the most recent month, historical performance data of LUNs within the past week, historical performance data of LUNs within the past day, etc.

[0097] In this step, data can be obtained from sources such as log files or databases specifically used to store historical performance data for various storage devices. Taking a database as an example, various performance metrics for LUNs can be collected and saved periodically. During the query, appropriate query conditions can be set based on the target time period to retrieve the corresponding historical performance data. For instance, assuming the target time period is from start time T1 to end time T2, a database query can be used to specify the time range as T1 to T2, filtering out the performance data of LUNs within that time period.

[0098] Optionally, historical data warehouses can also be partitioned for storage. For example, partitioning by time allows data from different time periods to be stored in different partitions. During a query, only the partition containing the target time period can be scanned, reducing unnecessary data scanning and thus improving the efficiency of retrieving historical performance data.

[0099] S302. Extract the periodic time-series features of historical performance data.

[0100] Among them, the periodic time-series characteristics are related to the business cycle of the storage system.

[0101] One possible approach is to perform a Fourier transform on the historical performance data to convert it from the time domain to the frequency domain, and then analyze the frequency components in the frequency domain to determine the periodic components, which correspond to the periodic time-series characteristics of the historical performance data.

[0102] Another possible approach is to extract periodic time-series features from historical performance data using a pre-trained time-series feature extraction model. This pre-trained model can be, for example, a deep learning-based neural network model, such as a Recurrent Neural Network (CNN) or a Long Short-Term Memory (LSTM) network. Before using the pre-trained model, it can be fine-tuned. For instance, a portion of the historical performance data can be used as training data to adjust the model's parameters, making it more suitable for the business characteristics of the storage system. By inputting the historical performance data into the time-series feature extraction model, it can automatically learn the time-series features in the data and output the corresponding periodic time-series features.

[0103] S303. Determine the preset performance range based on the periodic timing characteristics.

[0104] The preset performance ranges differ for different business cycles. For example, taking financial business as an example, during peak business hours (such as daytime), a wider preset performance range can be set to meet the processing capacity of large data volumes and reduce false alarms caused by unreasonable preset performance ranges. During off-peak business hours (such as nighttime), a narrower preset performance range can be set to ensure the sensitivity and accuracy of anomaly monitoring when data volumes are small.

[0105] One possible implementation involves statistically analyzing the maximum, minimum, and average values ​​of historical performance data across different business cycles based on periodic time-series characteristics. Then, a preset performance range can be determined using these statistical values. For example, historical performance data can be divided into different business cycles based on periodic time-series characteristics. If the periodic time-series characteristics show daily periodic changes in historical performance data, the data can be divided into daily periods, with each period corresponding to one day's data. For each business cycle, the maximum, minimum, and average values ​​of historical performance data within that cycle are calculated. Statistical analysis tools can be used to perform these calculations. For example, for IOPS performance metrics, the maximum, minimum, and average values ​​of all IOPS data within a given business cycle are statistically analyzed. A reasonable fluctuation range is determined based on business requirements and system stability requirements. A fluctuation coefficient can be set based on experience or experimental data. For example, the average value plus or minus the fluctuation coefficient multiplied by the standard deviation can be used as the upper and lower limits of the preset performance range. Different fluctuation coefficients can be set for different business cycles to adapt to different business characteristics.

[0106] Another possible implementation is to determine the preset performance range based on periodic time-series characteristics and threshold generation rules. Specifically, this implementation can be achieved through the following sub-steps:

[0107] S3031. Based on the periodic time series characteristics and threshold generation rules, generate the threshold change curve of the performance data.

[0108] Threshold generation rules are pre-defined rules based on the business needs and performance requirements of the storage system. By combining periodic time-series characteristics with threshold generation rules, threshold variation curves for performance data can be generated.

[0109] In this step, periodic time-series characteristics can be associated with threshold generation rules. For example, the threshold generation rule can stipulate that the threshold for performance data is higher during peak business periods than during off-peak periods. The peak and off-peak time periods are determined based on the periodic time-series characteristics, and then thresholds for different time periods are set according to the threshold generation rules.

[0110] Then, these thresholds are connected in chronological order to form a threshold variation curve. For example, interpolation algorithms can be used to smooth the thresholds, making the curve more continuous and smooth. For instance, using spline interpolation, the threshold at intermediate time points is calculated based on known threshold points, thus obtaining a continuous threshold variation curve.

[0111] When performance data exceeds the threshold change curve, it indicates abnormal performance data; if it does not exceed the threshold, it indicates normal performance data. For example, taking processor utilization at a certain moment as an example, assume the threshold change curve at that moment has an upper limit of 80% and a lower limit of 50% (here, it is assumed that the upper and lower limits are each a separate threshold change curve; in this application, only a threshold change curve controlling the upper limit can be set, or only a threshold change curve controlling the lower limit can be set, etc.). If the processor utilization is 60%, it indicates that the processor utilization is normal; if the processor utilization is 85% or 48%, it indicates that the processor utilization is abnormal.

[0112] S3032. Determine the preset performance range based on the threshold change curve.

[0113] Based on the generated threshold change curve, the reasonable range of performance data corresponding to each time point can be determined, which is the preset performance range.

[0114] Specifically, for the threshold change curve, its upper and / or lower limits represent the maximum and minimum allowable values ​​of the performance data, respectively. At each time point, the upper and lower limits of the threshold change curve are used as the upper and lower limits of the preset performance range. For example, at time point t, if the upper limit of the threshold change curve is U(t) and the lower limit is L(t), then the preset performance range at that time point is [L(t), U(t)].

[0115] Different business cycles correspond to different threshold change curves, and therefore different preset performance ranges. In this way, the preset performance range can be dynamically adjusted according to the business cycle of the storage system, improving the accuracy of performance monitoring.

[0116] The method provided in this application embodiment obtains historical performance data of LUN within a target time period, extracts periodic time-series features from it, and determines a preset performance range based on these features, so as to realize dynamic monitoring of storage system performance and thus better adapt to business changes in the storage system.

[0117] Figure 4 This is a flowchart illustrating another monitoring method provided in an embodiment of this application. Figure 4 As shown, the method may further include:

[0118] S401. Obtain incremental data of historical performance data of LUN.

[0119] As the storage system continues to operate, new LUN performance data will be constantly generated. This newly generated data, relative to previous historical data, constitutes incremental data. Incremental data can be collected using the methods described above for collecting LUN performance data.

[0120] Optionally, incremental data can be collected either periodically or in real-time. Periodic collection refers to collecting incremental data at regular intervals (e.g., every 10 minutes); real-time collection involves collecting data immediately when new data is generated. The appropriate collection method can be chosen based on actual business needs and system performance requirements.

[0121] S402. Fine-tune the preset performance range based on the incremental data to obtain the updated preset performance range.

[0122] One possible implementation involves merging and analyzing the acquired incremental data with previous historical performance data. The incremental data is inserted into the historical performance data in chronological order to form a new dataset. Then, based on this dataset, a new preset performance range is generated using the aforementioned method of generating preset performance ranges from historical performance data; this is the updated preset performance range.

[0123] Another possible implementation is to fine-tune the pre-generated preset performance range directly based on incremental data. For example, key time-series patterns can be extracted from the incremental data, and the preset performance range can be adjusted in real time by identifying short-term trends and periodic clues in the data points. For instance, if the incremental data shows a continuous upward trend within three consecutive time windows (e.g., the last value of each window is higher than the last value of the previous window), and the increase exceeds 10% of the current preset range width, it can be determined as a "short-term performance surge." In this case, the upper limit of the preset range can be dynamically increased by 1.1 times the last value of the incremental data (e.g., adjusted from 2000 IOPS to 2200 IOPS), while keeping the lower limit unchanged to avoid false low-load anomalies. Conversely, if the incremental data shows a continuous decline within three consecutive windows and the decline exceeds 10% of the range width, it is determined as a "short-term performance decline," and the lower limit is tightened by 0.9 times the last value (e.g., adjusted from 500 IOPS to 450 IOPS).

[0124] Optionally, the stability and reliability of the incremental data also need to be considered during fine-tuning. Abnormal fluctuations in the incremental data may lead to over-adjustment of the preset performance range. To avoid this, the incremental data can be smoothed. For example, a moving average method can be used to calculate the average value of the incremental data over a recent period, and this average value can be used to replace the original incremental data for fine-tuning the preset performance range. This reduces the impact of abnormal fluctuations on the fine-tuning results, making the adjustment of the preset performance range more stable and reasonable.

[0125] Optionally, an adjustment threshold can be set. Adjustments are only made when the change in the preset performance range caused by incremental data exceeds this threshold. For example, the upper limit of the preset performance range is only updated if the difference between the newly calculated upper limit and the original upper limit exceeds the adjustment threshold T; similarly, the lower limit is judged in the same way. This avoids frequent small adjustments and improves the stability and efficiency of the system.

[0126] The method provided in this application embodiment obtains incremental data of historical performance data of LUN and fine-tunes the preset performance range based on this incremental data to update the preset performance range in real time, thereby better adapting to the dynamic changes in storage system performance.

[0127] Optionally, a multi-level fault response mechanism can be constructed to improve the ability to handle abnormal situations, thereby ensuring the stable operation of the storage system. For example, a preset performance range can be divided into multiple regions, each corresponding to a first-level fault response mechanism. At least some regions may have different fault response strategies to achieve different fault responses and handling for different degrees of anomalies. For instance, at least two levels of fault response threshold curves can be generated based on threshold change curves. These threshold change curves and the at least two levels of fault response threshold curves are used to determine the anomaly level of the abnormal virtual machine.

[0128] Taking the fault response threshold curve, including the threshold alarm curve and the threshold isolation curve, as an example, Figure 5 This is a flowchart illustrating another monitoring method provided in an embodiment of this application. Figure 5 As shown, the method may specifically include:

[0129] S501. Determine the threshold alarm curve based on the threshold change curve and the first preset value.

[0130] The first preset value is greater than zero.

[0131] The threshold variation curve reflects the reasonable range of performance data across different business cycles. The first preset value is set based on the storage system's business needs and risk tolerance; for example, it can be a positive number. The first preset value represents an additional range of performance data fluctuations allowed beyond the threshold variation curve.

[0132] In this step, for each time point on the threshold change curve, the threshold corresponding to that time point is added to a first preset value to obtain the value of the threshold alarm curve at that time point. For example, at time point t, the value of the threshold change curve is T(t), and the first preset value is 20%. Then, the value A(t) of the threshold alarm curve at time point t can be expressed as A(t) = T(t) * (1 + 20%). By performing this operation on all time points on the threshold change curve, the complete threshold alarm curve is obtained.

[0133] If the first preset value is set too low, it may lead to frequent alarms, increasing the burden on administrators; if the first preset value is set too high, some potential anomalies may not be detected in a timely manner. Therefore, in practical applications, the setting of the first preset value can be comprehensively considered based on factors such as historical data and actual business experience, and this application does not impose any restrictions on it.

[0134] S502. Determine the threshold isolation curve based on the threshold change curve and the second preset value, wherein the second preset value is greater than the first preset value.

[0135] The second preset value is set based on the storage system's business needs and risk tolerance, and it is greater than the first preset value. The second preset value represents a more lenient range of performance data fluctuations than the threshold alarm curve, based on the threshold change curve, and is used to identify intervals where anomalies are more severe.

[0136] In this step, the method for determining the threshold isolation curve is similar to that for determining the threshold alarm curve. For each time point on the threshold change curve, the threshold corresponding to that time point is added to a second preset value to obtain the value of the threshold isolation curve at that time point. For example, at time point t, the value of the threshold change curve is T(t), and the second preset value is 50%. Then, the value of the threshold isolation curve at time point t, I(t), can be expressed as I(t) = T(t) * (1 + 50%). By performing this operation on all time points on the threshold change curve, the threshold isolation curve is obtained.

[0137] S503. If it is determined that there is an abnormal virtual machine based on the target performance data and threshold change curve, then determine the fault response strategy for the abnormal virtual machine based on the target performance data, threshold change curve, threshold alarm curve, and threshold isolation curve.

[0138] Once the target performance data is obtained, it is compared with the threshold change curve. If the target performance data exceeds the range of the threshold change curve, it indicates the presence of an abnormal virtual machine. Then, the target performance data is further compared with the threshold alarm curve and the threshold isolation curve to determine the severity of the anomaly and the corresponding fault response strategy.

[0139] The comparison process between the target performance data and each curve can be performed point-by-point. For the target performance data at each time point, determine its relationship with the values ​​of the threshold change curve, threshold alarm curve, and threshold isolation curve at that time point. If the target performance data is between the threshold change curve and the threshold alarm curve, it indicates that the anomaly is relatively minor and may only be a temporary performance fluctuation; in this case, proceed to step S504. If the target performance data is between the threshold alarm curve and the threshold isolation curve, it indicates that the anomaly is more serious and may have a significant impact on the normal operation of the storage system; in this case, proceed to step S505.

[0140] During the comparison process, it is crucial to ensure that the data dimensions and timestamps of the target performance data, threshold change curves, threshold alarm curves, and threshold isolation curves are consistent. Inconsistent timestamps may lead to inaccurate comparison results. Data alignment can be used to organize and match data from different data sources according to the same timestamp to guarantee comparison accuracy.

[0141] S504, Output alarm notification information.

[0142] When the target performance data is located between the threshold change curve and the threshold alarm curve, alarm prompts can be output in various ways.

[0143] For example, alarm messages can be sent to the storage system administrator's terminal device. Alarm information can be sent to the administrator via SMS, email, or instant messaging tools. The alarm message can include detailed information such as the identifier of the abnormal virtual machine, the abnormal performance indicators, the time the abnormality occurred, and the approximate severity of the abnormality, so that the administrator can understand the situation promptly.

[0144] Alternatively, an alarm window can pop up on the storage system's monitoring interface to prominently alert the administrator. The alarm window can display key information about the anomaly and provide links or buttons for further detailed information. Administrators can click these links or buttons to view detailed performance data and historical records of the abnormal virtual machine for more in-depth analysis and handling.

[0145] Alternatively, different levels of alarm sounds or alerts can be set, issuing different sound signals according to the severity of the anomaly. For example, a softer alert can be issued for relatively minor anomalies, while a sharp alarm can be issued for more serious anomalies to attract the administrator's attention.

[0146] S505: Isolate abnormal virtual machines and hot-migrate them to other host machines.

[0147] When the target performance data is between the threshold alarm curve and the threshold isolation curve, the abnormal virtual machine can be isolated to prevent it from having a greater impact on other parts of the storage system.

[0148] Isolation can be performed at both the network and resource levels. At the network level, firewalls or virtual network devices can be used to restrict communication between abnormal virtual machines and other virtual machines and network devices. For example, the network access permissions of an abnormal virtual machine can be set to allow communication only with a specific management network, prohibiting it from interacting with the business network.

[0149] At the resource level, the use of host machine resources by abnormal virtual machines can be restricted. For example, their CPU usage, memory allocation, and disk I / O bandwidth can be limited to prevent abnormal virtual machines from excessively consuming resources and affecting the normal operation of other virtual machines.

[0150] Meanwhile, to ensure business continuity, faulty virtual machines can be hot-migrated to other host machines. Hot migration refers to moving a virtual machine from one host machine to another without interrupting its operation. During the migration process, it is necessary to ensure that the virtual machine's state and data can be completely saved and restored.

[0151] Optionally, a prompt message can be output to indicate the presence of a seriously abnormal virtual machine.

[0152] The method provided in this application determines the threshold alarm curve and threshold isolation curve based on the threshold change curve and different preset values, then compares the target performance data with these curves to determine the degree of anomaly, and takes targeted fault response strategies such as outputting alarm prompts or isolating and hot-migrating abnormal virtual machines based on the determination results. By constructing different levels of performance anomaly judgment standards, the method can accurately locate anomalies and match corresponding handling measures, thereby more accurately and efficiently discovering and handling virtual machine anomalies in the storage system, ensuring the stable operation of the storage system, and reducing the impact of anomalies on business.

[0153] For example, Figure 6 This is a schematic diagram of a monitoring system provided in an embodiment of this application. Figure 6 As shown, the monitoring system includes: an indicator acquisition module, an artificial intelligence (AI) analysis module, and a fault location module.

[0154] The monitoring system is connected to a storage system, which can be, for example, the aforementioned RoCE-SAN-based storage system. The RoCE-SAN storage system includes cloud-based host machines, each containing virtual machines, such as… Figure 6 The host machine shown includes virtual machines KVM1 and KVM2. The host machine is connected to the storage device via a RoCE switch, which may be, for example, a... Figure 6 The centralized disk drive shown can also be any other storage device that can be used in a storage system. The storage device may include multiple LUNs, such as... Figure 6 As shown, the storage device includes LUN1 and LUN2.

[0155] The monitoring system's metrics acquisition module can periodically collect static configuration data (i.e., the storage system configuration data mentioned in the aforementioned method embodiments) from the RoCE-SAN storage system. For example, it can collect this configuration data from the RoCE-SAN storage system's configuration database. Here, Server ID is the host identifier, VM ID is the virtual machine identifier, LUN ID is the LUN identifier, and Stor ID is the storage device identifier. Furthermore, the metrics acquisition module can periodically collect dynamic performance data of the storage devices during the operation of the RoCE-SAN storage system (i.e., the storage device performance data mentioned in the aforementioned method embodiments, such as IOPS, throughput, latency, and CPU utilization).

[0156] After data collection is completed, a full-process relationship mapping table (i.e., the process relationship mapping table mentioned in the aforementioned method embodiment) can be generated based on the collected storage system configuration data, so as to obtain the mapping relationship between virtual machines and LUNs from the process relationship mapping table in the future.

[0157] The collected performance data of the storage devices (i.e., the historical performance data mentioned above) is input into the AI ​​analysis module. The AI ​​analysis module, for example, can use the pre-trained time-series feature extraction model to perform feature analysis on the storage device performance data, extract the periodic time-series features of the performance data, and determine the threshold change curves corresponding to each performance data point based on these time-series features. Examples include IOPS threshold change curves, throughput threshold change curves, latency threshold change curves, and CPU utilization threshold change curves.

[0158] During the operation of the RoCE-SAN storage system, the fault location module can perform anomaly monitoring based on real-time collected performance data of the storage devices and threshold change curves of the performance data. If target performance data exceeds the threshold change curve, the abnormal virtual machine can be located based on the mapping relationship between virtual machines and LUNs and the hierarchical response strategy, and the corresponding fault response strategy can be triggered.

[0159] Figure 7 This is a schematic diagram of a monitoring device provided in an embodiment of this application. Figure 7 As shown, the monitoring device may include: a processing module 11, an acquisition module 12, and a control module 13.

[0160] Processing module 11 is used to determine the mapping relationship between storage logical unit (LUN) and virtual machine based on the configuration data of the storage system.

[0161] The acquisition module 12 is used to acquire LUN performance data during the operation of the storage system. The performance data includes at least one of the following: input / output operation count (IOPS), throughput, latency, and processor utilization.

[0162] The control module 13 is used to determine the abnormal virtual machine based on the LUN and mapping relationship corresponding to the target performance data when there is target performance data outside the preset performance range, and to trigger the fault response strategy.

[0163] Optionally, processing module 11 is specifically used to determine the data processing flow chain of the host, virtual machine, storage device, and LUN in the storage system based on the configuration data. Based on the data processing flow chain, the identifier of the host, the identifier of the virtual machine, the identifier of the storage device, and the identifier of the LUN, a flow relationship mapping table is generated. The flow relationship mapping table is used to represent the mapping relationship between LUN and virtual machine.

[0164] Specifically, the acquisition module 12 is also used to acquire historical performance data of LUNs within a target time period. The processing module 11 also uses periodic time-series characteristics for each historical performance data to be extracted, and these periodic time-series characteristics are related to the business cycle of the storage system. Based on the periodic time-series characteristics, a preset performance range is determined, and the preset performance ranges are different for different business cycles.

[0165] Optionally, the processing module 11 is also used to extract periodic time-series features from historical performance data through a pre-trained time-series feature extraction model.

[0166] Optionally, processing module 11 is specifically used to generate a threshold change curve for performance data based on periodic time-series characteristics and threshold generation rules. A preset performance range is then determined based on the threshold change curve.

[0167] Specifically, the acquisition module 12 is also used to acquire incremental data of historical performance data of the LUN. The processing module 11 is also used to fine-tune the preset performance range based on the incremental data to obtain the updated preset performance range.

[0168] Optionally, the processing module 11 is also used to generate at least two levels of fault response threshold curves based on the threshold change curves. The threshold change curves and the at least two levels of fault response threshold curves are used to determine the abnormality level of the abnormal virtual machine.

[0169] Optionally, when the fault response threshold curve includes a threshold alarm curve and a threshold isolation curve, the processing module 11 is specifically used to determine the threshold alarm curve based on the threshold change curve and a first preset value, wherein the first preset value is greater than zero. It also determines the threshold isolation curve based on the threshold change curve and a second preset value, wherein the second preset value is greater than the first preset value. The control module 13 is specifically used to output an alarm message if the target performance data is located between the threshold change curve and the threshold alarm curve. If the target performance data is located between the threshold alarm curve and the threshold isolation curve, it isolates the abnormal virtual machine and hot-migrates the abnormal virtual machine to another host machine.

[0170] The monitoring device provided in this application embodiment can execute the monitoring method in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0171] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device is used to execute the aforementioned monitoring method, and may be, for example, the aforementioned data processing system. Figure 8 As shown, the electronic device 800 may include at least one processor 801, a memory 802, and a communication interface 803.

[0172] The memory 802 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0173] The memory 802 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0174] The processor 801 is used to execute computer execution instructions stored in the memory 802 to implement the method described in the foregoing method embodiments. The processor 801 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0175] The processor 801 can communicate and interact with external devices through the communication interface 803. These external devices can be, for example, the electronic devices used by staff, as mentioned earlier. In specific implementations, if the communication interface 803, memory 802, and processor 801 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0176] Optionally, in a specific implementation, if the communication interface 803, memory 802, and processor 801 are integrated on a single chip, then the communication interface 803, memory 802, and processor 801 can communicate through an internal interface.

[0177] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0178] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of a computing device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the computing device to implement the above-described monitoring method.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A monitoring method, characterized in that, The method includes: Based on the storage system configuration data, determine the mapping relationship between storage logical units (LUNs) and virtual machines; During the operation of the storage system, the performance data of the LUN is acquired, and the performance data includes at least one of the following: input / output operation count (IOPS), throughput, latency, and processor utilization. If target performance data exists outside the preset performance range, the abnormal virtual machine is determined based on the LUN corresponding to the target performance data and the mapping relationship, and a fault response strategy is triggered.

2. The method according to claim 1, characterized in that, Determining the mapping relationship between LUNs and virtual machines based on the storage system configuration data includes: Based on the configuration data, the data processing flow chain of the host machine, the virtual machine, the storage device, and the LUN in the storage system is determined; Based on the data processing flow chain, the host machine identifier, the virtual machine identifier, the storage device identifier, and the LUN identifier, a flow relationship mapping table is generated. The flow relationship mapping table is used to represent the mapping relationship between the LUN and the virtual machine.

3. The method according to claim 1, characterized in that, Also includes: Obtain historical performance data of the LUN within the target time period; Extract the periodic time-series features of the historical performance data, wherein the periodic time-series features are related to the business cycle of the storage system; Based on the periodic time sequence characteristics, the preset performance range is determined, and the preset performance range is different for different business cycles.

4. The method according to claim 3, characterized in that, The extraction of periodic time-series features from the historical performance data includes: The periodic time-series features are extracted from the historical performance data using a pre-trained time-series feature extraction model.

5. The method according to claim 3, characterized in that, Determining the preset performance range based on the periodic time characteristics includes: Based on the periodic time-series characteristics and threshold generation rules, a threshold change curve for the performance data is generated; The preset performance range is determined based on the threshold change curve.

6. The method according to claim 3, characterized in that, Also includes: Obtain incremental data of the historical performance data of the LUN; The preset performance range is fine-tuned based on the incremental data to obtain the updated preset performance range.

7. The method according to claim 5, characterized in that, Also includes: Based on the threshold change curve, at least two levels of fault response threshold curves are generated. The threshold change curve and the at least two levels of the fault response threshold curves are used to determine the abnormality level of the abnormal virtual machine.

8. The method according to claim 7, characterized in that, The fault response threshold curve includes a threshold alarm curve and a threshold isolation curve. The generation of at least two levels of fault response threshold curves based on the threshold change curve includes: The threshold alarm curve is determined based on the threshold change curve and the first preset value, wherein the first preset value is greater than zero; The threshold isolation curve is determined based on the threshold change curve and the second preset value, wherein the second preset value is greater than the first preset value; The fault response triggering strategy includes: If the target performance data is located between the threshold change curve and the threshold alarm curve, an alarm message will be output. If the target performance data is located between the threshold alarm curve and the threshold isolation curve, then the abnormal virtual machine is isolated and hot-migrated to another host machine.

9. A monitoring device, characterized in that, The device includes: The processing module is used to determine the mapping relationship between logical storage units (LUNs) and virtual machines based on the configuration data of the storage system. The acquisition module is used to acquire performance data of the LUN during the operation of the storage system. The performance data includes at least one of the following: input / output operation count (IOPS), throughput, latency, and processor utilization. The control module is used to determine the abnormal virtual machine based on the LUN corresponding to the target performance data and the mapping relationship when target performance data exists outside the preset performance range, and to trigger a fault response strategy.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images