A general method for monitoring abnormal block storage and file storage services in cloud computing

Abstract

The present invention relates to a method for monitoring abnormal block storage and file storage services in general use in cloud computing. The method comprises: creating a cloud host ECS and a data disk; creating an alarm rule; logging in to the ECS and installing relevant components, issuing read and write services to the storage, running a vdbench script for storage testing, and replacing the result file of the vdbench operation with the vdbench_file parameter of the monitoring script; executing the monitoring script and setting the maximum service delay to the timeout delay; monitoring the service execution process, when the service delay is greater than the timeout delay or other errors occur, the script uses the stress tool to modify the indicator parameters monitored by the alarm rule to achieve fault simulation, thereby triggering the alarm rule. The method improves the system fault response speed through automated testing and alarm mechanisms, can more effectively monitor and use cloud resources, and ensures efficient use of resources. At the same time, the method can discover and solve potential problems in advance through simulated faults, thereby enhancing the stability and reliability of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of storage service anomaly monitoring methods, and in particular, relates to a block storage and file storage service anomaly monitoring method and a monitoring system commonly used in cloud computing. Background Art

[0002] In the field of cloud computing, monitoring of storage abnormal IO is an important research direction, which involves the detection and processing of problems such as business interruption, EIO (device input / output error) and excessive latency. Currently used storage monitoring mainly includes the following categories: (1) Current performance monitoring and management: One of the key use cases of cloud computing monitoring is performance monitoring and management, the purpose of which is to understand the performance of applications and services in real time, and to promptly discover and solve potential performance problems; (2) Security monitoring: Security monitoring can help enterprises and organizations monitor the security of cloud computing environments, promptly discover and respond to security threats, and help solve storage-related security issues; (3) Storage space monitoring: Cloud monitoring services can promptly obtain information about instances with insufficient storage space, which helps avoid data being unable to be written and business impacts caused by insufficient storage space.

[0003] In summary, the current monitoring of storage services is mostly aimed at business performance and space under normal circumstances. However, there is still a lack of reliable and effective monitoring methods for abnormal situations of storage services, such as temporary IO drop to 0, EIO or excessive latency. Therefore, it is difficult to meet users' multi-faceted monitoring needs for storage services. Summary of the invention

[0004] In order to overcome the defect that the existing storage monitoring method cannot effectively monitor the storage business when an abnormal situation occurs, the present invention proposes a new cloud computing universal block storage and file storage business abnormality monitoring method. Through this method, the business abnormalities of block storage and file storage can be monitored and reported in time, so that the fault can be handled in time, which helps to improve the operation and maintenance efficiency.

[0005] The design key points of the method and system of the present invention include:

[0006] 1. Performance threshold setting and response mechanism: The present invention covers a system and method for automatically triggering response measures (such as memory compression) based on a preset performance threshold (such as memory usage exceeding 60%).

[0007] 2. Fault simulation and stress testing: The present invention includes a method of using a specific tool (such as stress) to pressurize the memory of the cloud host to simulate faults and perform stress testing.

[0008] 3. Real-time performance monitoring and log recording: The present invention includes a system for real-time monitoring of cloud host performance and recording test results, including a mechanism for printing results every 1 second. The system of the present invention is particularly sensitive to flash interruption monitoring of services and is more friendly to services with high requirements for storage service continuity.

[0009] 4. Timeout setting and anomaly detection: The present invention covers a method for setting a maximum service delay and detecting a delay exceeding the limit or anomaly (such as service drop to 0 or ERR io). This method can be applied to monitoring of various storage types.

[0010] 5. Automated script execution: The present invention provides a method for executing automated scripts for performance testing and monitoring, where the script can automatically adjust behavior based on preset parameters (such as timeout period).

[0011] Specifically, the present invention provides a method for monitoring abnormal block storage and file storage services in cloud computing. Figure 8 As shown, the method comprises the following steps:

[0012] S1. Create a cloud host ECS on the cloud platform and create a data disk at the same time;

[0013] S2. Create an alarm rule on the cloud platform for the cloud host ECS created in step S1;

[0014] S3. Log in to the cloud host ECS created in step S1 and install relevant components, issue read and write services to block storage or file storage, run the vdbench script for storage testing, and replace the result file of the vdbench operation with the vdbench_file parameter of the monitoring script;

[0015] S4. Set the maximum service delay, execute the monitoring script and set the maximum service delay to the timeout delay;

[0016] S5. Continuously monitor the business execution process. When the delay in executing the business is greater than the timeout delay set in step S4 or other errors occur, the script uses the stress tool to modify the indicator parameters monitored by the alarm rule to simulate the fault, thereby triggering the alarm rule set in step S2.

[0017] Furthermore, the indicators monitored by the alarm rules described in step S2 of the method for monitoring abnormal block storage and file storage services in cloud computing of the present invention include but are not limited to: virtual memory usage, CPU usage, and average CPU load;

[0018] The parameters of the monitoring indicators can be modified through the stress tool.

[0019] Furthermore, the related components described in step S3 of the method for monitoring abnormal block storage and file storage services in cloud computing of the present invention include but are not limited to: stress component, bc calculator component and monitoring execution script, vdbench tool, vdbench running script and vdbench configuration file.

[0020] Furthermore, in step S3 of the method for monitoring abnormal block storage and file storage services in cloud computing of the present invention, the read and write services are sent to the block storage or file storage, and the sent read and write services need to ensure that vdbench prints the results once every 1 second.

[0021] Optionally, in step S3 of the method for monitoring abnormal block storage and file storage services in cloud computing of the present invention, the read and write services for block storage or file storage are issued, and the issued read and write services need to ensure that vdbench prints the results every 2 seconds or every 5 seconds.

[0022] Furthermore, other errors described in step S5 of the method for monitoring abnormal block storage and file storage services in cloud computing of the present invention include but are not limited to: IO service drop to 0, ERR io.

[0023] Preferably, the script described in step S5 of the method for monitoring abnormal block storage and file storage services commonly used in cloud computing of the present invention uses the stress tool to modify the indicator parameters monitored by the alarm rule to implement fault simulation, thereby triggering the alarm rule set in step S2. When the indicator monitored by the alarm rule is the virtual memory usage rate, the script uses the stress tool to pressurize the memory of the ECS, causing the ECS memory usage ratio to increase abnormally, thereby triggering the virtual memory usage rate alarm rule.

[0024] On the other hand, the present invention also provides a block storage and file storage business anomaly monitoring system common to cloud computing. When the monitoring system is running, the steps of the above-mentioned block storage and file storage business anomaly monitoring method common to cloud computing are implemented.

[0025] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned cloud computing general block storage and file storage service anomaly monitoring method are implemented.

[0026] In summary, the block storage and file storage service anomaly monitoring method for cloud computing of the present invention has the following advantages:

[0027] (1) This method integrates cloud computing resource management, performance monitoring, automated testing, and alarm systems, providing a new solution for cloud resource performance management and fault simulation. Through automated testing and alarm mechanisms, this method can more effectively monitor and use cloud resources, ensuring efficient use of resources.

[0028] (2) By simulating failures and performance bottlenecks, this method can detect and solve potential problems in advance, thereby enhancing the stability and reliability of the system. In addition, during the execution of this method, it will not affect the use of other cloud resources, but will only affect the current cloud host ECS, thus ensuring the independence and security of cloud resources.

[0029] (3) The automated monitoring and alarm process used in this method reduces manual inspection and intervention, lowers labor costs, and the automated monitoring and alarm system can quickly respond to performance issues and take timely measures, thereby improving response speed and reducing system failure time.

[0030] (4) This method adopts data-driven decision-making. By collecting and analyzing performance data, more reasonable resource management and expansion decisions can be made based on actual data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the following drawings are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 The present invention provides an operation flow chart of a storage service abnormality monitoring method according to an embodiment of the present invention.

[0033] Figure 2 This is an example diagram of the operation of configuring alarm rules on a cloud platform in the method of the present invention.

[0034] Figure 3-Figure 5 This is an operation example diagram of running a vdbench script to perform a storage test in the method of the present invention, and replacing the result file of the vdbench operation with the vdbench_file parameter in the monitoring script.

[0035] Figure 6 This is an example diagram of the operation of running a storage anomaly monitoring script in the method of the present invention. When the service delay is greater than the timeout delay, the script uses the stress tool to pressurize the memory of the ECS.

[0036] Figure 7 This is an example diagram of the operation of a user viewing a corresponding alarm on the monitoring alarm side in the method of the present invention.

[0037] Figure 8 The figure is a flow chart of the overall implementation of the method of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. The present invention can also be implemented or applied through other different specific implementation methods. The details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0039] At the same time, it should be understood that the protection scope of the present invention is not limited to the specific embodiments described below; it should also be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the protection scope of the present invention.

[0040] Embodiment: A method for monitoring abnormal block storage and file storage services in cloud computing

[0041] The operation process of the storage service abnormality monitoring method provided in this embodiment is as follows: Figure 1 As shown, the following steps are included:

[0042] 1. Create a cloud host ECS on the cloud platform, and create a data disk when creating it.

[0043] 2. If Figure 2 As shown in the figure, create an alarm rule for the cloud host ECS created in step 1 on the cloud platform, select "Virtual Memory Usage" as the monitored indicator, and configure it to generate an alarm when the virtual memory usage exceeds 60%.

[0044] In addition, users can also select monitoring indicators such as "virtual memory usage", "CPU usage", "CPU average load" and so on according to their needs. The parameter modification of monitoring indicators is achieved through the stress tool.

[0045] 3. Log in to the cloud host ECS created in step 1, install the stress component, bc calculator component and monitoring execution script, vdbench tool, vdbench running script and vdbench configuration file.

[0046] 4. Run the vdbench script to perform storage testing and issue 4k read and write services to block storage or file storage (you can also adjust the service model according to your own needs). This read and write service is similar to the probe function and is used to detect storage service anomalies. In addition, the issued service must ensure that vdbench prints results every 1 second (you can also modify the vdbench configuration file according to your own needs and adjust it to print results every 2 seconds or every 5 seconds to reduce the sensitivity of service monitoring).

[0047] 5. Such as Figure 3-Figure 5 As shown in the figure, replace the result file of vdbench running with the vdbench_file parameter in the monitoring script.

[0048] 6. Set a timeout delay, which is the maximum delay acceptable to the business, in ms.

[0049] 7. Execute the monitoring script, and the timeout delay set in step 6 will be included when the script is executed.

[0050] 8.Continuously monitor the business execution process, such as Figure 6 As shown in the figure, when the delay of executing the business is greater than the timeout delay set in step 6 or errors such as business drop to 0 or ERR io occur, the script will use the stress tool to pressurize the ECS memory, causing the ECS memory usage ratio to increase abnormally, thereby triggering the alarm rule set in step 2, such as Figure 7 As shown, the user can now view the corresponding alarm on the monitoring alarm side.

[0051] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make some changes or modifications to the technical contents disclosed above without departing from the scope of the technical solution of the present invention to obtain equivalent embodiments of equivalent changes. However, any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for monitoring abnormal block storage and file storage services in cloud computing, characterized in that: The method comprises: S1. Create a cloud host ECS on the cloud platform and create a data disk at the same time; S2. Create an alarm rule on the cloud platform for the cloud host ECS created in step S1; S3. Log in to the cloud host ECS created in step S1 and install relevant components, issue read and write services to block storage or file storage, run the vdbench script for storage testing, and replace the result file of the vdbench operation with the vdbench_file parameter of the monitoring script; the relevant components include: stress component, bc calculator component and monitoring execution script, vdbench tool, vdbench operation script and vdbench configuration file; S4. Set the maximum service delay, execute the monitoring script and set the maximum service delay to the timeout delay; S5. Continuously monitor the business execution process. When the delay in executing the business is greater than the timeout delay set in step S4 or other errors occur, the script uses the stress tool to modify the indicator parameters monitored by the alarm rule to simulate the fault, thereby triggering the alarm rule set in step S2.

2. The method for monitoring abnormal block storage and file storage services for cloud computing according to claim 1, characterized in that: The indicators monitored by the alarm rule described in step S2 include: virtual memory usage, CPU usage, and CPU average load; The indicator parameters monitored by the alarm rule can be modified through the stress tool.

3. The method for monitoring abnormality of block storage and file storage services commonly used in cloud computing according to claim 1, characterized in that: The read and write services are sent to the block storage or file storage in step S3. The sent read and write services must ensure that vdbench prints the results once every 1 second.

4. The method for monitoring abnormal block storage and file storage services for cloud computing according to claim 1, characterized in that: The read and write services are sent to the block storage or file storage in step S3. The sent read and write services must ensure that vdbench prints the results every 2 seconds or every 5 seconds.

5. The method for monitoring abnormality of block storage and file storage services commonly used in cloud computing according to claim 1, characterized in that: Other errors described in step S5 include: IO business falls to 0, ERR io.

6. The method for monitoring abnormality of block storage and file storage services commonly used in cloud computing according to claim 1, characterized in that: The script described in step S5 uses the stress tool to modify the indicator parameters monitored by the alarm rule to implement fault simulation, thereby triggering the alarm rule set in step S2. When the indicator monitored by the alarm rule is the virtual memory usage, the script uses the stress tool to pressurize the ECS memory, causing the ECS memory usage ratio to increase abnormally, thereby triggering the virtual memory usage alarm rule.

7. A general-purpose block storage and file storage service anomaly monitoring system for cloud computing, characterized in that: When the monitoring system is running, the steps of the method for monitoring abnormalities of block storage and file storage services commonly used in cloud computing according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the steps of the method for monitoring abnormalities of block storage and file storage services commonly used in cloud computing as described in any one of claims 1 to 6.

Images