Block storage management method, device, equipment and storage medium
By monitoring and adjusting the storage volume performance parameters of the distributed block storage system, the problem that the distributed block storage system cannot ensure the normal operation of high traffic demand services is solved, and the system's long-term operation reliability and refined management capabilities are improved.
Patent Information
- Application Number
- CN202210183722.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-02-25
AI Technical Summary
The distributed block storage system cannot effectively ensure the normal operation of high-traffic demand services, resulting in low reliability of long-term operation within its capabilities and insufficient refined management of high-traffic demand services.
By monitoring the traffic of the target storage volume, obtaining the current overall traffic of the storage pool according to the first threshold condition, performing overload analysis, generating a second threshold condition, and adjusting the performance parameters of the storage volume according to this condition to achieve dynamic expansion and ensure that the system is in Its capability is reliable and reliable operation within its long-term scope.
It improves the long-term operation reliability of the distributed block storage system, overcomes the problem of inability to ensure the normal operation of high traffic demand services, strengthens the refined management of the distributed block system, and avoids the impact of internal traffic on external business performance.
Smart Images

Figure CN114546290B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the financial field, the big data technology field or the computer technology field, and more specifically, to a block storage management method, apparatus, device, storage medium and program product. Background Art
[0002] Distributed block storage is a standard infrastructure cloud storage backend and is widely used in the provision of virtual machine storage resources. With the deepening of architecture transformation, the diversity and complexity of the services carried by distributed block storage systems are gradually increasing. Therefore, it is more important to ensure the quality of key business services and the ability of the system to provide stable services. However, in related technologies, the strategy adopted by distributed block storage is to ensure the minimum performance and limit the maximum performance, which cannot effectively ensure the normal operation of high-traffic demand services. Summary of the invention
[0003] In view of the above problems, the present disclosure provides a block storage management method, apparatus, electronic device, storage medium and program product.
[0004] One aspect of the present disclosure provides a block storage management method, including: when it is monitored that the traffic of a target storage volume meets a first threshold condition, obtaining the current overall traffic of a storage pool from a first data table, wherein the first threshold condition is determined based on the performance parameter of the target storage volume; performing overload analysis according to the first threshold condition and the current overall traffic to obtain an analysis result; when the analysis result indicates performance redundancy, generating a second threshold condition based on the first threshold condition; and adjusting the performance parameter of the target storage volume according to the second threshold condition.
[0005] According to an embodiment of the present disclosure, the method further includes: setting a first scheduled task; in response to triggering the first scheduled task, monitoring the overall flow of the storage pool; and recording the overall flow of the storage pool in the first data table.
[0006] According to an embodiment of the present disclosure, the overload analysis is performed based on the first threshold condition and the current overall flow to obtain an analysis result, including: calculating the predicted overall flow based on the current overall flow and the first threshold condition; analyzing whether an overload trend alarm will be triggered based on the predicted overall flow; in the case where it is determined that the predicted overall flow will trigger the overload trend alarm, obtaining the analysis result indicating insufficient performance; and in the case where it is determined that the predicted overall flow will not trigger the overload trend alarm, obtaining the analysis result indicating performance redundancy.
[0007] According to an embodiment of the present disclosure, the above method also includes: triggering the above overload trend alarm when the ratio of the above predicted overall flow and the previous overall flow is greater than or equal to a preset ratio, wherein the above previous overall flow is recorded in the above first data table, and the above previous overall flow includes the overall flow of the above storage pool monitored when the above second timing task was triggered last; or triggering the above overload trend alarm when the above predicted overall flow meets a preset threshold condition.
[0008] According to an embodiment of the present disclosure, the method further includes: after adjusting the performance parameter of the storage volume, recording the number of times the performance parameter of the storage volume has been adjusted in the second data table.
[0009] According to an embodiment of the present disclosure, the method further includes: obtaining the number of times the performance parameters of the target storage volume have been adjusted; when the number of times the performance parameters of the target storage volume have been adjusted is greater than or equal to a preset number of times, sending a manual intervention request to an operation and maintenance terminal device; and when the number of times the performance parameters of the target storage volume have been adjusted is less than the preset number of times, adjusting the performance parameters of the target storage volume according to the second threshold condition.
[0010] According to an embodiment of the present disclosure, the method further includes: setting a second scheduled task; in response to triggering the second scheduled task, calling a preset script to monitor the traffic of each storage volume.
[0011] According to an embodiment of the present disclosure, the method further includes: creating a storage volume from the storage pool in response to a virtual machine deployment request; calling a virtual machine component to build a virtual machine based on the storage volume; and configuring performance parameters of the storage volume according to initial threshold conditions.
[0012] According to an embodiment of the present disclosure, the above-mentioned virtual machine deployment request carries a traffic identifier; the above-mentioned method also includes: when the above-mentioned traffic identifier is represented as a standard traffic identifier, configuring the performance parameters of the above-mentioned storage volume according to the above-mentioned initial threshold conditions; when the above-mentioned traffic identifier is represented as a high traffic identifier, configuring the performance parameters of the above-mentioned storage volume according to a preset strategy.
[0013] Another aspect of the present disclosure provides a block storage management device, including: a first acquisition module, used to acquire the current overall flow of the storage pool from a first data table when the flow of the target storage volume is monitored to meet a first threshold condition, wherein the first threshold condition is determined based on the performance parameters of the target storage volume; an analysis module, used to perform overload analysis according to the first threshold condition and the current overall flow to obtain an analysis result; a generation module, used to generate a second threshold condition based on the first threshold condition when the analysis result indicates performance redundancy; and an adjustment module, used to adjust the performance parameters of the target storage volume according to the second threshold condition.
[0014] Another aspect of the present disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors execute the block storage management method.
[0015] Another aspect of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the block storage management method.
[0016] Another aspect of the present disclosure provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when executed.
[0017] According to the embodiments of the present disclosure, by performing an overload analysis on the overall traffic of the storage pool and adjusting the performance parameters of the target storage volume when the analysis result indicates performance redundancy, dynamic expansion of the target storage volume is achieved, which can ensure that the distributed block storage system can operate reliably for a long time within its capabilities, at least partially overcome the technical problem in the related art that the distributed block storage cannot guarantee the normal operation of high-traffic demand services, effectively improve the reliability of the long-term operation of the distributed block storage, and strengthen the refined management of the distributed block system. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0019] Figure 1 An exemplary system architecture to which the block storage management method and apparatus according to an embodiment of the present disclosure can be applied is schematically shown;
[0020] Figure 2 A flowchart of a block storage management method according to an embodiment of the present disclosure is schematically shown;
[0021] Figure 3 A flowchart of block storage management according to an embodiment of the present disclosure is schematically shown;
[0022] Figure 4 A block diagram schematically shows a block storage management device according to an embodiment of the present disclosure; and
[0023] Figure 5 A block diagram of an electronic device suitable for implementing a block storage management method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0027] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0028] It should be noted that the block storage management method, device, electronic device, storage medium and program product disclosed in the present invention can be used in the financial field, big data technology field or computer field, and can also be used in any field other than the financial field, big data technology field or computer field. The present disclosure does not limit the application field of the block storage management method, device, electronic device, storage medium and program product.
[0029] In the technical solutions disclosed in this disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved are in compliance with the relevant laws and regulations, necessary confidentiality measures are taken, and they do not violate public order and good morals. In the technical solutions disclosed in this disclosure, the user's authorization or consent is obtained before obtaining or collecting user personal information.
[0030] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable control device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.
[0031] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memories, such as random access memories (RAMs) or flash memory; and / or wired / wireless communication links.
[0032] Distributed block storage is a standard infrastructure cloud storage backend and is widely used in the provision of virtual machine storage resources. With the deepening of architecture transformation, the diversity and complexity of the services carried by distributed block storage systems have gradually increased. Therefore, it is more important to ensure the quality of key business services and the ability of the system to provide stable services. To this end, the business community has proposed storage service quality, namely storage QoS policy (customer service quality, Quality of Service). However, the QoS policy provided by general distributed block storage systems is mainly used to support users to configure QoS policies for storage pools and storage volumes, guarantee the minimum performance, limit the maximum performance, and cannot guarantee the normal operation of high-traffic demand services.
[0033] Specifically, the distributed block storage in the related art has low reliability in long-term operation within its capabilities. Generally, in addition to the read / write operations of the client, there are a large number of read / write operations inside the distributed block storage, such as regular scrub, recovery and repair after failures, etc. In the event of abnormality / failure recovery, all of these operations have a significant impact on storage performance, thereby affecting the long-term reliability of the distributed block storage. The distributed block storage in the related art lacks refined management of high-traffic demand services. For high-traffic demand services that trigger the traffic limit, there is generally no reservation of IOPS (Input / Output Operations Per Second) and bandwidth capacity. For example, in a private cloud, even if the storage pool capacity is sufficient, it cannot cope with the normal operation of a sudden single high-traffic demand service.
[0034] In view of this, the embodiments of the present disclosure provide a block storage management method, device, electronic device, storage medium and program product. The method includes obtaining the current overall flow of the storage pool from a first data table when the flow of the target storage volume is monitored to meet a first threshold condition, wherein the first threshold condition is determined based on the performance parameters of the target storage volume; performing overload analysis based on the first threshold condition and the current overall flow to obtain an analysis result; generating a second threshold condition based on the first threshold condition when the analysis result indicates performance redundancy; and adjusting the performance parameters of the target storage volume according to the second threshold condition.
[0035] Figure 1 The following schematically illustrates an exemplary system architecture to which the block storage management method and apparatus according to an embodiment of the present disclosure can be applied. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0036] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0037] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).
[0038] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.
[0039] The server 105 may include multiple nodes, and the operation of each node may be understood as a software function implementation, providing various functions of a distributed storage system. The server 105 may include a storage node, a computing node, and a control node. Specifically, an overload control framework may be deployed on the storage node, a preset monitoring script may be deployed on the computing node, and a dynamic QoS policy and a static QoS policy may be deployed on the control node.
[0040] It should be noted that the block storage management method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the block storage management device provided in the embodiment of the present disclosure can generally be set in the server 105. The block storage management method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the block storage management device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the block storage management method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the block storage management apparatus provided in the embodiment of the present disclosure may also be arranged in the terminal device 101 , 102 , or 103 , or in other terminal devices different from the terminal device 101 , 102 , or 103 .
[0041] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0042] Figure 2 The flowchart of the block storage management method according to the embodiment of the present disclosure is schematically shown.
[0043] like Figure 2 As shown, the method includes operations S201 to S204. It should be noted that, unless it is clearly stated that there is an execution sequence between different operations shown in the flowchart in the embodiment of the present disclosure, or different operations have an execution sequence in technical implementation, otherwise, the execution sequence between multiple operations may not be prioritized, and multiple operations may be executed simultaneously.
[0044] In operation S201, when it is monitored that the traffic of the target storage volume meets a first threshold condition, the current overall traffic of the storage pool is obtained from a first data table, wherein the first threshold condition is determined based on a performance parameter of the target storage volume.
[0045] In operation S202, an overload analysis is performed according to the first threshold condition and the current overall traffic to obtain an analysis result.
[0046] In operation S203, in a case where the analysis result indicates performance redundancy, a second threshold condition is generated based on the first threshold condition.
[0047] In operation S204, the performance parameter of the target storage volume is adjusted according to the second threshold condition.
[0048] According to an embodiment of the present disclosure, the storage pool may include a solid state drive (SSD), a mechanical hard disk (HDD), etc. When the server responds to a deployment request of a virtual machine, a storage volume may be created from the storage pool and supplied to the virtual machine. In the subsequent application process, the IOPS, bandwidth, etc. of the virtual machine may be monitored, and the storage volume associated with the monitored virtual machine may be used as the target storage volume. The performance parameters of the storage volume and the traffic of the storage volume may include, but are not limited to, IOPS and bandwidth.
[0049] According to an embodiment of the present disclosure, when a storage volume is created from a storage pool, the maximum flow value set for the storage volume can be used as a first threshold. The first threshold condition can be that the monitored flow of the target storage volume is equal to the first threshold. When it is predicted that the overall flow will not trigger an overload trend alarm, the second threshold can be obtained based on the first threshold. For example, the second threshold can be obtained by directly adding a preset value on the basis of the first threshold. For another example, the second threshold can be obtained by increasing a preset percentage on the basis of the first threshold. The second threshold condition can be that the monitored flow of the target storage volume is less than the second threshold.
[0050] According to an embodiment of the present disclosure, the first data table may include: the overall flow of the storage pool monitored each time. The current overall flow of the storage pool may be the overall flow of the storage pool monitored by the server during the execution of this task; the previous overall flow may be the overall flow of the storage pool monitored by the server during the execution of the previous task.
[0051] According to an embodiment of the present disclosure, the overload analysis may include analyzing whether the ratio of the predicted overall flow after pre-adjustment to the previous overall flow is greater than or equal to a preset ratio; or analyzing whether the predicted overall flow meets a preset threshold condition. When the ratio of the predicted overall flow to the previous overall flow is less than the preset ratio or the predicted overall flow does not meet the preset threshold condition, an analysis result indicating performance redundancy is obtained.
[0052] According to the embodiments of the present disclosure, by performing an overload analysis on the overall traffic of the storage pool and adjusting the performance parameters of the target storage volume when the analysis result indicates performance redundancy, dynamic expansion of the target storage volume is achieved, which can ensure that the distributed block storage system can operate reliably for a long time within its capabilities, at least partially overcome the technical problem in the related art that the distributed block storage cannot guarantee the normal operation of high-traffic demand services, effectively improve the reliability of the long-term operation of the distributed block storage, and strengthen the refined management of the distributed block system.
[0053] According to an embodiment of the present disclosure, the block storage management method further includes: creating a storage volume from a storage pool in response to a virtual machine deployment request; calling a virtual machine component to build a virtual machine based on the storage volume; and configuring performance parameters of the storage volume according to an initial threshold condition.
[0054] According to an embodiment of the present disclosure, when a server responds to a deployment request of a virtual machine, a storage volume can be created from a storage pool. The deployment request of a virtual machine can come from the issuance of a new virtual machine, the expansion of a virtual machine, new business needs, etc. A physical device, such as a physical machine, can be used to call a virtual machine component to build a virtual machine based on a newly created storage volume. After a storage volume is created from a storage pool, the performance parameters of the storage volume can be configured according to the initial threshold conditions. For example, a static QoS policy can be adopted to configure standardized QoS performance parameters for the newly created storage volume, so that the newly created storage volume has a certain IOPS and bandwidth capacity.
[0055] According to an embodiment of the present disclosure, a virtual machine deployment request carries a traffic identifier; the above-mentioned block storage management method also includes: when the traffic identifier is represented as a standard traffic identifier, configuring the performance parameters of the storage volume according to the initial threshold condition; when the traffic identifier is represented as a high traffic identifier, configuring the performance parameters of the storage volume according to a preset policy.
[0056] According to an embodiment of the present disclosure, when the traffic identifier of a storage volume is represented as a standard traffic identifier, the performance parameters of the storage volume can be configured according to the initial threshold conditions, and a maximum traffic value can be set for the storage volume as a first threshold, and the initial threshold conditions can be associated with a deployment request for a new virtual machine; when the traffic identifier is represented as a high traffic identifier, the performance parameters of the storage volume are configured according to a preset policy, and the preset policy can be to directly set performance parameters that match the high traffic identifier according to the high traffic identifier through a dynamic QoS policy, or it can be to request the operation and maintenance personnel to directly set them through the intervention of the operation and maintenance personnel.
[0057] According to an embodiment of the present disclosure, the block storage management method further includes: setting a second scheduled task; in response to triggering the second scheduled task, calling a preset script to monitor the traffic of each storage volume.
[0058] According to an embodiment of the present disclosure, a preset monitoring script may be deployed on a computing node of a physical machine to monitor the traffic of a storage volume supplied to a virtual machine, and the target storage volume may be a storage volume that needs to be monitored. Specifically, the preset monitoring script may be called to monitor the traffic and other performance parameters of each storage volume on a regular basis in response to a second scheduled task. The preset script may include, but is not limited to, a VmDiskStat monitoring script. For example, the VmDiskStat monitoring script may be called to monitor the traffic of a storage volume at the minute level.
[0059] According to an embodiment of the present disclosure, the block storage management method further includes: setting a first scheduled task; in response to triggering the first scheduled task, monitoring the overall flow of the storage pool; and recording the overall flow of the storage pool in a first data table.
[0060] According to an embodiment of the present disclosure, when the traffic of the target storage volume is monitored to meet the first threshold condition, the current overall traffic of the storage pool is obtained from the first data table. Specifically, an overload control framework can be deployed in the storage node of the physical machine. The overload control can monitor the overall traffic of the storage pool regularly by responding to the first scheduled task, and record the monitored overall traffic of the storage pool in the first data table. When it is monitored that the traffic of the target storage volume reaches the preset maximum traffic value, a dynamic expansion request can be sent to the overload control framework, so that the overload control framework can obtain the current overall traffic of the storage pool from the first data table according to the request, and analyze it according to the first threshold condition and the current overall traffic of the storage pool. While sending a dynamic expansion request to the overload control framework, a current limiting alarm can also be sent to centralized monitoring.
[0061] According to an embodiment of the present disclosure, operation S202 may include the following operations.
[0062] According to an embodiment of the present disclosure, the overload control framework may calculate based on the current overall flow obtained from the first data table and the received first threshold condition to predict the adjusted overall flow. Whether the adjusted overall flow will trigger an overload trend alarm is analyzed based on the predicted overall flow. When the overall flow is predicted to trigger an overload trend alarm, an analysis result indicating insufficient performance can be obtained; when the overall flow is predicted not to trigger an overload trend alarm, an analysis result indicating performance redundancy can be obtained, and the analysis result is sent to a preset monitoring script, and an alarm can also be sent to centralized monitoring.
[0063] According to an embodiment of the present disclosure, the above-mentioned block storage management method also includes: triggering an overload trend alarm when the ratio of the predicted overall flow and the previous overall flow is greater than or equal to a preset ratio, wherein the previous overall flow is recorded in a first data table, and the previous overall flow includes the overall flow of the storage pool monitored when the second scheduled task was triggered last; or triggering an overload trend alarm when the predicted overall flow meets a preset threshold condition.
[0064] According to an embodiment of the present disclosure, when the ratio of the predicted overall flow and the previous overall flow is greater than or equal to a preset ratio, an overload trend alarm can be triggered. Specifically, the previous overall flow can be recorded in the first data table, and the previous overall flow can be the overall flow of the storage pool obtained by the previous second scheduled task; the preset ratio can be configured according to the upper limit of the storage system performance, for example, it can be 70%, 85%, etc. of the upper limit of the storage system flow (data is only an example). When the ratio of the predicted overall flow and the previous overall flow is greater than or equal to the preset ratio, an overload trend alarm is triggered, and an alarm is sent to centralized monitoring at the same time. The performance pressure of the storage system can naturally increase with the applications and usage it carries, and when the trend alarm is triggered, the storage system still has a certain amount of performance redundancy.
[0065] According to the embodiments of the present disclosure, when the overall traffic is predicted to meet the preset threshold conditions, an overload trend alarm can also be triggered. Specifically, when the overall pressure of the storage system increases suddenly, and the performance pressure increases suddenly during the same period exceeding the preset threshold conditions, such as a sudden increase of 20% (data is only an example), an overload trend alarm will be triggered, and an alarm will be sent to centralized monitoring at the same time. This sudden increase is generally more common in IOPS and traffic anomalies after the version is put into production.
[0066] According to the embodiments of the present disclosure, the overload control framework can also send analysis results to the preset monitoring script, provide an evaluation of the overall performance redundancy of the storage system for subsequent dynamic QoS policy adjustments, and make decisions and provide feedback on whether the performance of the storage system can be dynamically increased. The overload control framework can also distinguish between external and internal services, and has the function of controlling bandwidth for internal services. When the pressure on the storage system changes, the internal service traffic can be adaptively adjusted within the upper limit of 50% (data is only an example) of the total bandwidth to avoid excessive business pressure generated within the storage system and affecting the service quality of external services.
[0067] According to the embodiments of the present disclosure, overload control can ensure that the distributed block storage system operates reliably for a long time within the system capacity, avoiding the impact of internal traffic such as distributed storage scrub, recovery and repair after failure on the performance of external services.
[0068] According to an embodiment of the present disclosure, operation S203 may include the following operations.
[0069] According to an embodiment of the present disclosure, when the preset monitoring script obtains an analysis result indicating performance redundancy, a dynamic QoS policy is triggered to perform dynamic capacity expansion. Specifically, the dynamic QoS policy determines the second threshold condition according to the first threshold condition, and the second threshold condition can be obtained by adjusting a certain proportion based on the current first threshold, for example, the second threshold condition can be obtained by adjusting 20% (data is only an example) based on the current first threshold.
[0070] According to an embodiment of the present disclosure, operation S204 may include the following operations.
[0071] According to an embodiment of the present disclosure, the dynamic QoS policy can dynamically expand the storage volume according to the second threshold condition. The dynamic QoS policy can also adjust the performance parameters of the virtual machine such as IOPS and bandwidth online through KVM QEMU.
[0072] According to an embodiment of the present disclosure, the block storage management method further includes: after adjusting the performance parameter of the storage volume, recording the number of times the performance parameter of the storage volume has been adjusted in the second data table.
[0073] According to an embodiment of the present disclosure, the above-mentioned block storage management method also includes: obtaining the number of times the performance parameters of the target storage volume have been adjusted; when the number of times the performance parameters of the target storage volume have been adjusted is greater than or equal to a preset number of times, sending a manual intervention request to the operation and maintenance terminal device; and when the number of times the performance parameters of the target storage volume have been adjusted is less than the preset number of times, adjusting the performance parameters of the target storage volume according to a second threshold condition.
[0074] According to the embodiments of the present disclosure, the dynamic QoS policy can be adaptively and dynamically adjusted up to a certain preset number of times, not unlimitedly. Specifically, the number of adjustments to the storage volume can be recorded in the second data table, and when the number of adjustments obtained is greater than or equal to the preset number, a request for manual intervention can be sent to the operation and maintenance terminal device, and when the number of adjustments is less than the preset number, the performance parameters of the target storage volume can continue to be adjusted according to the second threshold condition.
[0075] According to the embodiments of the present disclosure, the QoS policy can effectively control the service IOPS and bandwidth capacity, change the natural competition for resources by the service, guarantee the storage performance of a single or part of the service, ensure the stable operation of the storage system, and enable the upper-level service virtual machines to better share the underlying storage capacity. Through the adjustment of the dynamic QoS policy, the refined management of the high-traffic restriction of the distributed block storage system can be achieved, and the normal operation of a sudden single high-traffic demand service can be effectively guaranteed when the storage pool has performance redundancy.
[0076] Figure 3 The flowchart of block storage management according to an embodiment of the present disclosure is schematically shown.
[0077] like Figure 3 As shown, when there is a new business demand, a storage volume 303 is created from the storage pool 301, and the static QoS policy 302 configures the associated performance parameters according to the initial threshold conditions of the storage volume 303, and supplies the storage volume to the virtual machine 304. The preset monitoring script 305 will monitor the traffic of the storage volume 303 in the virtual machine 304 regularly through the corresponding second scheduled task. When it is monitored that the overall traffic of the storage volume 303 meets the first threshold, it sends an alarm to the centralized monitoring 306 and also sends a request for a dynamic QoS policy to the overload control framework 307. According to the request of the dynamic QoS policy, the overload control framework 307 obtains the current overall traffic of the storage pool 301 from the first data table, and performs overload analysis on the first threshold condition and the current overall traffic. When the analysis result indicates performance redundancy, the analysis result is sent to the preset monitoring script 305, and an alarm is also sent to the centralized monitoring 306, indicating that there is still redundant space. The preset monitoring script 306 triggers the dynamic QoS policy 308 to achieve dynamic expansion according to the analysis results. The dynamic QoS policy 308 can generate a second threshold condition according to the current first threshold condition, and adjust the performance parameters of the storage volume 303 according to the second threshold condition. The dynamic QoS policy can also adjust the performance parameters of the virtual machine 304 online according to KVM QEMU.
[0078] Figure 4 The block diagram of the block storage management device according to the embodiment of the present disclosure is schematically shown.
[0079] like Figure 4As shown, the block storage management device 400 includes a first acquisition module 410 , an analysis module 420 , a generation module 430 , and an adjustment module 440 .
[0080] The first acquisition module 410 is used to acquire the current overall flow of the storage pool from the first data table when the flow of the target storage volume is monitored to meet the first threshold condition, wherein the first threshold condition is determined based on the performance parameter of the target storage volume. In one embodiment, the first acquisition module 410 can be used to perform the operation S201 described above, which will not be described in detail here.
[0081] The analysis module 420 is used to perform overload analysis according to the first threshold condition and the current overall traffic to obtain an analysis result. In one embodiment, the acquisition module 420 can be used to perform the operation S202 described above, which will not be described in detail here.
[0082] The generating module 430 is used to generate the second threshold condition based on the first threshold condition when the analysis result indicates performance redundancy. In one embodiment, the acquiring module 430 can be used to perform the operation S203 described above, which will not be described in detail here.
[0083] The adjustment module 440 is used to adjust the performance parameter of the target storage volume according to the second threshold condition. In one embodiment, the acquisition module 440 can be used to perform the operation S204 described above, which will not be described in detail here.
[0084] According to the embodiments of the present disclosure, by performing an overload analysis on the overall traffic of the storage pool and adjusting the performance parameters of the target storage volume when the analysis result indicates performance redundancy, dynamic expansion of the target storage volume is achieved. This can not only ensure the long-term reliable operation of the distributed block storage system within its capabilities, but also prevent the back-end internal traffic from affecting the quality of other external businesses, while ensuring the normal operation of high-traffic demand businesses.
[0085] According to an embodiment of the present disclosure, the block storage management device may further include a first timing module, a monitoring module, and a first recording module.
[0086] The first timing module is used to set a first timing task.
[0087] The monitoring module is used to monitor the overall flow of the storage pool in response to triggering the first scheduled task.
[0088] The first recording module is used to record the overall flow of the storage pool in a first data table.
[0089] According to an embodiment of the present disclosure, the analysis module 420 may include a calculation unit, an analysis unit, and a conclusion unit.
[0090] The calculation unit is used to calculate the predicted overall flow rate according to the current overall flow rate and the first threshold condition.
[0091] The analysis unit is used to determine whether an overload trend alarm will be triggered based on the predicted overall traffic analysis.
[0092] The conclusion unit is used to obtain an analysis result indicating insufficient performance when it is determined that the predicted overall flow will trigger an overload trend alarm; it can also obtain an analysis result indicating performance redundancy when it is determined that the predicted overall flow will not trigger an overload trend alarm.
[0093] According to an embodiment of the present disclosure, the block storage management device may further include a first determination module.
[0094] The first judgment module is used to trigger an overload trend alarm when the ratio of the predicted overall flow and the previous overall flow is greater than or equal to a preset ratio, wherein the previous overall flow is recorded in the first data table, and the previous overall flow includes the overall flow of the storage pool monitored when the second scheduled task was triggered last; or trigger an overload trend alarm when the predicted overall flow meets a preset threshold condition.
[0095] According to an embodiment of the present disclosure, the block storage management device may further include a second recording module.
[0096] The second recording module is used to record the number of times the performance parameter of the storage volume has been adjusted in the second data table after the performance parameter of the storage volume is adjusted.
[0097] According to an embodiment of the present disclosure, the block storage management device may further include a second acquisition module and a second determination module.
[0098] The second acquisition module is used to acquire the number of times the performance parameter of the target storage volume has been adjusted.
[0099] The second judgment module is used to send a manual intervention request to the operation and maintenance terminal device when the number of times the performance parameters of the target storage volume have been adjusted is greater than or equal to a preset number of times; or to adjust the performance parameters of the target storage volume according to a second threshold condition when the number of times the performance parameters of the target storage volume have been adjusted is less than a preset number of times.
[0100] According to an embodiment of the present disclosure, the block storage management device may further include a second timing module and a calling module.
[0101] The second timing module is used to set the second timing task.
[0102] The calling module is used to call a preset script to monitor the traffic of each storage volume in response to triggering the second scheduled task.
[0103] According to an embodiment of the present disclosure, the block storage management device may further include a creation module, a construction module, and a configuration module.
[0104] The creation module is used to create a storage volume from a storage pool in response to a virtual machine deployment request.
[0105] The building module is used to call the virtual machine component to build a virtual machine based on the storage volume.
[0106] The configuration module is used to configure the performance parameters of the storage volume according to the initial threshold conditions.
[0107] According to an embodiment of the present disclosure, the block storage management device further includes a third judgment module.
[0108] The third judgment module is used to configure the performance parameters of the storage volume according to the initial threshold conditions when the flow identifier is represented as a standard flow identifier; when the flow identifier is represented as a high flow identifier, configure the performance parameters of the storage volume according to a preset policy.
[0109] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.
[0110] For example, any multiple of the first acquisition module 410, the analysis module 420, the generation module 430, and the adjustment module 440 can be combined in one module / unit / subunit for implementation, or any one of the modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functions of one or more of these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to an embodiment of the present disclosure, at least one of the first acquisition module 410, the analysis module 420, the generation module 430, and the adjustment module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware or by a suitable combination of any of them. Alternatively, at least one of the first acquisition module 410 , the analysis module 420 , the generation module 430 , and the adjustment module 440 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.
[0111] It should be noted that the block storage management device part in the embodiment of the present disclosure corresponds to the block storage management method part in the embodiment of the present disclosure. The description of the block storage management device part specifically refers to the storage management method part, which will not be repeated here.
[0112] Figure 5 A block diagram of an electronic device suitable for implementing a block storage management method according to an embodiment of the present disclosure is schematically shown. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0113] like Figure 5 As shown, the computer electronic device 500 according to the embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.
[0114] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0115] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.
[0116] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0117] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0118] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.
[0119] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 502 and / or the RAM 503 described above and / or one or more memories other than the ROM 502 and the RAM 503 .
[0120] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the block storage management method provided by the embodiments of the present disclosure.
[0121] When the computer program is executed by the processor 501, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0122] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0123] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0125] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A block storage management method, comprising: When it is monitored that the traffic of the target storage volume meets a first threshold condition, obtaining the current overall traffic of the storage pool from the first data table, wherein the first threshold condition is determined based on the performance parameter of the target storage volume; Performing an overload analysis according to the first threshold condition and the current overall flow to obtain an analysis result; In the case where the analysis result indicates performance redundancy, generating a second threshold condition based on the first threshold condition; and adjusting the performance parameter of the target storage volume according to the second threshold condition, and recording the number of times the performance parameter of the target storage volume has been adjusted in a second data table; The method further comprises: If the number of times the performance parameter of the target storage volume has been adjusted is greater than or equal to a preset number of times, sending a manual intervention request to the operation and maintenance terminal device; and When the number of times the performance parameter of the target storage volume has been adjusted is less than the preset number of times, the performance parameter of the target storage volume is adjusted according to the second threshold condition.
2. The method according to claim 1, further comprising: Set the first scheduled task; In response to triggering the first scheduled task, monitoring the overall flow of the storage pool; as well as The overall flow of the storage pool is recorded in the first data table.
3. The method according to claim 2, wherein: The performing overload analysis according to the first threshold condition and the current overall flow to obtain an analysis result includes: Calculate the predicted overall flow rate according to the current overall flow rate and the first threshold condition; Whether an overload trend alarm will be triggered based on the predicted overall flow analysis; In the case where it is determined that the predicted overall flow rate will trigger the overload trend alarm, obtaining the analysis result indicating insufficient performance; and When it is determined that the predicted overall flow will not trigger the overload trend alarm, the analysis result indicating performance redundancy is obtained.
4. The method according to claim 3, further comprising: In the case where the ratio of the predicted overall flow rate to the previous overall flow rate is greater than or equal to a preset ratio, the overload trend alarm is triggered, wherein the previous overall flow rate is recorded in the first data table, and the previous overall flow rate includes the overall flow rate of the storage pool monitored when the second scheduled task was triggered last time; or When the predicted overall flow rate meets a preset threshold condition, the overload trend alarm is triggered.
5. The method according to claim 1, further comprising: Set up the second scheduled task; In response to triggering the second scheduled task, a preset script is called to monitor the traffic of each storage volume.
6. The method according to claim 1, further comprising: In response to a virtual machine deployment request, creating a storage volume from the storage pool; Invoking a virtual machine component to build a virtual machine based on the storage volume; as well as The performance parameters of the storage volume are configured according to the initial threshold conditions.
7. The method according to claim 6, wherein: The virtual machine deployment request carries a flow identifier; The method further comprises: When the flow identifier is a standard flow identifier, configuring the performance parameter of the storage volume according to the initial threshold condition; When the traffic identifier indicates a high traffic identifier, the performance parameter of the storage volume is configured according to a preset policy.
8. A block storage management device, comprising: A first acquisition module is used to acquire the current overall flow of the storage pool from a first data table when it is monitored that the flow of the target storage volume meets a first threshold condition, wherein the first threshold condition is determined based on the performance parameter of the target storage volume; An analysis module, configured to perform an overload analysis according to the first threshold condition and the current overall flow, and obtain an analysis result; a generating module, configured to generate a second threshold condition based on the first threshold condition when the analysis result indicates performance redundancy; and an adjustment module, which adjusts the performance parameter of the target storage volume according to the second threshold condition, and records the number of times the performance parameter of the target storage volume has been adjusted in a second data table; The device also includes: The second judgment module is used to send a manual intervention request to the operation and maintenance terminal device when the number of times the performance parameters of the target storage volume have been adjusted is greater than or equal to a preset number of times; and to adjust the performance parameters of the target storage volume according to the second threshold condition when the number of times the performance parameters of the target storage volume have been adjusted is less than the preset number of times.
9. An electronic device, comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7. 10 . A computer-readable storage medium having executable instructions stored thereon, wherein when the executable instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1 to 7.
11. A computer program product, comprising computer executable instructions, wherein the computer executable instructions are used to implement the method according to any one of claims 1 to 7 when executed.
