Storage node operation trend prediction method and device, medium and program product

Abstract

The invention discloses a storage node running trend prediction method and device, a medium and a program product, and relates to the field of distributed technologies. The method comprises the following steps: acquiring monitoring data of a storage node from a registration system running in registration equipment through a monitoring system, and storing the monitoring data; a running trend prediction request is obtained, and the running trend prediction request comprises a to-be-predicted label and a to-be-predicted monitoring index; the method comprises the following steps: determining a value of a first target monitoring index matched with a to-be-predicted label and a to-be-predicted monitoring index through a monitoring system according to a label of a storage node in monitoring data and the monitoring index; and according to the value of the first target monitoring index, determining the operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring index. According to the method, a self-discovery mechanism of a monitoring system is introduced, and the manpower maintenance cost is reduced, so that the monitoring efficiency and the monitoring flexibility are improved, meanwhile, the running trend prediction is realized, and the monitoring comprehensiveness and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed technology, and in particular to a storage node operation trend prediction method, device, medium and program product. Background Art

[0002] In a distributed system, storage nodes are used to store configuration data and metadata. Therefore, monitoring of storage nodes is crucial.

[0003] In the related art, the monitoring of storage nodes can be achieved through a monitoring system: by manually configuring the Internet Protocol (IP) address and port and other address information of the storage node in the configuration file of the monitoring system, and then obtaining monitoring data from the storage node based on the address information.

[0004] However, in the above process, when a new storage node comes online, the address information of the storage node needs to be manually added to the configuration file of the monitoring system; when a storage node goes offline, the address information of the storage node needs to be manually deleted from the configuration file of the monitoring system, resulting in low monitoring efficiency and flexibility. Summary of the invention

[0005] The present invention provides a storage node operation trend prediction method, device, medium and program product to solve the technical problem that the monitoring method in the related art has low monitoring efficiency and monitoring flexibility.

[0006] According to one aspect of the present invention, a storage node operation trend prediction method is provided, which is applied to a monitoring device, wherein a monitoring system is running on the monitoring device, and the method comprises:

[0007] Obtaining, through the monitoring system, monitoring data of a storage node from a registration system running in a registration device, and storing the monitoring data; wherein the storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node;

[0008] Obtaining an operation trend prediction request; wherein the operation trend prediction request includes a tag to be predicted and a monitoring indicator to be predicted;

[0009] Determine, by the monitoring system, a value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator;

[0010] According to the value of the first target monitoring indicator, an operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring indicator is determined.

[0011] According to another aspect of the present invention, a storage node operation trend prediction device is provided, the device is arranged on a monitoring device, a monitoring system is running on the monitoring device, and the device comprises:

[0012] A first acquisition module is used to acquire monitoring data of a storage node from a registration system running in a registration device through the monitoring system, and store the monitoring data; wherein the storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node;

[0013] A second acquisition module is used to acquire an operation trend prediction request; wherein the operation trend prediction request includes a tag to be predicted and a monitoring indicator to be predicted;

[0014] A first determination module is used to determine, through the monitoring system, a value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator;

[0015] The second determination module is used to determine the operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring indicator according to the value of the first target monitoring indicator.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the storage node operation trend prediction method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the storage node operation trend prediction method described in any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the storage node operation trend prediction method according to any embodiment of the present invention is implemented.

[0022] The technical solution of an embodiment of the present invention includes: obtaining monitoring data of a storage node from a registration system running in a registration device through a monitoring system, and storing the monitoring data, wherein the storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node; obtaining an operation trend prediction request, wherein the operation trend prediction request includes a label to be predicted and a monitoring indicator to be predicted; determining, through the monitoring system, the value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator; determining, according to the value of the first target monitoring indicator, the operation trend of the storage node with the label to be predicted based on the monitoring indicator to be predicted. The method has the following technical effects: on the one hand, the monitoring system realizes pulling monitoring data through the registration system, and the storage nodes can dynamically register or deregister from the registration system. Therefore, the monitoring system can perceive the changes of storage nodes in the distributed environment, so that the monitoring process can adapt to the changing storage nodes, which is equivalent to introducing a self-discovery mechanism of the monitoring system, reducing the manpower maintenance cost, thereby improving the monitoring efficiency, monitoring flexibility and adaptability; on the other hand, it is also possible to realize operation trend prediction based on the monitoring data, improve the comprehensiveness of monitoring, and realize operation trend prediction based on label granularity and monitoring indicator granularity, thereby improving the accuracy of monitoring.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a flow chart of a storage node operation trend prediction method provided by an embodiment of the present invention;

[0026] Figure 2 It is a schematic diagram of an application scenario of the storage node operation trend prediction method provided by an embodiment of the present invention;

[0027] Figure 3 is a flowchart of another storage node operation trend prediction method provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of various thresholds in an embodiment of the present invention;

[0029] Figure 5 It is a structural diagram of a storage node operation trend prediction device provided by an embodiment of the present invention;

[0030] Figure 6 It is a structural schematic diagram of an electronic device for implementing the storage node operation trend prediction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the embodiments of the present invention all comply with the relevant provisions of national laws and regulations.

[0033] Figure 1 This is a flowchart of a storage node operation trend prediction method provided by an embodiment of the present invention. This embodiment can be applied to a scenario in which storage nodes in a distributed system are monitored. The method can be executed by a storage node operation trend prediction device, which can be implemented in the form of hardware and / or software. The storage node operation trend prediction device can be configured in an electronic device, for example, a monitoring device. A monitoring system is running on the monitoring device in this embodiment. Figure 1 As shown, the method includes the following steps 101 to 104.

[0034] Step 101: Obtain monitoring data of a storage node from a registration system running in a registration device through a monitoring system, and store the monitoring data.

[0035] The storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node.

[0036] The storage node in this embodiment refers to a device used to store configuration data and metadata in a distributed system. The normal operation of the storage node is of great significance to the stability and reliability of the entire distributed system. Therefore, monitoring of the storage node is very important.

[0037] In this embodiment, in order to realize automatic monitoring access of storage nodes, avoid frequent operation of monitoring device configuration files due to changes in storage nodes, and reduce the pressure on operation and maintenance personnel, monitoring can be realized by registering devices.

[0038] The registration system in the registration device of this embodiment can provide storage node registration and realize the service discovery function of the monitoring system, so that the monitoring system can dynamically discover and monitor the storage node. The storage node in this embodiment exposes a monitoring indicator interface to the outside. When the storage node is online, it is registered in the registration system; when the storage node is offline, it is unregistered from the registration system.

[0039] Optionally, the registration system in this embodiment can provide a health check. The registration system detects whether the operation of the storage node is normal at a preset frequency; when a storage node is detected to be operating abnormally, the storage node can be actively deregistered.

[0040] When registering, the storage node can register by sending a registration request to the registration system. Optionally, the registration request may include: the port and IP address of the registration system, the tag list of the storage node, and the health check configuration (including the address and check interval of the health check).

[0041] Exemplarily, a possible registration request may be: curl-X PUT-d'{"id":"'${IP}'","name":"'XXX'","address":"'${IP}}'","port":2379,"tags":["slb-***"],"checks":[{"http":"http: / / '${IP}':2379 / metrics","interval":"5s"}]}'http: / / ${SSS}:8500 / v1 / agent / service / register. Wherein, 8500 represents the port of the registration system, and / v1 / agent / service / register represents the Uniform Resource Location (URL) of the registration system. http: / / '${IP}':2379 / metrics represents the URL of the health check, and the check interval is 5 seconds.

[0042] When deregistering, the storage node may implement deregistration by sending a deregistration request to the registration system. Optionally, the deregistration request may include: a port and an IP address of the registration system.

[0043] For example, a possible deregistration request may be: curl--request PUT http: / / ${SSS}:8500 / v1 / agent / service / deregister / ${ip}, where 8500 represents the port of the registration system, / v1 / agent / service / deregister / represents the URL of the registration system, and ${ip} represents the IP address of the registration system.

[0044] The monitoring system in this embodiment refers to a system that can obtain monitoring data of a storage node and analyze the monitoring data. The monitoring system in this embodiment runs in a monitoring device.

[0045] Optionally, in order to realize monitoring storage nodes through registration devices, before step 101, the method further includes the following steps: modifying the configuration file of the monitoring system to register the registration system with the monitoring system. The modified configuration file includes the address information of the registration system. This implementation method realizes the registration of the registration system with the monitoring system, which facilitates the subsequent pulling of monitoring data through the registration system, and improves the monitoring efficiency, monitoring flexibility and adaptability.

[0046] Optionally, a possible implementation of the modified configuration file is:

[0047] scrape_configs:

[0048] -job_name:***

[0049] ***_sd_configs:

[0050] -server:$#{ENV_ELC_DOMAIN}:8500

[0051] relabel_configs:

[0052] -source_labels:[__meta_tags]

[0053] regex:.*ccc.*

[0054] action:keep

[0055] Among them, $#{ENV_ELC_DOMAIN}:8500 specifies the IP address and port of the registration system.

[0056] The monitoring system can obtain the monitoring data of the storage nodes from the registration system at a preset frequency and store the monitoring data. This embodiment realizes the dynamic update of the monitoring configuration by registering and deregistering the storage nodes on the registration system, ensuring that the monitoring data obtained by the monitoring system changes synchronously with the storage node structure. A possible monitoring data acquisition process is: the monitoring system sends a data acquisition request to the registration system; after receiving the data acquisition request, the registration system obtains monitoring data from each storage node that has been registered and passed the health check; the registration system sends the acquired monitoring data to the monitoring system.

[0057] Optionally, the storage system can also selectively store monitoring data. In the above modified configuration file, regex:.*ccc.* represents a regular expression, and only monitoring data that meets .*ccc.* will be stored in the monitoring device.

[0058] In this embodiment, in order to improve the accuracy of monitoring, each monitoring data includes a label of the storage node. The label in this embodiment is used to characterize a certain attribute of the storage node. Exemplarily, the label can be information indicating the location of the storage node, such as Park 1, Park 2, etc., and the label can also be information indicating the purpose of the storage node, such as production environment, test environment, etc.

[0059] The monitoring indicators in this embodiment are used to characterize the dimensions of monitoring. The monitoring indicators in this embodiment can be at least one. Exemplarily, the monitoring indicators in this embodiment may include at least one of the following: storage space occupancy, central processing unit (CPU) running time, disk space, network traffic, disk response, disk latency, server response, etc. The value of the monitoring indicator in this embodiment refers to the specific value of the monitoring indicator in the storage node. For example, the storage space occupancy is 30%.

[0060] In one implementation, in order to facilitate the implementation of subsequent steps based on labels, the implementation process of step 101 is: through the monitoring system, the original monitoring data of the storage node is obtained from the registration system running in the registration device, wherein the original monitoring data includes the original label of the storage node and the value of the monitoring indicator of the storage node; through the monitoring system, the original label of the storage node is converted into the label of the storage node, and the converted monitoring data is used as the monitoring data; and the monitoring data is stored.

[0061] Continuing with the example based on the modified configuration file above, relabel_configs: represents the renamed labels (i.e., the labels in this embodiment), and source_labels: represents the original labels.

[0062] In this implementation, when the original label of the storage node is converted into the label of the storage node, the converted label is a standardized label. Optionally, the monitoring system can convert the original label of the storage node into the label of the storage node based on a pre-stored label conversion mapping relationship. The label conversion mapping relationship is used to characterize the mapping relationship between the original label and the standardized label. This implementation can achieve the standardization of labels, which is convenient for the subsequent data query or operation trend prediction based on label granularity.

[0063] It should be noted that in step 101, monitoring data may be acquired multiple times at a preset frequency, and the monitoring data acquired multiple times may be stored. For example, the frequency in this embodiment may be 0.1 Hz.

[0064] Step 102: Obtain an operation trend prediction request.

[0065] The running trend prediction request includes tags to be predicted and monitoring indicators to be predicted.

[0066] The operation trend prediction request in this embodiment can be sent under the trigger of the operation and maintenance personnel, or can be generated by other devices and sent to the monitoring device. This embodiment can realize the operation trend prediction based on the tag granularity and the monitoring indicator granularity.

[0067] For example, the tag to be predicted may be, for example, Park 1, and the monitoring indicator to be predicted may be, for example, CPU running time and storage space occupancy rate.

[0068] Step 103: Determine, through the monitoring system, the value of the first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node and the monitoring indicator in the monitoring data.

[0069] After obtaining the operation trend prediction request, the value of the first target monitoring indicator that matches both the tag to be predicted and the monitoring indicator to be predicted in the operation trend prediction request may be determined.

[0070] A possible implementation of step 103 is: determining monitoring data that matches the label to be predicted based on the label of the node stored in the monitoring data; and determining the value of the first target monitoring indicator that matches the monitoring indicator to be predicted from the monitoring data that matches the label to be predicted.

[0071] In one implementation, the monitoring system can determine the value of the first target monitoring indicator that matches both the tag to be predicted and the monitoring indicator to be predicted based on the tags and monitoring indicators of the storage nodes in the monitoring data acquired within a preset time period. For example, the preset time period here can be the monitoring data acquired in the past 12 hours, or the monitoring data acquired in the past 48 hours.

[0072] In another implementation, in order to improve the accuracy of the operation trend prediction, the monitoring data also includes the timestamp of the value of each monitoring indicator, and the operation trend prediction request also includes the prediction duration. Correspondingly, the implementation process of step 103 includes the following steps: through the monitoring system, from the timestamp of the value of the monitoring indicator, determine the candidate monitoring data that matches the prediction duration, wherein the longer the prediction duration, the earlier the timestamp of the candidate monitoring data; according to the label of the storage node and the monitoring indicator in the candidate monitoring data, determine the value of the first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted.

[0073] This implementation corresponds to the following scenarios: If you want to predict the operating trend after 3 days, you need the monitoring data of the past 10 days; if you want to predict the operating trend after 5 days, you need the monitoring data of the past 20 days. Here, 3 days and 5 days are the prediction duration. The longer the prediction duration, the more monitoring data is required, that is, the earlier the timestamp of the required monitoring data is. This implementation realizes the prediction based on the monitoring data that matches the prediction duration, which improves the prediction accuracy.

[0074] Step 104: According to the value of the first target monitoring indicator, determine the operation trend of the storage node with the label to be predicted based on the monitoring indicator to be predicted.

[0075] In this embodiment, the operation trend of the storage node refers to the change process of the monitoring indicator to be predicted of the storage node over time after the current moment.

[0076] Optionally, step 104 may also be implemented through a monitoring system, that is, through the monitoring system, according to the value of the first target monitoring indicator, determine the operation trend of the storage node with the label to be predicted based on the monitoring indicator to be predicted.

[0077] In step 104, in a scenario where there are multiple monitoring indicators to be predicted, it is possible to determine the operating trend of each monitoring indicator to be predicted of the storage node with the label to be predicted.

[0078] For example, assuming that the tag to be predicted is Park 1, and the monitoring indicators to be predicted include: CPU running time and storage space occupancy rate, the value of the first target monitoring indicator includes: the value of the CPU running time of each storage node in Park 1, and the value of the storage space occupancy rate of each storage node in Park 1. In step 104, the operating trend of the CPU running time of each storage node in Park 1 can be determined according to the value of the CPU running time of each storage node in Park 1, and the operating trend of the storage space occupancy rate of each storage node in Park 1 can be determined according to the value of the storage space occupancy rate of each storage node in Park 1.

[0079] In one implementation, a big data prediction model may be used to determine the operating trend of a storage node with a label to be predicted based on a monitoring indicator to be predicted.

[0080] In another implementation, the operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring indicator may be determined by a trend prediction function in the monitoring system.

[0081] Furthermore, after determining the operation trend of a storage node, the time when the value of the monitoring indicator to be predicted exceeds the threshold in the operation trend can be determined according to a predetermined threshold, and the time can be fed back to the operation and maintenance personnel or the control device for reminder. For example, the threshold is 90%, the monitoring indicator to be predicted is the storage space occupancy rate, the label to be predicted is Park 1, and the operation trend of the storage space occupancy rate of a storage node indicates that the storage space occupancy rate of the storage node will exceed 90% in 10 days. Then, the storage space occupancy rate of the storage node will exceed 90% in 10 days. Feedback to the operation and maintenance personnel or the control device allows the operation and maintenance personnel to determine the subsequent processing strategy and improve the operation reliability of the storage node.

[0082] Optionally, the method further comprises the following steps: sending monitoring data to a display device through a monitoring system, wherein the data visualization platform in the display device is used to visualize the monitoring data according to predefined display rules and with tags as granularity, and to send warning information when the monitoring data meets the warning rules according to predefined warning rules.

[0083] The data visualization platform in this embodiment can visualize monitoring data with tags as the granularity. For example, monitoring data with the same tag can be displayed in the same graph for visualization. Furthermore, the data visualization platform can also visualize monitoring data with monitoring indicators as the granularity based on tags as the granularity.

[0084] Optionally, the data visualization platform in this embodiment can also send warning information according to predefined warning rules when the monitoring data meets the warning rules. In this embodiment, the warning information can be sent by email, instant messaging software, etc.

[0085] Optionally, the data visualization platform in this embodiment can also summarize the monitoring data of each storage node in the storage cluster at the storage cluster granularity, for example, summarizing the values ​​of monitoring indicators such as throughput and response delay of each storage node in a storage cluster.

[0086] Figure 2 Schematic diagram of an application scenario of the storage node operation trend prediction method provided by an embodiment of the present invention. Figure 2 As shown, each storage node registers or unregisters in the registration system of the registration device. The monitoring device obtains monitoring data from the registration system through the monitoring system. The monitoring device can also send monitoring data to the data visualization platform of the display device through the monitoring system to realize the visualization and alarm of the monitoring data. The storage node operation trend prediction method provided in this embodiment can be applied to the monitoring process in the field of data development, analysis and industrial chain.

[0087] The storage node operation trend prediction method provided in this embodiment includes: obtaining monitoring data of the storage node from a registration system running in a registration device through a monitoring system, and storing the monitoring data, wherein the storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node; obtaining an operation trend prediction request, wherein the operation trend prediction request includes a label to be predicted and a monitoring indicator to be predicted; determining, through the monitoring system, the value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator; determining, according to the value of the first target monitoring indicator, the operation trend of the storage node with the label to be predicted based on the monitoring indicator to be predicted. The method has the following technical effects: on the one hand, the monitoring system realizes pulling monitoring data through the registration system, and the storage nodes can dynamically register or deregister from the registration system. Therefore, the monitoring system can perceive the changes of storage nodes in the distributed environment, so that the monitoring process can adapt to the changing storage nodes, which is equivalent to introducing a self-discovery mechanism of the monitoring system, reducing the manpower maintenance cost, thereby improving the monitoring efficiency, monitoring flexibility and adaptability; on the other hand, it is also possible to realize operation trend prediction based on the monitoring data, improve the comprehensiveness of monitoring, and realize operation trend prediction based on label granularity and monitoring indicator granularity, thereby improving the accuracy of monitoring.

[0088] Figure 3 is a flowchart of another storage node operation trend prediction method provided by an embodiment of the present invention. The storage node operation trend prediction method provided by this embodiment, Figure 1 Based on the illustrated embodiment and various optional implementations, the implementation of the subsequent query process is described in detail. Figure 3 As shown, the storage node operation trend prediction method provided in this embodiment includes the following steps 301 to 308.

[0089] Step 301: Obtain monitoring data of a storage node from a registration system running in a registration device through a monitoring system, and store the monitoring data.

[0090] The storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node.

[0091] Step 302: Obtain an operation trend prediction request.

[0092] The running trend prediction request includes tags to be predicted and monitoring indicators to be predicted.

[0093] Step 303: Determine, through the monitoring system, the value of the first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node and the monitoring indicator in the monitoring data.

[0094] Step 304: According to the value of the first target monitoring indicator, determine the operation trend of the storage node with the tag to be predicted based on the monitoring indicator to be predicted.

[0095] The implementation process and technical principle of step 301 and step 101, step 302 and step 102, step 303 and step 103, and step 304 and step 104 are similar, and will not be repeated here.

[0096] Step 305: Obtain the query request sent by the query device.

[0097] The query request includes the tags to be queried and the calculation method of each monitoring indicator.

[0098] This embodiment can realize flexible query based on monitoring data. The query request in this embodiment can be sent by the query device under the trigger of the operation and maintenance personnel, or it can be generated by the query device and sent to the monitoring device. The query request in this embodiment can include the tag to be queried and the calculation method of each monitoring indicator.

[0099] Optionally, the operation method in this embodiment may be an operation method such as calculating an average value, determining a maximum value, determining a minimum value, determining a median, etc.

[0100] Step 306: Determine, through the monitoring system, the value of the second target monitoring indicator that matches the label to be queried according to the label of the storage node in the monitoring data.

[0101] This step is similar to step 103. After the query request is obtained, the value of the second target monitoring indicator matching the label to be queried is determined through the monitoring system according to the label of the storage node in the monitoring data.

[0102] In one implementation, the monitoring system may determine the value of the second target monitoring indicator that matches the tag to be queried based on the tag of the storage node in the monitoring data acquired within a preset time period.

[0103] In another implementation, the monitoring data also includes the timestamp of the value of each monitoring indicator, and the query request also includes the query duration. Correspondingly, the implementation process of step 306 includes the following steps: determining the monitoring data matching the query duration from the timestamp of the value of the monitoring indicator through the monitoring system; and determining the value of the second target monitoring indicator matching the label to be queried based on the label of the storage node in the monitoring data matching the query duration.

[0104] Step 307: For each monitoring indicator, according to the calculation method corresponding to the monitoring indicator, calculate the value corresponding to the monitoring indicator in the value of the second target monitoring indicator to obtain the calculated value.

[0105] Optionally, step 307 may also be implemented through a monitoring system.

[0106] In this embodiment, the calculation methods corresponding to different monitoring indicators can be the same or different, and this embodiment is not limited to this.

[0107] In step 307, each monitoring indicator may be operated according to a corresponding operation mode to obtain a calculated value.

[0108] Optionally, the monitoring indicators in this embodiment include: storage space occupancy rate and central processing unit CPU running time. The storage space occupancy rate is calculated by determining a minimum value, and the CPU running time is calculated by determining an average value. In step 307, for the storage space occupancy rate, the minimum value of the storage space occupancy rate can be determined. For the CPU running time, the average value of the CPU running time can be determined.

[0109] Step 308: Feedback the calculated value to the query device.

[0110] Furthermore, in this embodiment, when the calculated value includes the minimum storage space occupancy rate and the average value of the CPU running time, in the first scenario, after step 308, the method provided by this embodiment also includes the following steps: If the minimum storage space occupancy rate of each storage node with the label to be queried belonging to the same storage cluster is greater than a preset first storage space occupancy rate threshold, and the average value of the CPU running time is greater than the preset first CPU running time threshold, it is determined that the storage cluster needs to add a storage node with the label to be queried.

[0111] The monitoring data uploaded by each storage node may also include an identifier of the storage node. The storage cluster to which the storage node belongs is determined based on the identifier of the storage node. In the first scenario, the preset first storage space occupancy threshold is a predetermined value used to represent the upper limit of the storage space occupancy. The preset first CPU runtime threshold is a predetermined value used to represent the upper limit of the CPU runtime. If the minimum value of the storage space occupancy of each storage node with a tag to be queried in the same storage cluster is greater than the preset first storage space occupancy threshold, and the average value of the CPU runtime is greater than the preset first CPU runtime threshold, it means that the number of storage nodes in the storage cluster cannot meet the actual business needs. In order to improve the reliability of business operations, the storage cluster needs to be expanded.

[0112] Optionally, in the second scenario, after step 308, the method provided in this embodiment further includes the following steps: if the minimum value of the storage space occupancy rate of each storage node with the tag to be queried belonging to the same storage cluster is less than the preset second storage space occupancy rate threshold, and the average value of the CPU running time is less than the preset second CPU running time threshold, it is determined that the storage cluster needs to reduce the storage nodes with the tag to be queried. Among them, the preset first storage space occupancy rate threshold is greater than the preset second storage space occupancy rate threshold, and the preset first CPU running time threshold is greater than the preset second CPU running time threshold.

[0113] In the second scenario, the preset second storage space occupancy rate threshold is a predetermined value used to represent the lower limit of the storage space occupancy rate. The preset second CPU runtime threshold is a predetermined value used to represent the lower limit of the CPU runtime. The minimum value of the storage space occupancy rate of each storage node with a tag to be queried in the same storage cluster is less than the preset second storage space occupancy rate threshold, and the average value of the CPU runtime is less than the preset second CPU runtime threshold, indicating that there is redundancy in the number of storage nodes in the storage cluster, and the performance of some storage nodes is not fully utilized. In order to save storage resources, the storage cluster needs to be scaled down.

[0114] For example, the preset first storage space occupancy rate threshold may be 60%, the preset second storage space occupancy rate threshold may be 20%, the preset first CPU operation time threshold may be 18 hours, and the preset second CPU operation time threshold may be 2 hours.

[0115] Optionally, the implementation process of the above three scenarios can also be achieved through a monitoring system.

[0116] Figure 4 Schematic diagram of various thresholds in the embodiment of the present invention. Figure 4 As shown, when the minimum storage space occupancy rate of each storage node with a to-be-queried tag belonging to the same storage cluster is greater than a preset first storage space occupancy rate threshold, and the average value of the CPU running time is greater than the preset first CPU running time threshold, it is determined that the storage cluster needs to be expanded.

[0117] If the minimum storage space occupancy rate of the storage nodes with the to-be-queried tag belonging to the same storage cluster is less than the preset second storage space occupancy rate threshold, and the average CPU running time is less than the preset second CPU running time threshold, it is determined that the storage cluster needs to be reduced in capacity.

[0118] In the third scenario, if the minimum value of the storage space occupancy rate of each storage node with the label to be queried belonging to the same storage cluster is greater than or equal to the preset second storage space occupancy rate threshold and less than or equal to the preset first storage space occupancy rate threshold, and / or the average value of the CPU running time of each storage node with the label to be queried belonging to the same storage cluster is greater than or equal to the preset second CPU running time threshold and less than or equal to the preset first CPU running time threshold, then it is determined that the number of storage nodes included in the storage cluster is more appropriate.

[0119] In the above three scenarios, based on the query with tags as the granularity and the monitoring indicators as the granularity, it is possible to determine whether the capacity of the storage cluster is appropriate, so as to optimize resource utilization and improve data development and analysis efficiency.

[0120] The storage node operation trend prediction method provided in this embodiment can realize the query of monitoring data with labels as granularity, and can calculate the value of monitoring indicators according to the specified calculation method, so as to meet different monitoring requirements. Furthermore, on the basis of querying with labels as granularity, it is also possible to determine whether the capacity of the storage cluster is appropriate with monitoring indicators as granularity, so as to optimize the utilization of storage resources, improve the reliability of business operation and save storage resources.

[0121] Figure 5 1 is a schematic diagram of a storage node operation trend prediction device provided by an embodiment of the present invention. The device is arranged on a monitoring device, and a monitoring system is running on the monitoring device. Figure 5 As shown, the storage node operation trend prediction device provided in this embodiment includes the following modules: a first acquisition module 51 , a second acquisition module 52 , a first determination module 53 and a second determination module 54 .

[0122] The first acquisition module 51 is used to acquire the monitoring data of the storage node from the registration system running in the registration device through the monitoring system, and store the monitoring data.

[0123] The storage node is a node pre-registered in the registration system. The monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node.

[0124] The second acquisition module 52 is used to acquire an operation trend prediction request.

[0125] The operation trend prediction request includes tags to be predicted and monitoring indicators to be predicted.

[0126] The first determination module 53 is used to determine, through the monitoring system, the value of the first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator.

[0127] The second determination module 54 is used to determine the operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring indicator according to the value of the first target monitoring indicator.

[0128] In one embodiment, the device further comprises a configuration file modification module, which is used to modify the configuration file of the monitoring system so as to register the registration system with the monitoring system, wherein the modified configuration file comprises the address information of the registration system.

[0129] In one embodiment, the monitoring data also includes the timestamp of the value of each monitoring indicator, and the operation trend prediction request also includes the prediction duration. The first determination module 53 is specifically used to: determine the candidate monitoring data that matches the prediction duration from the timestamp of the value of the monitoring indicator through the monitoring system, wherein the longer the prediction duration, the earlier the timestamp of the candidate monitoring data; and determine the value of the first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted based on the label of the storage node in the candidate monitoring data and the monitoring indicator.

[0130] In one embodiment, the device further includes: a third acquisition module, a third determination module, a calculation module and a feedback module.

[0131] The third acquisition module is used to acquire the query request sent by the query device.

[0132] The query request includes the tag to be queried and the calculation method of each monitoring indicator.

[0133] The third determination module is used to determine, through the monitoring system, the value of the second target monitoring indicator that matches the label to be queried according to the label of the storage node in the monitoring data.

[0134] The operation module is used to operate on the value corresponding to the monitoring indicator in the value of the second target monitoring indicator for each monitoring indicator according to the operation method corresponding to the monitoring indicator, so as to obtain the value after the operation.

[0135] A feedback module is used to feed back the calculated value to the query device.

[0136] In one embodiment, the monitoring index includes: storage space occupancy rate and central processing unit CPU operation time, the storage space occupancy rate is calculated by determining a minimum value, the CPU operation time is calculated by determining an average value, and the calculated value includes the minimum value of the storage space occupancy rate and the average value of the CPU operation time. The device also includes: a fourth determination module and a fifth determination module.

[0137] The fourth determination module is used to determine that the storage cluster needs to add a storage node with a label to be queried if the minimum value of the storage space occupancy rate of each storage node with a label to be queried belonging to the same storage cluster is greater than a preset first storage space occupancy rate threshold, and the average value of the CPU running time is greater than the preset first CPU running time threshold.

[0138] The fifth determination module is used to determine that the storage cluster needs to reduce the storage nodes with the label to be queried if the minimum value of the storage space occupancy rate of the storage nodes with the label to be queried belonging to the same storage cluster is less than the preset second storage space occupancy rate threshold, and the average value of the CPU running time is less than the preset second CPU running time threshold.

[0139] Among them, the preset first storage space occupancy rate threshold is greater than the preset second storage space occupancy rate threshold, and the preset first CPU operation time threshold is greater than the preset second CPU operation time threshold.

[0140] In one embodiment, the first acquisition module 51 is specifically used to: obtain original monitoring data of the storage node from the registration system running in the registration device through the monitoring system, wherein the original monitoring data includes the original label of the storage node and the value of the monitoring indicator of the storage node; convert the original label of the storage node into the label of the storage node through the monitoring system, and use the converted monitoring data as the monitoring data; and store the monitoring data.

[0141] In one embodiment, the device further includes: a sending module, which is used to send the monitoring data to the display device through the monitoring system. The data visualization platform in the display device is used to visualize the monitoring data according to predefined display rules and with tags as granularity, and send warning information according to predefined warning rules when the monitoring data meets the warning rules.

[0142] The storage node operation trend prediction device provided in the embodiment of the present invention can execute the storage node operation trend prediction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] Figure 61 is a schematic diagram of the structure of an electronic device that implements the storage node operation trend prediction method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0144] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0145] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0146] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a storage node operation trend prediction method.

[0147] In some embodiments, the storage node operation trend prediction method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the storage node operation trend prediction method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the storage node operation trend prediction method in any other appropriate manner (e.g., by means of firmware).

[0148] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0150] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0153] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0154] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the storage node operation trend prediction method provided by any embodiment of the present invention.

[0155] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting storage node operation trends, characterized in that: Applied to a monitoring device, wherein a monitoring system is running on the monitoring device, the method comprises: Obtaining, through the monitoring system, monitoring data of a storage node from a registration system running in a registration device, and storing the monitoring data; wherein the storage node is a node pre-registered in the registration system, and the monitoring data includes a label of the storage node and a value of a monitoring indicator of the storage node; Obtaining an operation trend prediction request; wherein the operation trend prediction request includes a tag to be predicted and a monitoring indicator to be predicted; Determine, by the monitoring system, a value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator; According to the value of the first target monitoring indicator, an operation trend of the storage node with the to-be-predicted label based on the to-be-predicted monitoring indicator is determined.

2. The method according to claim 1, characterized in that Before obtaining the monitoring data of the storage node from the registration system running in the registration device through the monitoring system, the method further includes: The configuration file of the monitoring system is modified to register the registration system to the monitoring system; wherein the modified configuration file includes the address information of the registration system.

3. The method according to claim 1, characterized in that The monitoring data also includes the timestamp of the value of each monitoring indicator, and the operation trend prediction request also includes the prediction duration; The step of determining, by the monitoring system, a value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted according to the label of the storage node in the monitoring data and the monitoring indicator, comprises: Determining, by means of the monitoring system, candidate monitoring data matching the predicted duration from the timestamp of the value of the monitoring indicator; wherein the longer the predicted duration, the earlier the timestamp of the candidate monitoring data; According to the label of the storage node in the candidate monitoring data and the monitoring indicator, a value of a first target monitoring indicator that matches both the label to be predicted and the monitoring indicator to be predicted is determined.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining a query request sent by a query device; wherein the query request includes a tag to be queried and a calculation method of each monitoring indicator; Determining, by the monitoring system, a value of a second target monitoring indicator matching the tag to be queried according to the tag of the storage node in the monitoring data; For each monitoring indicator, according to the calculation method corresponding to the monitoring indicator, the value corresponding to the monitoring indicator in the value of the second target monitoring indicator is calculated to obtain a calculated value; The calculated value is fed back to the query device.

5. The method according to claim 4, characterized in that The monitoring indicators include: storage space occupancy rate and central processing unit CPU operation time, the storage space occupancy rate is calculated by determining a minimum value, the CPU operation time is calculated by determining an average value, and the calculated values ​​include the minimum value of the storage space occupancy rate and the average value of the CPU operation time; The method further comprises: If the minimum value of the storage space occupancy rate of each storage node with the tag to be queried belonging to the same storage cluster is greater than the preset first storage space occupancy rate threshold, and the average value of the CPU running time is greater than the preset first CPU running time threshold, it is determined that the storage cluster needs to add a storage node with the tag to be queried; If the minimum value of the storage space occupancy rate of each storage node with the tag to be queried belonging to the same storage cluster is less than the preset second storage space occupancy rate threshold, and the average value of the CPU running time is less than the preset second CPU running time threshold, it is determined that the storage cluster needs to reduce the storage node with the tag to be queried; Among them, the preset first storage space occupancy rate threshold is greater than the preset second storage space occupancy rate threshold, and the preset first CPU operation time threshold is greater than the preset second CPU operation time threshold.

6. The method according to any one of claims 1 to 5, characterized in that: The step of obtaining the monitoring data of the storage node from the registration system running in the registration device through the monitoring system and storing the monitoring data includes: Obtaining original monitoring data of the storage node from the registration system running in the registration device through the monitoring system; wherein the original monitoring data includes the original label of the storage node and the value of the monitoring indicator of the storage node; By means of the monitoring system, the original label of the storage node is converted into the label of the storage node, and the converted monitoring data is used as the monitoring data; The monitoring data is stored.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The monitoring data is sent to a display device through the monitoring system; wherein the data visualization platform in the display device is used to visualize the monitoring data based on predefined display rules and labels as granularity, and to send warning information when the monitoring data meets the warning rules based on predefined warning rules.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the storage node operation trend prediction method described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the storage node operation trend prediction method according to any one of claims 1 to 7 when executed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the storage node operation trend prediction method according to any one of claims 1 to 7.

Images