Early warning method and device for storage resource, electronic equipment and storage medium

Abstract

The application provides a kind of early warning method, device, electronic equipment, storage medium and data platform of storage resource, comprising: obtaining the resource usage of storage resource to be predicted, the resource usage to be predicted is predicted according to the prediction model set in advance, and early warning time is generated, wherein the prediction model is generated based on historical resource acquisition amount and historical resource acquisition time, and early warning operation is performed according to early warning time, the resource usage to be predicted is predicted by prediction model, early warning time is generated, which is equivalent to predicting the time when the usage of storage resource is a certain value, the storage resource may not be enough, so that the user can release the storage resource in advance and other processing, so as to avoid the disadvantages of insufficient storage resources in related technologies due to the user's late learning of the time when the storage resource is insufficient, so as to realize the accuracy and reliability of early warning operation, improve the technical effect of resource utilization of storage resource.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, cloud platform and cloud computing, and particularly relates to a storage resource early warning method and device, electronic equipment, storage medium and data platform. BACKGROUND

[0002] With the continuous development of economy, the scale of each enterprise is growing, and the enterprise data is growing explosively. In order to realize enterprise digitization, each enterprise has built a big data platform to store and analyze enterprise data.

[0003] In the prior art, the main storage resource early warning method includes setting a threshold of storage resources, and triggering an early warning of the storage resources when the storage resource usage reaches the threshold.

[0004] However, the inventors have found that the prior art at least has the following problems: the early warning of the amount of storage resources and the early warning threshold is a post-event warning, and the accuracy and reliability are low. SUMMARY

[0005] The present application provides a storage resource early warning method, device, electronic equipment, storage medium and data platform to solve the problem of low accuracy and reliability of early warning.

[0006] In one aspect, the present application provides a storage resource early warning method, comprising:

[0007] obtaining a to-be-predicted resource usage of storage resources;

[0008] predicting the to-be-predicted resource usage according to a pre-set prediction model to generate an early warning time, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times;

[0009] performing an early warning operation according to the early warning time.

[0010] In some embodiments, the prediction model is a one-dimensional linear regression prediction model, and the method further comprises:

[0011] determining model parameters according to the historical resource collection amounts and the historical resource collection times;

[0012] generating the one-dimensional linear regression prediction model according to the model parameters.

[0013] In some embodiments, the determination of the model parameters according to the historical resource collection amounts and the historical resource collection times comprises:

[0014] determining the covariance of the historical resource collection amounts and the historical resource collection times;

[0015] determining a variance of the historical resource collection times;

[0016] determining the model parameters according to the historical resource collection amounts, the historical resource collection times, the covariance and the variance.

[0017] In some embodiments, the model parameters include a coefficient term parameter and a constant term parameter, and the determining the model parameters according to the historical resource collection amounts, the historical resource collection times, the covariance and the variance includes:

[0018] determining the constant term parameter according to the covariance and the variance;

[0019] determining the coefficient term parameter according to the constant term parameter, the historical resource collection amounts and the historical resource collection times.

[0020] In some embodiments, the determining the coefficient term parameter according to the constant term parameter, the historical resource collection amounts and the historical resource collection times includes:

[0021] respectively determining a mean value corresponding to each of the historical resource collection amounts and the historical resource collection times;

[0022] determining the coefficient term parameter according to the mean values and the constant term parameter.

[0023] In some embodiments, the obtaining the to-be-predicted resource usage of the storage resource includes:

[0024] obtaining a resource quota of the storage resource;

[0025] determining the to-be-predicted resource usage according to a preset proportion threshold and the resource quota.

[0026] In some embodiments, the method further includes:

[0027] determining an exhaustion time of the resource quota according to the prediction model;

[0028] and the performing a warning operation according to the warning time includes performing the warning operation according to the warning time and the exhaustion time.

[0029] In some embodiments, the performing a warning operation according to the warning time includes:

[0030] determining a time difference value between the warning time and a current time;

[0031] if the time difference value is less than or equal to a preset time threshold, outputting warning information carrying the warning time.

[0032] In another aspect, the application provides a warning device for a storage resource, the device comprising:

[0033] an acquisition module configured to acquire a to-be-predicted resource usage of the storage resource;

[0034] a prediction module configured to predict the to-be-predicted resource usage according to a pre-set prediction model, and generate a warning time, wherein the prediction model is generated based on historical resource acquisition amounts and historical resource acquisition times;

[0035] a warning module configured to perform a warning operation according to the warning time.

[0036] In some embodiments, the prediction model is a one-dimensional linear regression prediction model, and the device further comprises:

[0037] a determination module configured to determine model parameters according to the historical resource acquisition amounts and the historical resource acquisition times;

[0038] a generation module configured to generate the one-dimensional linear regression prediction model according to the model parameters.

[0039] In some embodiments, the determination module is configured to determine a covariance of the historical resource acquisition amounts and the historical resource acquisition times, determine a variance of the historical resource acquisition times, and determine the model parameters according to the historical resource acquisition amounts, the historical resource acquisition times, the covariance, and the variance.

[0040] In some embodiments, the determination module is configured to determine the constant term parameter according to the covariance and the variance, and determine the coefficient term parameter according to the constant term parameter, the historical resource acquisition amounts, and the historical resource acquisition times.

[0041] In some embodiments, the determination module is configured to determine respective mean values of the historical resource acquisition amounts and the historical resource acquisition times, respectively, and determine the coefficient term parameter according to the mean values and the constant term parameter.

[0042] In some embodiments, the acquisition module is configured to acquire a resource quota of the storage resource, and determine the to-be-predicted resource usage according to a pre-set proportion threshold and the resource quota.

[0043] In some embodiments, the prediction module is configured to determine a use-up time of the resource quota according to the prediction model.

[0044] The warning module is configured to perform the warning operation according to the warning time and the use-up time.

[0045] In some embodiments, the early warning module is configured to determine a time difference between the early warning time and a current time, and output early warning information carrying the early warning time if the time difference is less than or equal to a preset time threshold.

[0046] In another aspect, the present application provides an electronic device, comprising: a memory, a processor;

[0047] The memory is configured to store instructions executable by the processor.

[0048] When the instructions in the memory are executed, the processor is configured to implement the method of any one of the above embodiments.

[0049] In another aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method of any one of the above embodiments.

[0050] In another aspect, the present application provides a data platform, comprising:

[0051] A communication module configured to receive a prediction request carrying a to-be-predicted resource usage amount sent by a user.

[0052] A storage module configured to store a prediction model, the prediction model being generated based on historical resource collection amounts and historical resource collection times.

[0053] A processing module configured to predict the to-be-predicted resource usage amount according to the prediction model, and generate an early warning time.

[0054] The communication module is further configured to perform an early warning operation according to the early warning time.

[0055] The present application provides an early warning method and device for storage resources, an electronic device, a storage medium, and a data platform, comprising: obtaining a to-be-predicted resource usage amount of a storage resource, predicting the to-be-predicted resource usage amount according to a prediction model, and generating an early warning time, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times, performing an early warning operation according to the early warning time, predicting the to-be-predicted resource usage amount according to the prediction model, and generating an early warning time, which is equivalent to predicting the time when the storage resource usage amount is a certain value, and the storage resource may not be sufficient, so that the user can release the storage resource in advance and perform other processing, thereby avoiding the disadvantages in the related art, such as the storage resource not being sufficient due to the user learning the time when the storage resource is not sufficient, and improving the accuracy and reliability of the early warning operation and the technical effect of improving the resource utilization rate of the storage resource. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0057] Figure 1 The framework diagram of the early warning system of the storage resource of the embodiment of the present application;

[0058] Figure 2 The flowchart of the early warning method of the storage resource of one embodiment of the present application;

[0059] Figure 3 The flowchart of the early warning method of the storage resource of another embodiment of the present application;

[0060] Figure 4 The prediction analysis diagram of the embodiment of the present application;

[0061] Figure 5 The schematic diagram of the early warning device of the storage resource of one embodiment of the present application;

[0062] Figure 6 The schematic diagram of the early warning device of the storage resource of another embodiment of the present application;

[0063] Figure 7 The block diagram of the electronic device of the embodiment of the present disclosure;

[0064] Figure 8 The schematic diagram of the data platform of the embodiment of the present application.

[0065] Through the above-mentioned drawings, the specific embodiments of the present disclosure have been shown, and more detailed descriptions will be given hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure by any means, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0067] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed as interchangeable in order to comfort with the context of the description and the claims. Furthermore, the terms "comprising", "having", "including", and "containing" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises, has, includes or contains an item or list of items who does not also preclude any other item or list of items from being present or added to the process, method, system, product or apparatus. The terms "comprise", "comprising", "comprises", "including", "includes" and "contain" or "containing" are used interchangeably and mean the same thing.

[0068] Referring to Figure 1 , Figure 1 The framework diagram of the early warning system of the storage resource of the embodiments of the present application (hereinafter referred to as the early warning system) is shown in the figure. The early warning method of the storage resource of the embodiments of the present application can be applied to the early warning system as shown in Figure 1 .

[0069] As shown in Figure 1 , the early warning system comprises a data platform, data collection, a storage resource prediction engine, an early warning API and a user.

[0070] The data platform can be a big data platform, a data platform, a cloud platform, etc. The type of the data platform is not limited in the embodiments of the present application. As shown in Figure 1 , the data platform can be a Hadoop cluster platform supporting functions such as computing and Hadoop Distributed File System (HDFS) storage. The data platform can provide storage resources for users based on the distributed file system storage function to store data of the users, and can perform corresponding calculations on the storage resources stored based on the distributed file system storage function, such as calculating the occupied storage resources (i.e. the usage of the storage resources).

[0071] The data collection can be used to collect information related to the storage resources, such as collecting usage information of the storage resources, which can be the usage of the storage resources or the remaining amount of the storage resources, etc., or collecting quota information of the storage resources (i.e. the resource quota of the storage resources allocated to the user by the data platform), etc.

[0072] The storage resource early warning engine can be used to implement storage resource early warning analysis. For example, the storage resource early warning engine can perform early warning analysis on the storage resource based on the storage resource quota information and the storage resource usage information, and generate an early warning task. In some embodiments, the storage resource early warning engine can assign storage resource early warning tasks of different users in parallel, such as tenant storage resource early warning tasks and platform storage resource early warning tasks.

[0073] The early warning application programming interface (API) can be used to send the early warning information generated by the storage resource early warning engine to the user. For example, as shown in FIG. 1, the early warning API can send the early warning information to the platform tenant A if the early warning information corresponds to the platform tenant A, send the early warning information to the platform tenant B if the early warning information corresponds to the platform tenant B, and send the early warning information to the platform administrator if the early warning information corresponds to the platform administrator. Figure 1

[0074] The user can include a user of the data platform, such as a tenant, and can also include an administrator of the data platform. For example, the user can be the platform tenant A, the platform tenant B, and the platform administrator as shown in FIG. 1. It should be understood that the user shown in FIG. 1 is only used to exemplarily illustrate the user that can exist in the data platform, and cannot be understood as a limitation on the number and / or type of the user. Figure 1 Figure 1

[0075] It should be noted that the early warning system shown in FIG. 1 is only used to exemplarily illustrate the components that can be included in the early warning system, and the forms, manners, and numbers of the components that can exist in the early warning system, and cannot be understood as a limitation on the components and the early warning system. For example, in some embodiments, the components of the early warning system can be increased accordingly, and for another example, in some other embodiments, the components of the early warning system can be reduced accordingly, and the embodiments of the present application are not limited. Figure 1 In the related art, the main early warning method of the storage resource includes comparing the usage of the storage resource with a pre-set threshold value, and if the usage of the storage resource reaches the threshold value, performing an early warning operation.

[0076] However, through the solution in the related art, the early warning can not be timely, which can cause the problem of insufficient storage resource.

[0077] The inventors of the present application have obtained the inventive concept of predicting the early warning time after creative labor, thereby realizing early warning.

[0078]

[0079] ​​​​The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0080] According to one aspect of the embodiments of the present application, the embodiments of the present application provide a warning method of storage resources, wherein the warning method of storage resources of the embodiments of the present application can be applied to a warning system as shown in Figure 1 .

[0081] Please refer to Figure 2 , Figure 2 the flowchart of the warning method of storage resources of one embodiment of the present application.

[0082] As shown in Figure 2 , the method comprises:

[0083] S101: Obtain a to-be-predicted resource usage of storage resources.

[0084] Wherein, the execution subject of the warning method of storage resources of the embodiments of the present application can be a warning device of storage resources (hereinafter referred to as a warning device), and the warning device can be a storage resource warning engine as shown in Figure 1 , and the warning device can also be a computer and a server (including a local server and a cloud server), etc.

[0085] Wherein, the to-be-predicted resource usage can be used to represent the usage of the storage resources predicted by the warning time set in advance, or can be used to represent the current usage of the storage resources.

[0086] For example, if the to-be-predicted resource usage is the usage of the storage resources predicted by the warning time set in advance, it can be understood that the warning device can continuously update the prediction of the warning time of the predicted resource usage.

[0087] For another example, if the to-be-predicted resource usage is the current usage of the storage resources, it can be understood that the warning device can predict the warning time of the usage of different storage resources.

[0088] S102: Predict the to-be-predicted resource usage according to a prediction model set in advance, and generate a warning time, wherein the prediction model is generated based on historical resource collection amount and historical resource collection time.

[0089] Wherein, the historical resource collection amount can be used to represent the usage of the storage resources collected before the current time, and the historical resource collection time can be used to represent the time when the historical resource collection amount is collected.

[0090] That is, in the embodiment of the present application, the usage amount of the storage resource before the current time can be collected, and the time of collecting the usage amount of the storage resource can be collected, and a prediction model is generated according to the usage amount of the storage resource before the current time and the corresponding time.

[0091] The prediction model can be used to represent a model for predicting the warning time of the usage amount of the storage resource based on the usage amount of the storage resource. For example, when the usage amount of the storage resource to be predicted is the resource usage amount to be predicted, the warning device can predict the warning time of the resource usage amount to be predicted based on the prediction model, that is, the warning device can predict that in the case of the resource usage amount to be predicted, at which time point the remaining amount of the storage resource of a certain user will not be able to meet the use of the user, or at which time point the remaining amount of the storage resource of the data platform will not be able to meet the allocation, etc.

[0092] For example, when the warning device learns that the resource usage amount to be predicted is y, the prediction model can generate a warning time x based on y, x can be used to represent the time point when the resource usage amount to be predicted is used up, or the time point when the resource usage amount to be predicted is about to be used up (which can be set based on demand, historical records and experiments, etc.), wherein the use up in the present example can be understood as use up at the user level, or use up at the data platform level, which is not limited in the embodiment of the present application.

[0093] In the embodiment of the present application, the prediction model is used to predict the resource usage amount to be predicted, and the warning time is generated, which is equivalent to predicting the time when the usage amount of the storage resource is a certain value, and the storage resource may not be sufficient. Therefore, the user can perform release of the storage resource in advance, thereby avoiding the disadvantages of the related art that the user learns the time when the storage resource is not sufficient, and improving the accuracy and reliability of the warning operation, and improving the resource utilization rate of the storage resource.

[0094] S103: performing a warning operation according to the warning time.

[0095] The warning operation can be used to represent a warning strategy performed by the warning device according to the warning time, such as the warning device feeding back the warning time to the user.

[0096] For example, in the warning system as shown in Figure 1 , the storage resource prediction engine (i.e. the warning device) can send the warning time to the platform tenant A (or the platform tenant B, or the platform manager) through the warning API.

[0097] In order for the reader to have a more thorough understanding of the method for generating the prediction model of the present application, the embodiment of the present application will be described in detail in combination with Figure 3 . Among them,Figure 3 A flowchart of a storage resource early warning method according to another embodiment of the present application.

[0098] As shown in Figure 3 The method comprises the following steps.

[0099] S201: Obtain historical resource collection quantity and historical resource collection time.

[0100] Based on the above example, the historical resource collection quantity and the historical resource collection time are the storage resource usage collected in a certain period of time before the current time (the time period can be set by the early warning device based on demand, historical records and tests, and the present application embodiment is not limited) and the corresponding collection time.

[0101] In the present application embodiment, in order to avoid tedious repetition, the historical resource collection quantity and the historical resource collection time are exemplarily described taking platform tenant A as an example.

[0102] Among them, the time period can be the first week to the 50th week of 2019, that is, the historical resource collection time obtained by the early warning device includes any week from the first week to the 50th week of 2019, and the historical resource collection quantity obtained by the early warning device includes the storage resource usage corresponding to each week, which can be referred to Table 1:

[0103] Week Usage of storage resources Week Usage of storage resources 2019W01 452.22 2019W26 637.56 2019W02 461.60 2019W27 674.54 2019W03 469.28 2019W28 689.04 2019W04 468.15 2019W29 717.19 2019W05 458.48 2019W30 701.97 2019W06 439.31 2019W31 714.60 2019W07 425.28 2019W32 743.71 2019W08 411.11 2019W33 771.15 2019W09 403.99 2019W34 774.87 2019W10 426.46 2019W35 753.75 2019W11 437.00 2019W36 751.22 2019W12 436.27 2019W37 749.51 2019W13 456.21 2019W38 753.73 2019W14 459.37 2019W39 766.58 2019W15 471.92 2019W40 768.67 2019W16 456.58 2019W41 795.94 2019W17 482.65 2019W42 796.21 2019W18 466.10 2019W43 789.36 2019W19 480.07 2019W44 798.39 2019W20 485.56 2019W45 810.35 2019W21 426.83 2019W46 797.55 2019W22 577.05 2019W47 807.64 2019W23 582.92 2019W48 807.67 2019W24 594.04 2019W49 788.53 2019W25 614.43 2019W50 796.57

[0104] In some embodiments, when the storage resource early warning method of the present application embodiment is applied to the early warning system as shown in Figure 1 The method for obtaining the historical resource collection quantity can be realized by the following way:

[0105] The storage resource usage information of tenant A of the data platform is recorded in the storage image file FsImag of the hadoop hdfs file system, and the storage location of the image file can be obtained through the following configuration item in the hadoop configuration file:

[0106] <name>dfs.namenode.name.dir< / name>

[0107] The data platform parses the storage image file FsImag by using the parsing class PBImageXmlWriter, and stores the parsed information for subsequent use.

[0108] For example, the storage image file FsImag contains file name, path, user, last access date, file size and file copy number, etc. The amount of storage resource used by tenant A (i.e. the historical resource collection quantity) can be calculated according to the file sum (file size * file copy number).

[0109] It is worth mentioning that the historical resource collection time in the embodiments of the present application is exemplarily described by taking a natural week as an example, and in other embodiments, the natural day, natural month, or other date frequencies can be used to collect and predict the usage of the stored resources (i.e., the historical resource collection amount) according to the needs, and the embodiments of the present application are not limited.

[0110] S202: Determine the model parameters according to the historical resource collection amount and the historical resource collection time.

[0111] S202 can include the following steps:

[0112] S2021: Determine the covariance of the historical resource collection amount and the historical resource collection time.

[0113] In the above example, y represents the historical resource collection amount, and x represents the historical resource collection time, and the covariance of the historical resource collection amount and the historical resource collection time can be represented by cov(x, y).

[0114] S2022: Determine the variance of the historical resource collection time.

[0115] Based on the above example, the variance of the historical resource collection time can be represented by var(x).

[0116] S2023: Determine the constant term parameter according to the covariance and the variance.

[0117] Based on the above example, if the constant term parameter is represented by b, then b = cov(x, y) / var(x).

[0118] S2024: Determine the mean values of the historical resource collection amount and the historical resource collection time, respectively.

[0119] That is, the early warning device can determine the mean value of the historical resource collection amount, and can determine the mean value of the historical resource collection time.

[0120] Based on the above example, the mean value of the historical resource collection amount can be represented by avg(y), and the mean value of the historical resource collection time can be represented by avg(x).

[0121] S2025: Determine the coefficient term parameter according to the mean values and the constant term parameter.

[0122] In the above example, the coefficient term parameter can be represented by a, and a = avg(y) - b * avg(x).

[0123] S203: Generate a linear regression prediction model according to the model parameters.

[0124] Wherein, the model parameters include constant term parameter b and coefficient term parameter a, the monomial linear regression prediction model can be represented by y=ax+b, and according to Table 1, a=10.74, b=350.19, so the monomial linear regression prediction model can be y=10.74*x+350.19.

[0125] In some embodiments, the monomial linear regression prediction model can be solved based on the Python programming language and the scientific computing extension library NumPy (Numerical Python), of course, it can also be solved by other ways, and the embodiments of the present application are not limited.

[0126] S204: Obtain the resource quota of the storage resource.

[0127] Wherein, the resource quota can be used to represent the usage of the available storage resource allocated to the user, in combination with the early warning system shown in Figure 1 , the resource quota can be the usage of the available storage resource allocated to the tenant A by the data platform.

[0128] In some embodiments, when the early warning method of the storage resource of the embodiments of the present application is applied to the early warning system as shown in Figure 1 , the obtaining of the resource quota can be realized by the following method:

[0129] Through the hadoop command, the hdfs directory quota size of each tenant is queried and stored, for example, when the hadoop directories of all tenants are under the user directory, the resource quota can be obtained by hdfs dfs-count-q-h / user / username.

[0130] S205: Determine the to-be-predicted resource usage according to the pre-set proportion threshold and the resource quota.

[0131] Wherein, the proportion threshold can be set by the early warning device based on demand, historical records and experiments, etc.

[0132] In some embodiments, the proportion threshold can be set to 0.8, that is, the to-be-predicted resource usage=0.8*resource quota, as shown in Figure 4 , if the resource quota is 1000TB, the to-be-predicted resource usage can be 800TB.

[0133] S206: Predict the to-be-predicted resource usage according to the monomial linear regression prediction model, and generate the early warning time.

[0134] Wherein, the description of generating the monomial linear regression prediction model can be referred to S201 to S203, and the part of the description of S206 can be referred to S102, which will not be repeated here.

[0135] Based on the above example, the univariate linear regression prediction model can be represented as y=ax+b, and based on Table 1, the univariate linear regression prediction model can be y=10.74*x+350.19. In the embodiments of the present application, when the to-be-predicted resource usage is determined, i.e., y is determined, x can be solved based on the univariate linear regression prediction model y=10.74*x+350.19, and x is the warning time, i.e., x is the time when the to-be-predicted resource usage is likely to be exhausted.

[0136] S207: performing a warning operation according to the warning time.

[0137] The description of S207 can be referred to S103, and details are not described herein.

[0138] In some embodiments, the exhaustion time of the resource quota can also be predicted based on the univariate linear regression prediction model, i.e., the warning time can be predicted based on the univariate linear regression prediction model, and the exhaustion time can also be obtained. When the warning device performs S207, the warning device can perform a warning operation according to the warning time and the exhaustion time.

[0139] For example, in combination with the warning system shown in Figure 1 , the storage resource prediction engine can send a warning operation carrying the warning time and the exhaustion time to the user through the warning API.

[0140] In some embodiments, S207 can include:

[0141] S2071: determining a time difference value between the warning time and the current time.

[0142] For example, the warning device obtains the current time and calculates the time difference value between the warning time and the current time, i.e., time difference value=warning time-current time.

[0143] S2072: if the time difference value is less than or equal to a pre-set time threshold, outputting warning information carrying the warning time.

[0144] The time threshold can be set by the warning device based on demand, historical records, and experiments, etc.

[0145] This step can specifically include: the warning device determines whether the time difference value is less than or equal to the time threshold, and if yes, outputs the warning information.

[0146] That is, if the time difference value between the warning time and the current time is relatively small, i.e., the current time is about to reach the warning time, the warning device outputs the warning information, and if the time difference value between the warning time and the current time is relatively large, i.e., the current time is far from the warning time, the warning device can temporarily not output the warning information.

[0147] The warning information may include the warning time, and may also include the current storage resource usage, the current remaining storage resource, etc. This application embodiment does not limit the scope of the warning information.

[0148] Combination Figure 1 The warning system shown here provides a detailed explanation of the output warning information as follows:

[0149] The storage resource prediction engine can send alert information to users (such as tenant A) via SMS and / or email through an alert API that calls SMS and / or email services.

[0150] The warning message (i.e., SMS and / or email) may contain the following: Based on the current storage resource usage trend, the resource quota is expected to be reached on November 20, 2020. Please manage storage resources and / or increase the resource quota in a timely manner. The current storage resource usage is 89.65% of the resource quota.

[0151] In some embodiments, the warning information (such as email) may include, for example, Figure 4 The predictive analysis chart shown is also sent to the user so that the user can more intuitively understand the current storage resource usage and other information.

[0152] According to another aspect of the embodiments of this application, the embodiments of this application also provide a storage resource early warning device, used to execute the storage resource early warning method as described in any of the above embodiments, such as executing... Figure 2 or Figure 3 The method for early warning of storage resources is shown.

[0153] Please see Figure 5 , Figure 5 This is a schematic diagram of a storage resource early warning device according to an embodiment of this application.

[0154] like Figure 5 As shown, the device includes:

[0155] The acquisition module 11 is used to acquire the predicted resource usage of storage resources;

[0156] Prediction module 12 is used to predict the usage of the resource to be predicted according to a pre-set prediction model and generate an early warning time, wherein the prediction model is generated based on historical resource collection volume and historical resource collection time.

[0157] The early warning module 13 is used to perform an early warning operation based on the early warning time.

[0158] CombinationFigure 6 It can be known that, in some embodiments, the prediction model is a one-dimensional linear regression prediction model, and the device further comprises:

[0159] The determination module 14 is configured to determine model parameters according to the historical resource collection amount and the historical resource collection time;

[0160] The generation module 15 is configured to generate the one-dimensional linear regression prediction model according to the model parameters.

[0161] In some embodiments, the determination module 14 is configured to determine a covariance of the historical resource collection amount and the historical resource collection time, determine a variance of the historical resource collection time, and determine the model parameters according to the historical resource collection amount, the historical resource collection time, the covariance, and the variance.

[0162] In some embodiments, the determination module 14 is configured to determine the constant term parameter according to the covariance and the variance, and determine the coefficient term parameter according to the constant term parameter, the historical resource collection amount, and the historical resource collection time.

[0163] In some embodiments, the determination module 14 is configured to determine a mean value corresponding to each of the historical resource collection amount and the historical resource collection time, respectively, and determine the coefficient term parameter according to the mean value and the constant term parameter.

[0164] In some embodiments, the acquisition module 11 is configured to acquire a resource quota of the storage resource, and determine the to-be-predicted resource usage according to a pre-set proportion threshold and the resource quota.

[0165] In some embodiments, the prediction module 12 is configured to determine a use-up time of the resource quota according to the prediction model.

[0166] The warning module 13 is configured to perform the warning operation according to the warning time and the use-up time.

[0167] In some embodiments, the warning module 13 is configured to determine a time difference value between the warning time and a current time, and output warning information carrying the warning time if the time difference value is less than or equal to a pre-set time threshold.

[0168] According to another aspect of the embodiments of the present application, the embodiments of the present application further provide an electronic device, comprising: a memory, a processor;

[0169] The memory is configured to store a memory of processor-executable instructions;

[0170] When the instructions in the memory are executed, the processor is configured to implement the method as claimed in any one of the preceding embodiments, such as to implement the method for pre-warning of storage resources as shown in Figure 2 or Figure 3 the pre-warning method of storage resources.

[0171] Referring to Figure 7 , Figure 7 is a block diagram of an electronic device according to an embodiment of the present application.

[0172] As shown in Figure 7 , the electronic device is intended to represent various forms including digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0173] As shown in Figure 7 , the electronic device includes one or more processors 101, a memory 102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected using different buses, and can be mounted on a common motherboard or in other manners, as desired. The processor can process instructions for execution within the electronic device, including instructions stored in the memory or on the memory to display graphical information for a GUI on an external input / output device, such as a display device coupled to the interface. In other implementations, multiple processors and / or multiple buses can be employed as desired to implement the described functionality, and a plurality of memories and types of memory can be used. Also, various items of the described functionality can be implemented with hardware alone, software alone, or a combination of both. Figure 7 The processor 101 is taken as an example in

[0174] The memory 102 is a non-transitory computer-readable storage medium provided by the present application. The memory stores instructions executable by at least one processor, so that the at least one processor executes the pre-warning method of storage resources provided by the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to execute the pre-warning method of storage resources provided by the present application.

[0175] The memory 102, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules in the embodiments of the present application. The processor 101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 102, that is, implements the pre-warning method of the storage resource in the above-mentioned method embodiments.

[0176] The memory 102 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; and the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 102 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 102 can optionally include a memory remotely arranged with respect to the processor 101, and these remote memories can be connected to the electronic device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a block-chain-based service network (BSN), a mobile communication network, and combinations thereof.

[0177] The electronic device can further include an input device 103 and an output device 104. The processor 101, the memory 102, the input device 103 and the output device 104 can be connected by a bus or other means, Figure 7 For example, by way of bus connection.

[0178] The input device 103 can receive inputted digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 104 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), etc. The display device can include but is not limited to a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0179] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0180] These computer programs (also known as programs, software, software applications or code) include machine instructions for the programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0181] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0182] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a block-chain-based service network (BSN), a wide area network (WAN), and the Internet.

[0183] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0184] In other words, as will be appreciated by one skilled in the art, embodiments of the present application can be comprised of various computer-readable media, various memories, logic devices, and / or various processors. Computer-readable media can include computer storage media and communication media. Computer storage media can include volatile and non-volatile, removable and non-removable media implemented in a method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The system memory can include various types of memory modules, such as static random access memory (SRAM), dynamic random access memory (DRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other type of memory. The system memory can be used to store various modules and data structures used in connection with embodiments of the present application, such as the operating system, one or more applications, program modules, program data, and the like. In one embodiment, the system memory can also include a computer-readable medium comprising pre-boot execution environment (PXE) enabled media. The communication media can comprise any medium that facilitates the transfer of a computer program from one place to another. For example, a communication medium can include a modem, a wired / wireless communication network, or the like. The foregoing examples of storage media and communication media are not meant to limit the scope of the application.

[0185] According to another aspect of the embodiments of the present application, the embodiments of the present application further provide a data platform.

[0186] Please refer to Figure 8 , Figure 8 for a schematic diagram of the data platform of the embodiments of the present application.

[0187] As Figure 8 shown, the data platform comprises:

[0188] The communication module 21 is configured to receive a prediction request sent by a user and carrying a to-be-predicted resource usage amount.

[0189] The storage module 22 is configured to store a prediction model, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times.

[0190] The processing module 23 is configured to predict the resource usage according to the prediction model, and generate a warning time;

[0191] The communication module 21 is further configured to perform a warning operation according to the warning time.

[0192] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes ROM, RAM, magnetic disk or optical disk and various storage media that can store program codes.

[0193] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A pre-warning method of a storage resource, characterized in that, The method comprises: acquiring a to-be-predicted resource usage of a storage resource; the to-be-predicted resource usage is determined based on a resource quota of the storage resource and a pre-set proportion threshold; predicting the to-be-predicted resource usage according to a pre-set prediction model to generate a warning time, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times; performing a warning operation according to the warning time; the prediction model is a one-variable linear regression prediction model, and the method further comprises: determining a covariance of the historical resource collection amounts and the historical resource collection times; determining a variance of the historical resource collection times; determining a model parameter according to the historical resource collection amounts, the historical resource collection times, the covariance and the variance; generating the one-variable linear regression prediction model according to the model parameter; the model parameter comprises a coefficient term parameter and a constant term parameter, and the determination of the model parameter according to the historical resource collection amounts, the historical resource collection times, the covariance and the variance comprises: determining the constant term parameter according to the covariance and the variance; determining respective mean values corresponding to the historical resource collection amounts and the historical resource collection times respectively; determining the coefficient term parameter according to the mean values and the constant term parameter; the historical resource collection amounts are acquired by: acquiring usage information of the storage resource from a storage image file FsImag of a hadoop hdfs file system, and parsing the storage image file FsImag by using a parsing class PBImageXmlWriter to obtain the historical resource collection amounts after the parsing.

2. The method of claim 1, wherein, the acquisition of the to-be-predicted resource usage of the storage resource comprises: acquiring a resource quota of the storage resource; determining the to-be-predicted resource usage according to a pre-set proportion threshold and the resource quota.

3. The method of claim 2, wherein, the method further comprises: determining a use-up time of the resource quota according to the prediction model; and the performance of the warning operation according to the warning time comprises: performing the warning operation according to the warning time and the use-up time.

4. The method of claim 1, wherein, the performance of the warning operation according to the warning time comprises: determining a time difference value between the warning time and a current time; if the time difference value is less than or equal to a pre-set time threshold, outputting warning information carrying the warning time.

5. A pre-warning device of a storage resource, characterized in that, the apparatus comprises: an acquisition module, configured to acquire a to-be-predicted resource usage of a storage resource; the to-be-predicted resource usage is determined based on a resource quota of the storage resource and a pre-set proportion threshold; a prediction module, configured to predict the to-be-predicted resource usage according to a pre-set prediction model to generate a warning time, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times; a warning module, configured to perform a warning operation according to the warning time. The prediction model is a one-dimensional linear regression prediction model, the determining module is configured to determine a covariance of the historical resource collection amount and the historical resource collection time, determine a variance of the historical resource collection time, and determine a model parameter according to the historical resource collection amount, the historical resource collection time, the covariance, and the variance; The generating module is configured to generate the one-dimensional linear regression prediction model according to the model parameter; The model parameter includes a coefficient term parameter and a constant term parameter, and the determining module is specifically configured to determine the constant term parameter according to the covariance and the variance, determine respective mean values of the historical resource collection amount and the historical resource collection time, and determine the coefficient term parameter according to the mean values and the constant term parameter. The historical resource collection amount is obtained by obtaining usage information of a storage resource from a storage image file FsImag in a hadoop hdfs file system, and parsing the storage image file FsImag by using a parsing class PBImageXmlWriter to obtain the historical resource collection amount after the parsing.

6. An electronic device, comprising: Comprise: A memory and a processor; The memory is configured to store processor-executable instructions; When the instructions in the memory are executed, the processor is configured to implement the method in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method in any one of claims 1 to 4.

8. A data platform, characterized by The data platform comprises: A communication module configured to receive a prediction request carrying a to-be-predicted resource usage amount sent by a user, wherein the to-be-predicted resource usage amount is determined based on a resource quota of a storage resource and a pre-set proportion threshold; A storage module configured to store a prediction model, wherein the prediction model is generated based on historical resource collection amounts and historical resource collection times, the prediction model is a one-dimensional linear regression prediction model, and a model parameter of the prediction model is determined based on a covariance of the historical resource collection amounts and the historical resource collection times and a variance determined based on the historical resource collection times; A processing module configured to predict the to-be-predicted resource usage amount according to the prediction model and generate an early warning time; The communication module is further configured to perform an early warning operation according to the early warning time; The model parameter includes a coefficient term parameter and a constant term parameter, the constant term parameter is determined according to the covariance and the variance, the coefficient term parameter is determined according to a mean value and the constant term parameter, and the mean value is a respective mean value of the historical resource collection amount and the historical resource collection time; The historical resource collection amount is obtained by obtaining usage information of a storage resource from a storage image file FsImag in a hadoop hdfs file system, and parsing the storage image file FsImag by using a parsing class PBImageXmlWriter to obtain the historical resource collection amount after the parsing.

Images