Prediction model training method and device, resource management method and device

Abstract

The present disclosure provides a prediction model training method and device, and a resource management method and device. The prediction model training method includes: labeling first sample data using a preset labeling rule to obtain second sample data; training a machine learning model using the second sample data; inputting the first sample data into the trained machine learning model to obtain a prediction result; updating the preset labeling rule according to the prediction result to obtain the current labeling rule; labeling the first sample data using the current labeling rule to obtain a third sample data; and training the trained machine learning model using the third sample data to obtain a prediction model.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing, and in particular to a prediction model training method and device, and a resource management method and device. Background Art

[0002] The scale of existing machine resources is getting larger and larger, and business components are becoming more and more complex, resulting in geometric growth in the complexity of operation and maintenance. Human resources can no longer effectively support business development (stability, efficiency, cost, and security), so the demand for intelligent operation and maintenance is becoming more and more urgent. Cost optimization and resource optimization are important parts of intelligent operation and maintenance, including resource recovery and capacity prediction. Through reasonable use, enterprises can save a lot of costs and avoid resource waste. At present, the resource optimization platform mainly uses the rule base to classify and judge the acquired resource indicator data. Summary of the invention

[0003] The inventors have found through research that in the prior art, the timeliness of classifying and judging the acquired resource indicator data through a cumbersome rule base cannot be effectively guaranteed, and the complexity and labor cost of maintaining the rule base are very high.

[0004] Based on this, the present disclosure provides a resource management solution, which realizes intelligent classification of acquired resource indicator data through machine learning algorithms, thereby effectively improving operation and maintenance management efficiency.

[0005] According to a first aspect of an embodiment of the present disclosure, a prediction model training method is provided, including: labeling first sample data using a preset labeling rule to obtain second sample data; training a machine learning model using the second sample data; inputting the first sample data into the trained machine learning model to obtain a prediction result; updating the preset labeling rule according to the prediction result to obtain a current labeling rule; labeling the first sample data using the current labeling rule to obtain a third sample data; and training the trained machine learning model using the third sample data to obtain a prediction model.

[0006] In some embodiments, erroneous labels in the second sample data are corrected according to the prediction results, and incremental sample data are generated according to the correction content; using the third sample data to train the trained machine learning model to obtain a prediction model includes: using the third sample data and the incremental sample data to train the trained machine learning model to obtain a prediction model.

[0007] In some embodiments, after obtaining the prediction model, the first sample data is input into the prediction model to obtain a prediction result; the current labeling rules are updated according to the prediction result, the erroneous labels in the third sample data are corrected according to the prediction result, and incremental sample data is generated according to the correction content; and the step of labeling the first sample data using the current labeling rules is repeated until the training end conditions are met.

[0008] In some embodiments, the first sample data includes container sample data or physical machine sample data.

[0009] According to the second aspect of an embodiment of the present disclosure, a prediction model training device is provided, including: a first processing module, configured to label first sample data using a preset labeling rule to obtain second sample data; a second processing module, configured to train a machine learning model using the second sample data; a third processing module, configured to input the first sample data into the trained machine learning model to obtain a prediction result; a fourth processing module, configured to update the preset labeling rule according to the prediction result to obtain the current labeling rule; a fifth processing module, configured to label the first sample data using the current labeling rule to obtain a third sample data, and train the trained machine learning model using the third sample data to obtain a prediction model.

[0010] According to a third aspect of an embodiment of the present disclosure, a prediction model training device is provided, comprising: a memory configured to store instructions; a processor coupled to the memory, the processor being configured to execute a method as described in any of the above embodiments based on the instructions stored in the memory.

[0011] According to a fourth aspect of an embodiment of the present disclosure, a resource management method is provided, comprising: inputting container indicator data into a first prediction model to obtain a first prediction result, wherein the first prediction model is obtained by processing container sample data using the training method described in any of the above embodiments; aggregating container static data with the first prediction result to obtain a first processing result; inputting physical machine indicator data into a second prediction model to obtain a second prediction result, wherein the second prediction model is obtained by processing physical machine sample data using the training method described in any of the above embodiments; aggregating physical machine static data with the second prediction result to obtain a second processing result; and merging the first processing result and the second processing result to obtain a resource summary result.

[0012] According to a fifth aspect of an embodiment of the present disclosure, a resource management device is provided, comprising: a first management module, configured to input container indicator data into a first prediction model to obtain a first prediction result, wherein the first prediction model is obtained by processing container sample data using the training method described in any of the above embodiments; a second management module, configured to aggregate container static data with the first prediction result to obtain a first processing result; a third management module, configured to input physical machine indicator data into a second prediction model to obtain a second prediction result, wherein the second prediction model is obtained by processing physical machine sample data using the training method described in any of the above embodiments; a fourth management module, configured to aggregate physical machine static data with the second prediction result to obtain a second processing result; and a fifth management module, configured to merge the first processing result and the second processing result to obtain a resource summary result.

[0013] According to a sixth aspect of an embodiment of the present disclosure, there is provided a resource management device, comprising: a memory configured to store instructions; a processor coupled to the memory, the processor being configured to execute a method as described in any of the above embodiments based on the instructions stored in the memory.

[0014] According to a seventh aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method involved in any of the above embodiments is implemented.

[0015] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 A flowchart of a prediction model training method according to an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of a prediction model training method according to another embodiment of the present disclosure;

[0019] Figure 3 A schematic diagram of the structure of a prediction model training device according to an embodiment of the present disclosure;

[0020] Figure 4A schematic diagram of the structure of a prediction model training device according to another embodiment of the present disclosure;

[0021] Figure 5 A schematic diagram of a resource management method according to an embodiment of the present disclosure;

[0022] Figure 6 A schematic diagram of the structure of a resource management device according to an embodiment of the present disclosure;

[0023] Figure 7 A schematic diagram of the structure of a resource management device according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0025] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0026] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0027] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0028] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0029] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0030] Figure 1 The following is a flow chart of a prediction model training method according to an embodiment of the present disclosure. In some embodiments, the following prediction model training method is performed by a prediction model training device.

[0031] In step 101, first sample data is labeled using a preset labeling rule to obtain second sample data.

[0032] For example, the first sample data is container sample data or physical machine sample data. The first sample data does not include label information.

[0033] In step 102, the machine learning model is trained using the second sample data.

[0034] In step 103, the first sample data is input into the trained machine learning model to obtain a prediction result.

[0035] In step 104, the preset labeling rules are updated according to the prediction results to obtain the current labeling rules.

[0036] In some embodiments, erroneous labels in the second sample data are corrected according to the prediction result, and incremental sample data is generated according to the corrected content.

[0037] In step 105, the first sample data is labeled using the current labeling rule to obtain third sample data.

[0038] In step 106, the trained machine learning model is trained using the third sample data to obtain a prediction model.

[0039] In some embodiments, the trained machine learning model is trained using the third sample data and the incremental sample data to obtain a prediction model.

[0040] In some embodiments, after obtaining the prediction model, the first sample data is input into the prediction model to obtain a prediction result. The current labeling rules are updated according to the prediction result, the wrong labels in the third sample data are corrected according to the prediction result, and the incremental sample data is generated according to the correction content. The step of labeling the first sample data using the current labeling rules is repeatedly performed until the training end condition is met.

[0041] The above embodiments of the present disclosure are described below through specific examples.

[0042] like Figure 2 As shown, in the offline training part, the first sample data is labeled using a preset labeling rule to obtain the second sample data. The machine learning model is trained using the second sample data, and the obtained model parameters are provided to the prediction model in the online estimation scoring.

[0043] The prediction model processes the first sample data to obtain a prediction result. For example, the prediction result is idle, low utilization, normal, and high load. Next, after confirmation by the user, the preset labeling rules are updated according to the prediction result to obtain the current labeling rules. And according to the prediction result, the wrong labeling in the second sample data is corrected, and the incremental sample data is generated according to the correction content.

[0044] Next, the first sample data is labeled using the current labeling rule to obtain the third sample data. The trained machine learning model is trained using the third sample data and the incremental sample data to obtain a prediction model.

[0045] After obtaining the prediction model, the first sample data is input into the prediction model to obtain the prediction result. Next, after confirmation by the user, the preset labeling rules are updated according to the prediction result to obtain the current labeling rules. And the wrong labeling in the third sample data is corrected according to the prediction result, and the incremental sample data is generated according to the correction content. Then, the step of labeling the first sample data using the current labeling rule is repeated until the training end condition is met.

[0046] Figure 3 FIG. 1 is a schematic diagram of the structure of a prediction model training device according to an embodiment of the present disclosure. Figure 3 As shown, the prediction model training device includes a first processing module 31, a second processing module 32, a third processing module 33, a fourth processing module 34 and a fifth processing module 35.

[0047] The first processing module 31 is configured to label the first sample data using a preset labeling rule to obtain second sample data.

[0048] For example, the first sample data is container sample data or physical machine sample data. The first sample data does not include label information.

[0049] The second processing module 32 is configured to train the machine learning model using the second sample data.

[0050] The third processing module 33 is configured to input the first sample data into the trained machine learning model to obtain a prediction result.

[0051] The fourth processing module 34 is configured to update the preset labeling rule according to the prediction result to obtain the current labeling rule.

[0052] In some embodiments, the fourth processing module 34 corrects the erroneous labels in the second sample data according to the prediction result, and generates incremental sample data according to the corrected content.

[0053] The fifth processing module 35 is configured to label the first sample data using the current labeling rule to obtain the third sample data, and to train the trained machine learning model using the third sample data to obtain the prediction model.

[0054] In some embodiments, the fifth processing module 35 trains the trained machine learning model using the third sample data and the incremental sample data to obtain a prediction model.

[0055] In some embodiments, after obtaining the prediction model, the fifth processing module 35 inputs the first sample data into the prediction model to obtain a prediction result. The current labeling rule is updated according to the prediction result, the wrong labeling in the third sample data is corrected according to the prediction result, and the incremental sample data is generated according to the correction content. The step of labeling the first sample data using the current labeling rule is repeatedly performed until the training end condition is met.

[0056] Figure 4 FIG. 1 is a schematic diagram of the structure of a prediction model training device according to another embodiment of the present disclosure. Figure 4 As shown, the prediction model training device includes a memory 41 and a processor 42.

[0057] The memory 41 is used to store instructions. The processor 42 is coupled to the memory 41. The processor 42 is configured to execute the instructions stored in the memory to implement the following. Figure 1 or Figure 2 The method of any one of the embodiments.

[0058] like Figure 4 As shown, the prediction model training device also includes a communication interface 43 for information exchange with other devices. At the same time, the prediction model training device also includes a bus 44, through which the processor 42, the communication interface 43, and the memory 41 communicate with each other.

[0059] The memory 41 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory. The memory 41 may also be a memory array. The memory 41 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0060] In addition, the processor 42 may be a central processing unit (CPU), or may be an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0061] The present disclosure also relates to a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, which are executed by a processor to implement the following Figure 1 or Figure 2The method of any one of the embodiments.

[0062] Figure 5 The following is a flow chart of a resource management method according to an embodiment of the present disclosure. In some embodiments, the following resource management method is executed by a resource management device.

[0063] In step 501, the container indicator data is input into the first prediction model to obtain a first prediction result. The first prediction model uses Figure 1 or Figure 2 The prediction model training method is used to process the container sample data.

[0064] For example, container indicator data is the indicator data of the container in the past 14 days, including the container's CPU utilization, memory utilization, number of TCP connections, network inflow, and network output.

[0065] In step 502, the container static data is aggregated with the first prediction result to obtain a first processing result.

[0066] For example, container static data is container-related data obtained at a daily granularity.

[0067] In step 503, the physical machine indicator data is input into the second prediction model to obtain a second prediction result. Figure 1 or Figure 2 The prediction model training method is used to process the physical machine sample data.

[0068] For example, the physical machine indicator data is the indicator data of the container in the past 14 days, including the CPU utilization, memory utilization, number of TCP connections, network inflow and network output of the physical machine.

[0069] In step 504, the physical machine static data is aggregated with the second prediction result to obtain a second processing result.

[0070] For example, static data of a physical machine is data related to the physical machine obtained at a daily granularity.

[0071] In step 505, the first processing result and the second processing result are combined to obtain a resource summary result.

[0072] In some embodiments, based on the resource summary results, multi-dimensional data statistics and multi-dimensional fine query functions can be provided through the display page. Users can use the display page to accurately query the target physical machine or container, and can also use the statistics page to count the offline status of the container in their department. All data can be exported in the form of an interface. At the same time, departments or individuals with low utilization or a high proportion of idle containers can be notified regularly through emails.

[0073] Figure 6 FIG. 1 is a schematic diagram of the structure of a resource management device according to an embodiment of the present disclosure. Figure 6 As shown, the resource management device includes a first management module 61 , a second management module 62 , a third management module 63 , a fourth management module 64 and a fifth management module 65 .

[0074] The first management module 61 is configured to input the container indicator data into the first prediction model to obtain a first prediction result. Figure 1 or Figure 2 The prediction model training method is used to process the container sample data.

[0075] The second management module 62 is configured to aggregate the container static data with the first prediction result to obtain a first processing result.

[0076] The third management module 63 is configured to input the physical machine indicator data into the second prediction model to obtain a second prediction result. Figure 1 or Figure 2 The prediction model training method is used to process the physical machine sample data.

[0077] The fourth management module 64 is configured to aggregate the physical machine static data with the second prediction result to obtain a second processing result.

[0078] The fifth management module 65 is configured to merge the first processing result and the second processing result to obtain a resource summary result.

[0079] Figure 7 FIG. 1 is a schematic diagram of the structure of a resource management device according to another embodiment of the present disclosure. Figure 7 As shown, the resource management device includes a memory 71 , a processor 72 , a communication interface 73 and a bus 74 . Figure 7 and Figure 4 The difference is that in Figure 7 In the illustrated embodiment, the processor 72 is configured to execute instructions stored in the memory to implement the following Figure 5 The method of any one of the embodiments.

[0080] The present disclosure also relates to a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, which are executed by a processor to implement the following Figure 5 The method of any one of the embodiments.

[0081] In some embodiments, the functional unit module described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components or any appropriate combination thereof for performing the functions described in the present disclosure.

[0082] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0083] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A resource management method, executed by a resource management device, include: Inputting the container indicator data into a first prediction model to obtain a first prediction result, wherein the first prediction model is obtained by processing the container sample data using a prediction model training method; Aggregating the container static data with the first prediction result to obtain a first processing result; Inputting the physical machine indicator data into a second prediction model to obtain a second prediction result, wherein the second prediction model is obtained by processing the physical machine sample data using the prediction model training method; Aggregating the physical machine static data with the second prediction result to obtain a second processing result; Combining the first processing result and the second processing result to obtain a resource summary result; Wherein, the prediction model training method includes: Annotating the first sample data using a preset annotation rule to obtain second sample data; Training the machine learning model using the second sample data; Inputting the first sample data into the trained machine learning model to obtain a prediction result; Update the preset labeling rules according to the prediction results to obtain the current labeling rules; Using the current labeling rule to label the first sample data to obtain third sample data; The trained machine learning model is trained using the third sample data to obtain a prediction model.

2. The resource management method according to claim 1, in, The prediction model training method further includes: Correcting the erroneous annotations in the second sample data according to the prediction result, and generating incremental sample data according to the correction content; Using the third sample data to train the trained machine learning model to obtain a prediction model includes: The trained machine learning model is trained using the third sample data and the incremental sample data to obtain a prediction model.

3. The resource management method according to claim 2, in, The prediction model training method further includes: After obtaining the prediction model, inputting the first sample data into the prediction model to obtain a prediction result; The current labeling rule is updated according to the prediction result, the erroneous labeling in the third sample data is corrected according to the prediction result, and the incremental sample data is generated according to the correction content; Repeat the step of labeling the first sample data using the current labeling rule until the training end condition is met.

4. The resource management method according to any one of claims 1 to 3, in, The first sample data includes container sample data or physical machine sample data.

5. A resource management device, include: A first management module is configured to input the container indicator data into a first prediction model to obtain a first prediction result, wherein the first prediction model is obtained by processing the container sample data using a prediction model training method; a second management module configured to aggregate the container static data with the first prediction result to obtain a first processing result; A third management module is configured to input the physical machine indicator data into a second prediction model to obtain a second prediction result, wherein the second prediction model is obtained by processing the physical machine sample data using the prediction model training method; a fourth management module, configured to aggregate the physical machine static data with the second prediction result to obtain a second processing result; a fifth management module, configured to merge the first processing result and the second processing result to obtain a resource summary result; Wherein, the prediction model training method includes: Annotating the first sample data using a preset annotation rule to obtain second sample data; Training the machine learning model using the second sample data; Inputting the first sample data into the trained machine learning model to obtain a prediction result; Update the preset labeling rules according to the prediction results to obtain the current labeling rules; Using the current labeling rule to label the first sample data to obtain third sample data; The trained machine learning model is trained using the third sample data to obtain a prediction model.

6. A resource management device, include: a memory configured to store instructions; A processor is coupled to the memory, and the processor is configured to execute the resource management method according to any one of claims 1 to 4 based on instructions stored in the memory.

7. A computer-readable storage medium, in, The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the resource management method according to any one of claims 1 to 4 is implemented.

Images