Load prediction methods, apparatus, electronic devices and storage media

Abstract

This application provides a load prediction method, apparatus, electronic device, and storage medium. The method includes: acquiring load performance data of a target data source; inputting the load performance data into a pre-trained neural network model module and acquiring multiple resource load prediction scores of the target data source output by the neural network model module; acquiring pre-configured load selection rules; and, based on the load selection rules, selecting the target resource load prediction score corresponding to the target data source from the multiple resource load prediction scores. This application employs real-time calculation, significantly shortening the time for production system expansion, avoiding system failures caused by resource performance bottlenecks, rationally allocating cloud resource pool resources, improving resource utilization while ensuring performance, and saving costs.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a load prediction method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the implementation of the "cloud transformation and digital transformation" strategy, telecommunications networks, platforms, and systems are undergoing a comprehensive transformation towards cloudification. The number of network nodes, PaaS components, and application instances carrying services is growing exponentially. Existing traditional professional monitoring systems can no longer meet the operational monitoring needs in the cloud environment. Currently, resource usage is mainly assessed manually, leading to inaccurate resource allocation and low resource utilization. Secondly, the exponential growth of application instances means that operations and maintenance personnel cannot constantly monitor the resource usage of each production system, posing a risk of system failures due to untimely resource expansion.

[0003] Database failure: Database disk I / O and CPU bottlenecks lead to an increase in the number of active database sessions, resulting in fewer or even no database connections available.

[0004] Application server failure: When the number of users and access volume increases, the server becomes unable to meet the request pressure and crashes.

[0005] Streaming framework failure: When processing messages, insufficient Kafka consumption capacity can cause data backlog. Once the backlog exceeds the maximum storage time or the maximum amount of data that the streaming framework can store, data loss may occur. Summary of the Invention

[0006] The technical problem to be solved by the embodiments of this application is to provide a load prediction method, device, electronic device and storage medium, so as to adopt real-time calculation, significantly shorten the time for production system expansion, avoid system failures caused by resource performance bottlenecks, rationally allocate cloud resource pool resources, improve resource utilization while ensuring performance and saving costs.

[0007] In a first aspect, embodiments of this application provide a load prediction method, the method comprising:

[0008] Obtain load performance data from the target data source;

[0009] The load performance data is input into a pre-trained neural network model module, and multiple resource load prediction scores of the target data source are obtained from the output of the neural network model module.

[0010] Retrieve pre-configured load selection rules;

[0011] Based on the load selection rules, the target resource load prediction score corresponding to the target data source is selected from the multiple resource load prediction scores.

[0012] Optionally, before inputting the load performance data into a pre-trained neural network model module and obtaining multiple resource load prediction scores of the target data source output by the neural network model module, the method further includes:

[0013] Based on the acquired multiple neural network models, an initial neural network model module is generated;

[0014] Obtain model training samples, which are historical load performance data from the data source;

[0015] The initial neural network model module is trained based on the model training samples until each neural network model in the initial neural network model module converges, thus obtaining the neural network model module.

[0016] Optionally, the load performance data includes: configuration data, performance metric data, and alarm data.

[0017] The step of inputting the load performance data into a pre-trained neural network model module and obtaining multiple resource load prediction scores of the target data source output by the neural network model module includes:

[0018] The configuration data, performance index data, and alarm data are preprocessed to obtain preprocessed load data;

[0019] The preprocessed load data is input into each neural network model in the neural network model module, and the resource load prediction score output by each neural network model is obtained.

[0020] Optionally, the step of selecting the target resource load prediction score corresponding to the target data source from the plurality of resource load prediction scores based on the load selection rules includes:

[0021] Obtain the resource expansion impact parameters corresponding to the target data source;

[0022] Based on the load selection rules and the resource expansion impact parameters, the baseline resource load score corresponding to the target data source is determined;

[0023] Based on the baseline resource load score, the target resource load prediction score is selected from the plurality of resource load prediction scores.

[0024] Optionally, determining the baseline resource load score corresponding to the target data source based on the load selection rules and the resource expansion impact parameters includes:

[0025] Obtain the weighting coefficients corresponding to the resource expansion impact parameters;

[0026] Based on the weighting coefficient and the resource expansion impact parameter, the resource expansion impact coefficient corresponding to the resource expansion impact parameter is calculated;

[0027] The baseline resource load score is determined based on the resource expansion impact coefficient.

[0028] Optionally, the target data source includes at least one of a database, an application server, and a streaming processing framework.

[0029] Secondly, embodiments of this application provide a load prediction device, the device comprising:

[0030] The load performance data acquisition module is used to acquire load performance data from the target data source.

[0031] The load prediction score acquisition module is used to input the load performance data into a pre-trained neural network model module and acquire multiple resource load prediction scores of the target data source output by the neural network model module.

[0032] The load selection rule acquisition module is used to acquire pre-configured load selection rules;

[0033] The target load score acquisition module is used to filter out the target resource load prediction score corresponding to the target data source from the multiple resource load prediction scores based on the load selection rules.

[0034] Optionally, the device further includes:

[0035] The initial model module generation module is used to generate an initial neural network model module based on the acquired multiple neural network models;

[0036] The model training sample acquisition module is used to acquire model training samples, which are historical load performance data from the data source.

[0037] The network model module acquisition module is used to train the initial neural network model module based on the model training samples until each neural network model in the initial neural network model module converges, thereby obtaining the neural network model module.

[0038] Optionally, the load performance data includes: configuration data, performance metric data, and alarm data.

[0039] The load prediction score acquisition module includes:

[0040] A preprocessing data acquisition unit is used to preprocess the configuration data, the performance index data, and the alarm data to obtain preprocessed load data.

[0041] The load prediction score acquisition unit is used to input the preprocessed load data into each neural network model in the neural network model module, and to acquire the resource load prediction score output by each neural network model.

[0042] Optionally, the target load score acquisition module includes:

[0043] The expansion impact parameter acquisition unit is used to acquire the resource expansion impact parameters corresponding to the target data source;

[0044] The baseline load score determination unit is used to determine the baseline resource load score corresponding to the target data source based on the load selection rules and the resource expansion impact parameters.

[0045] The target load score filtering unit is used to filter the target resource load prediction score from the plurality of resource load prediction scores based on the benchmark resource load score.

[0046] Optionally, the reference load score determination unit includes:

[0047] The weighting coefficient acquisition subunit is used to acquire the weighting coefficients corresponding to the resource expansion impact parameters;

[0048] The influence coefficient calculation subunit is used to calculate the resource expansion influence coefficient corresponding to the resource expansion influence parameter based on the weighting coefficient and the resource expansion influence parameter;

[0049] The benchmark score determination subunit is used to determine the benchmark resource load score based on the resource expansion impact coefficient.

[0050] Optionally, the target data source includes at least one of a database, an application server, and a streaming processing framework.

[0051] Thirdly, embodiments of this application provide an electronic device, including:

[0052] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the load prediction method described above.

[0053] Fourthly, embodiments of this application provide a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the load prediction methods described above.

[0054] Compared with the prior art, the embodiments of this application have the following advantages:

[0055] In this embodiment, load performance data of the target data source is obtained. This load performance data is input into a pre-trained neural network model module, and multiple resource load prediction scores for the target data source are obtained from the output of the neural network model module. Pre-configured load selection rules are obtained, and based on these rules, the target resource load prediction score corresponding to the target data source is selected from the multiple resource load prediction scores. This embodiment, by employing real-time computing, significantly shortens the time required for production system expansion, avoids system failures caused by resource performance bottlenecks, rationally allocates cloud resource pool resources, and improves resource utilization while ensuring performance, thus saving costs.

[0056] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0057] Figure 1 A flowchart illustrating the steps of a load prediction method provided in this application embodiment;

[0058] Figure 2 A flowchart illustrating the steps of a method for obtaining a neural network model module provided in this application embodiment;

[0059] Figure 3 A flowchart illustrating the steps of a method for obtaining a resource load prediction score, as provided in an embodiment of this application;

[0060] Figure 4 A flowchart illustrating the steps of a target resource load prediction score screening method provided in this application embodiment;

[0061] Figure 5 A flowchart illustrating the steps of a method for determining a baseline resource load score, as provided in an embodiment of this application;

[0062] Figure 6 A schematic diagram of a load prediction framework provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram illustrating a data source standardization process provided in an embodiment of this application.

[0064] Figure 8 A schematic diagram illustrating a model algorithm implementation process provided in an embodiment of this application;

[0065] Figure 9 This is a schematic diagram of the structure of a load prediction device provided in an embodiment of this application;

[0066] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0069] Reference Figure 1 The flowchart illustrates the steps of a load prediction method provided in an embodiment of this application, as shown below. Figure 1 As shown, the load forecasting method may include steps 101, 102, 103 and 104.

[0070] Step 101: Obtain the load performance data of the target data source.

[0071] The embodiments of this application can be applied to scenarios where the resource load of a target data source is predicted using a neural network model module and pre-configured load selection rules.

[0072] In this embodiment, the neural network model module can be composed of multiple neural network models. This neural network model module can be obtained through online training. The training process of the neural network model module can be combined with... Figure 2 The following is a detailed description.

[0073] Reference Figure 2 The flowchart illustrates the steps of a method for obtaining a neural network model module according to an embodiment of this application. Figure 2 As shown, the method for obtaining the neural network model module may include steps 201, 202 and 203.

[0074] Step 201: Generate an initial neural network model module based on the acquired multiple neural network models.

[0075] In this embodiment, multiple neural network models can be obtained in advance, and an initial neural network model module can be formed based on the multiple neural network models.

[0076] In a practical implementation, the multiple neural network models can be at least two of the following: linear regression, logistic regression, ARIMA, random forest, gradient boosting, and other neural network models. Understandably, the number and type of neural network models in the generated initial neural network model module can be determined according to business requirements, and this embodiment does not impose any restrictions on this.

[0077] After generating an initial neural network model module based on the acquired multiple neural network models, step 202 is executed.

[0078] Step 202: Obtain model training samples, which are historical load performance data from the data source.

[0079] After generating an initial neural network model module based on the acquired multiple neural network models, model training samples can be obtained. These training samples can be historical load performance data from the data source, which can be used to train the initial neural network model.

[0080] After obtaining the model training samples, proceed to step 203.

[0081] Step 203: Train the initial neural network model module based on the model training samples until each neural network model in the initial neural network model module converges, thereby obtaining the neural network model module.

[0082] After obtaining the model training samples, the initial neural network model module can be trained based on these samples until each neural network model in the initial neural network model module converges, thus obtaining the neural network model module. In specific implementations, model training can be performed using mainstream AI algorithms (linear regression, logistic regression, ARIMA, random forest, gradient boosting, neural networks, etc.). By continuously adjusting its parameters and refining the algorithm, and selecting the optimal model algorithm combination from multiple algorithms, the neural network model module is obtained.

[0083] In this example, the target data source can be at least one of the following: a database, an application server, a streaming framework, etc.

[0084] Load performance data can include: configuration data, performance metrics data, and alarm data, etc.

[0085] When predicting the resource load of a target data source, the load performance data of the target data source can be obtained.

[0086] After obtaining the load performance data of the target data source, proceed to step 102.

[0087] Step 102: Input the load performance data into a pre-trained neural network model module, and obtain multiple resource load prediction scores of the target data source output by the neural network model module.

[0088] The resource load prediction score refers to the resource load score of the target data source output by multiple neural network models in the neural network model module.

[0089] After obtaining the load performance data of the target data source, the load performance data can be input into a pre-trained neural network model module. Each neural network model in the module processes the load performance data, and each model outputs a resource load prediction score corresponding to the target data source, thus obtaining multiple resource load prediction scores for the target data source. This implementation process can be combined with... Figure 3 The following is a detailed description.

[0090] Reference Figure 3 The flowchart illustrates the steps of a resource load prediction score acquisition method provided in an embodiment of this application, as follows: Figure 3 As shown, the method for obtaining the resource load prediction score may include steps 301 and 302.

[0091] Step 301: Preprocess the configuration data, the performance index data, and the alarm data to obtain preprocessed load data.

[0092] In this embodiment, the load performance data may include: configuration data, performance index data, and alarm data.

[0093] After obtaining the load performance data from the target data source, the load performance data can be preprocessed to obtain preprocessed load data. Specifically, preprocessing operations can include: data error correction, deleting duplicates, structure transformation, and standardization. The preprocessing process for load performance data can be combined with... Figure 7 The following is a detailed description.

[0094] Reference Figure 7 The diagram illustrates a data source standardization process provided in an embodiment of this application.

[0095] like Figure 7As shown, after obtaining the configuration data, performance metrics, and alarm data of the target data source, duplicate items can be removed from this load performance data, such as deleting duplicate data and duplicate attributes. Data correction operations can also be performed on this load performance data, such as correcting non-uniform intervals to uniform intervals, correcting missing values ​​to regression model prediction interpolation, and correcting outliers by retaining them. Furthermore, this load performance data will be standardized in terms of data type, abbreviation rules, units, value range, calculation methods, and decimal places. Finally, structural transformations will be performed on this load performance data, such as data type conversion, table / data splitting, data semantic transformation, attribute construction, data granularity transformation, and data compression.

[0096] After preprocessing the configuration data, performance index data, and alarm data to obtain preprocessed load data, step 302 is executed.

[0097] Step 302: Input the preprocessed load data into each neural network model in the neural network model module, and obtain the resource load prediction score output by each neural network model.

[0098] After preprocessing the configuration data, performance index data, and alarm data to obtain preprocessed load data, the preprocessed load data can be input into each neural network model in the neural network model module, and the resource load prediction score of the target data source output by each neural network model can be obtained.

[0099] Step 103: Obtain the pre-configured load selection rules.

[0100] Load selection rules are rules used to select the optimal forecast score from multiple resource load forecast scores. In this example, the load selection rule can be a function of a pre-defined baseline resource load score.

[0101] When predicting the resource load score of a target data source, pre-configured load selection rules can be obtained.

[0102] After obtaining the pre-configured load selection rules, proceed to step 104.

[0103] Step 104: Based on the load selection rules, select the target resource load prediction score corresponding to the target data source from the multiple resource load prediction scores.

[0104] After obtaining the pre-configured load selection rules and multiple resource load prediction scores, the target resource load prediction score corresponding to the target data source can be selected from the multiple resource load prediction scores based on the load selection rules. This target resource load prediction score can then be used to determine whether the target data source needs resource expansion. The selection process for the target resource load prediction score can be combined with... Figure 4 The following is a detailed description.

[0105] Reference Figure 4 The flowchart illustrates the steps of a target resource load prediction score screening method provided in an embodiment of this application, as follows: Figure 4 As shown, the target resource load prediction score screening method may include steps 401, 402 and 403.

[0106] Step 401: Obtain the resource expansion impact parameters corresponding to the target data source.

[0107] In this embodiment, the resource expansion impact parameters refer to parameters that can affect the expansion of the target data source. In this example, the resource expansion impact parameters may include parameters such as host CPU (Central Processing Unit) utilization, memory utilization, and disk utilization.

[0108] When obtaining the baseline resource load score of the target data source, the resource expansion impact parameters corresponding to the target data source can be obtained.

[0109] After obtaining the resource expansion impact parameters corresponding to the target data source, proceed to step 402.

[0110] Step 402: Based on the load selection rules and the resource expansion impact parameters, determine the baseline resource load score corresponding to the target data source.

[0111] After obtaining the resource expansion impact parameters corresponding to the target data source, the baseline resource load score corresponding to the target data source can be determined based on the load selection rules and the resource expansion impact parameters. This implementation process can be combined with... Figure 5 The following is a detailed description.

[0112] Reference Figure 5 The flowchart illustrates the steps of a method for determining a baseline resource load score according to an embodiment of this application. Figure 5 As shown, the method for determining the baseline resource load score may include steps 501, 502, and 503.

[0113] Step 501: Obtain the weighting coefficients corresponding to the resource expansion impact parameters.

[0114] In this embodiment, the weighting coefficient refers to the weighting ratio used to indicate the impact parameters of each resource expansion. In this example, the weighting coefficient can be calculated using objective weighting methods such as entropy method and combined with the scoring method of business personnel.

[0115] After obtaining the weighted coefficients corresponding to the parameters affecting resource expansion, proceed to step 502.

[0116] Step 502: Based on the weighting coefficient and the resource expansion impact parameter, calculate the resource expansion impact coefficient corresponding to the resource expansion impact parameter.

[0117] After obtaining the weighted coefficients corresponding to the resource expansion impact parameters, the resource expansion impact coefficients corresponding to the resource expansion impact parameters can be calculated based on the weighted coefficients and the resource expansion impact parameters.

[0118] After calculating the resource expansion impact coefficient corresponding to the resource expansion impact parameter based on the weighting coefficient and the resource expansion impact parameter, step 503 is executed.

[0119] Step 503: Determine the baseline resource load score based on the resource expansion impact coefficient.

[0120] After calculating the resource expansion impact coefficient corresponding to the resource expansion impact parameter based on the weighting coefficient and the resource expansion impact parameter, the benchmark resource load score can be determined based on the resource expansion coefficient.

[0121] In this example, the load selection rule can be as shown in the following formula (1):

[0122] S = aK1 + bK2 + cK3 + ... + xK n (1)

[0123] In the above formula (1), S is the baseline resource load score, a, b, c, etc. are weighting coefficients, and K is the resource expansion impact parameter.

[0124] Specifically, the product of the weighting coefficient and the corresponding resource expansion impact parameter is the resource expansion impact coefficient, and the sum of multiple resource expansion impact coefficients yields the baseline resource load score.

[0125] After determining the baseline resource load score corresponding to the target data source based on the load selection rules and resource expansion impact parameters, step 403 is executed.

[0126] Step 403: Based on the baseline resource load score, select the target resource load prediction score from the plurality of resource load prediction scores.

[0127] After determining the baseline resource load score corresponding to the target data source based on the load selection rules and resource expansion impact parameters, the target resource load prediction score can be selected from multiple resource load prediction scores based on the baseline resource load score. Specifically, the resource load prediction score with the smallest difference from the baseline resource load score can be selected as the target resource load prediction score.

[0128] This application embodiment significantly shortens the time for expanding the production system by employing real-time computing, avoids system failures caused by resource performance bottlenecks, rationally allocates cloud resource pool resources, and improves resource utilization while ensuring performance and saving costs.

[0129] The process of the above-described solution provided in this embodiment can be combined with... Figure 6 and Figure 8 The following is a detailed description.

[0130] like Figure 6 As shown, the implementation process can include three steps:

[0131] Step 1: Data acquisition. This involves combining the historical data warehouse of the target data source to filter out the configuration data, performance indicators, and alarm data of the target data source. Then, the filtered data is preprocessed and feature extracted to obtain the load performance characteristics required by the neural network model module.

[0132] Step 2, Model Training: Appropriate neural network models can be selected from the machine learning model library to generate model modules. The model modules are then trained online using the data obtained in Step 1 above to obtain a converged model module for each neural network model. Finally, the resource load score of the target data source is predicted.

[0133] Model training can include five steps, such as... Figure 8 As shown, the model training process can be as follows: 1. Model selection, 2. Model training, 3. Model evaluation based on model output, 4. If the model does not converge, adjust the model parameters, 5. After each model converges, the model module, i.e., model N (N is a positive integer greater than or equal to 2), can be obtained.

[0134] Real-time data can be collected when making real-time predictions. For example... Figure 8 As shown, the load performance data and other data from the Kafka queue of the target data source are collected and cached in Redis. Then, the data are combined with model N for real-time judgment to obtain the corresponding resource load prediction results.

[0135] Step 3, Result Selection: Based on the input selection rules and the output results of multiple prediction models, the best model is selected, and its output is taken as the optimal result. Based on the optimal result, it is determined whether to output an alarm message to indicate whether the target data source needs resource expansion. For example... Figure 8 As shown, the selection rules can be determined based on business experience. Based on the selection rules and the results output by the multiple prediction models, the best model among the multiple prediction models is determined, and then the prediction result output by the best model is taken as the optimal result.

[0136] Load forecasting was performed on a portion of historical data using a database and message queue, as illustrated in the following example:

[0137] For database load prediction:

[0138] Performance prediction for the distributed database (teledb) is achieved by analyzing performance metrics such as the number of slow SQL queries, disk I / O awaits, and memory usage. AI is used to identify trends in these metrics, and algorithms are then used to generate scenario-based load prediction indicators, enabling predictive alerts for database load scenarios. Since its trial launch at the end of September, the WeChat Work system experienced a significant increase in slow SQL queries, rapid growth in disk I / O awaits, high memory consumption, and an unusually significant increase in sessions in October. On October 20th, it was predicted that the system's database performance would reach a bottleneck in three days. Subsequent measures, including expanding the database cluster and optimizing slow SQL queries, have brought the system's database to a healthy state.

[0139] For message queue load prediction:

[0140] For the message queue (Kafka component), combining its host performance metrics (CPU, memory, disk, network I / O) with Kafka's performance metrics (data backlog, etc.), AI is used to identify trends for each, and then algorithms are used to generate scenario-based load prediction metrics, thereby predicting and alerting message queue load scenarios. Since its trial operation at the end of September, it was found that the Kafka memory usage of the intelligent operation and maintenance system showed a significant upward trend on October 18th, while the data backlog was also slowly increasing, and the disk space usage also showed a linear increasing trend. Analysis revealed that this was due to the increased data volume accessed by the intelligent operation and maintenance system in October, causing a bottleneck in the Kafka cluster's data processing capacity. Therefore, the Kafka cluster was expanded at the end of October 2022.

[0141] As can be seen from the above examples, the solution provided in this application embodiment can reasonably allocate cloud resource pool resources, improve asset utilization and save costs while ensuring performance.

[0142] The load prediction method provided in this application obtains load performance data of a target data source. The load performance data is input into a pre-trained neural network model module, and multiple resource load prediction scores of the target data source are obtained from the output of the neural network model module. Pre-configured load selection rules are obtained, and based on these rules, the target resource load prediction score corresponding to the target data source is selected from the multiple resource load prediction scores. This application embodiment significantly shortens the time required for production system expansion by employing real-time computing, avoids system failures caused by resource performance bottlenecks, rationally allocates cloud resource pool resources, and improves resource utilization while ensuring performance, thus saving costs.

[0143] Reference Figure 9 The diagram shows a structural schematic of a load prediction device provided in an embodiment of this application, such as... Figure 9 As shown, the load prediction device 900 may include the following modules:

[0144] The load performance data acquisition module 910 is used to acquire load performance data from the target data source.

[0145] The load prediction score acquisition module 920 is used to input the load performance data into a pre-trained neural network model module and acquire multiple resource load prediction scores of the target data source output by the neural network model module.

[0146] The load selection rule acquisition module 930 is used to acquire pre-configured load selection rules;

[0147] The target load score acquisition module 940 is used to filter out the target resource load prediction score corresponding to the target data source from the plurality of resource load prediction scores based on the load selection rules.

[0148] Optionally, the device further includes:

[0149] The initial model module generation module is used to generate an initial neural network model module based on the acquired multiple neural network models;

[0150] The model training sample acquisition module is used to acquire model training samples, which are historical load performance data from the data source.

[0151] The network model module acquisition module is used to train the initial neural network model module based on the model training samples until each neural network model in the initial neural network model module converges, thereby obtaining the neural network model module.

[0152] Optionally, the load performance data includes: configuration data, performance metric data, and alarm data.

[0153] The load prediction score acquisition module includes:

[0154] A preprocessing data acquisition unit is used to preprocess the configuration data, the performance index data, and the alarm data to obtain preprocessed load data.

[0155] The load prediction score acquisition unit is used to input the preprocessed load data into each neural network model in the neural network model module, and to acquire the resource load prediction score output by each neural network model.

[0156] Optionally, the target load score acquisition module includes:

[0157] The expansion impact parameter acquisition unit is used to acquire the resource expansion impact parameters corresponding to the target data source;

[0158] The baseline load score determination unit is used to determine the baseline resource load score corresponding to the target data source based on the load selection rules and the resource expansion impact parameters.

[0159] The target load score filtering unit is used to filter the target resource load prediction score from the plurality of resource load prediction scores based on the benchmark resource load score.

[0160] Optionally, the reference load score determination unit includes:

[0161] The weighting coefficient acquisition subunit is used to acquire the weighting coefficients corresponding to the resource expansion impact parameters;

[0162] The influence coefficient calculation subunit is used to calculate the resource expansion influence coefficient corresponding to the resource expansion influence parameter based on the weighting coefficient and the resource expansion influence parameter;

[0163] The benchmark score determination subunit is used to determine the benchmark resource load score based on the resource expansion impact coefficient.

[0164] Optionally, the target data source includes at least one of a database, an application server, and a streaming processing framework.

[0165] The load prediction device provided in this application acquires load performance data from a target data source. The load performance data is input into a pre-trained neural network model module, and multiple resource load prediction scores for the target data source are obtained from the output of the neural network model module. Pre-configured load selection rules are acquired, and based on these rules, the target resource load prediction score corresponding to the target data source is selected from the multiple resource load prediction scores. This application embodiment significantly shortens the time required for production system expansion by employing real-time computation, avoids system failures caused by resource performance bottlenecks, rationally allocates cloud resource pool resources, and improves resource utilization while ensuring performance, thus saving costs.

[0166] This application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described load prediction method.

[0167] Figure 10 A schematic diagram of the structure of an electronic device 1000 according to an embodiment of the present invention is shown. Figure 10 As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1002 or loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0168] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, microphone, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0169] The various processes and handling described above can be executed by processing unit 1001. For example, the methods of any of the above embodiments can be implemented as computer software programs tangibly contained in a computer-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by CPU 1001, one or more actions of the methods described above can be performed.

[0170] Additionally, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described load prediction method.

[0171] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0172] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminals (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal, causing a series of operational steps to be executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0176] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0177] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes said element.

[0178] The above provides a detailed description of a load prediction method, a load prediction device, an electronic device, and a computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A load prediction method characterized by, The method comprises: obtaining load performance data of a target data source; inputting the load performance data into a pre-trained neural network model module, and obtaining a plurality of resource load prediction scores of the target data source output by the neural network model module; obtaining a pre-configured load selection rule; based on the load selection rule, screening out a target resource load prediction score corresponding to the target data source from the plurality of resource load prediction scores; the method comprises: obtaining a resource expansion impact parameter corresponding to the target data source; based on the load selection rule and the resource expansion impact parameter, determining a benchmark resource load score corresponding to the target data source; based on the benchmark resource load score, screening out the target resource load prediction score from the plurality of resource load prediction scores; the method comprises: obtaining a weighting coefficient corresponding to the resource expansion impact parameter; based on the weighting coefficient and the resource expansion impact parameter, calculating a resource expansion impact coefficient corresponding to the resource expansion impact parameter; based on the resource expansion impact coefficient, determining the benchmark resource load score.

2. The method of claim 1, wherein, Before the load performance data is input into the pre-trained neural network model module, and a plurality of resource load prediction scores of the target data source output by the neural network model module are obtained, the method further comprises: generating an initial neural network model module according to the obtained plurality of neural network models; obtaining a model training sample, the model training sample being historical load performance data of a data source; based on the model training sample, training the initial neural network model module until each neural network model in the initial neural network model module converges, to obtain the neural network model module.

3. The method of claim 1, wherein, The load performance data comprises configuration data, performance indicator data and alarm data, the method comprises: preprocessing the configuration data, the performance indicator data and the alarm data to obtain preprocessed load data; inputting the preprocessed load data into each neural network model in the neural network model module, and obtaining the resource load prediction score output by each neural network model.

4. The method according to any one of claims 1 to 3, characterized in that, The target data source comprises at least one of a database, an application server and a stream processing framework.

5. A load prediction device characterized by comprising: The device comprises: a load performance data acquisition module for obtaining load performance data of a target data source; a load prediction score acquisition module for inputting the load performance data into a pre-trained neural network model module, and obtaining a plurality of resource load prediction scores of the target data source output by the neural network model module; The load selection rule acquisition module is configured to acquire a preconfigured load selection rule. The target load score acquisition module is configured to filter out a target resource load prediction score corresponding to the target data source from the plurality of resource load prediction scores based on the load selection rule. The target load score acquisition module includes: The expansion impact parameter acquisition unit is configured to acquire a resource expansion impact parameter corresponding to the target data source. The reference load score determination unit is configured to determine a reference resource load score corresponding to the target data source based on the load selection rule and the resource expansion impact parameter. The target load score filtering unit is configured to filter out the target resource load prediction score from the plurality of resource load prediction scores based on the reference resource load score. The reference load score determination unit includes: The weighting coefficient acquisition subunit is configured to acquire a weighting coefficient corresponding to the resource expansion impact parameter. The impact coefficient calculation subunit is configured to calculate a resource expansion impact coefficient corresponding to the resource expansion impact parameter based on the weighting coefficient and the resource expansion impact parameter. The reference score determination subunit is configured to determine the reference resource load score based on the resource expansion impact coefficient.

6. The apparatus of claim 5, wherein, The device further includes: The initial model module generation module is configured to generate an initial neural network model module based on the acquired plurality of neural network models. The model training sample acquisition module is configured to acquire a model training sample, which is historical load performance data of a data source. The network model module acquisition module is configured to train the initial neural network model module based on the model training sample until each neural network model in the initial neural network model module converges, and obtain the neural network model module.

7. An electronic device, comprising: The device includes: A processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the processor implements the load prediction method of any one of claims 1 to 4 when executing the program.

8. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can perform the load prediction method of any one of claims 1 to 4.

Images