A method for processing communication big data
By preprocessing historical network traffic and resource demand data and training and verification of prediction models, the problem of difficult prediction of future network traffic and resource demand in the existing technology is solved, and the optimization of the prediction model and the stability of the system are achieved.
Patent Information
- Application Number
- CN202410322493.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-20
- Publication Date
- 2025-05-06
- Estimated Expiration
- 2044-03-20
AI Technical Summary
The existing technology is inconvenient to predict future network traffic and resource requirements by monitoring historical network traffic and equipment resource utilization.
By obtaining the historical network traffic and resource requirements data of the pending device for preprocessing, the data is processed based on the trained prediction model, predicting future network traffic and resource requirements, and determining whether the prediction model needs to be retrained and verified based on the comparison results of the actual value and the predicted value.
The comprehensive management and optimization of the prediction model is realized, and a practical, flexible and adaptive solution is provided, which can more stably determine whether the prediction model needs to be adjusted, improving the stability and prediction accuracy of the system.
Smart Images

Figure CN118158115B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of communication data processing, and in particular to a communication big data processing method. Background Art
[0002] Gartner, a research organization, defines "big data" as follows. "Big data" requires new processing models to have stronger decision-making power, insight discovery power and process optimization capabilities to adapt to massive, high-growth and diversified information assets; McKinsey Global Institute defines it as: a data set that is so large that it far exceeds the capabilities of traditional database software tools in terms of acquisition, storage, management and analysis, and has four major characteristics: massive data scale, fast data flow, diverse data types and low value density; the strategic significance of big data technology lies not in mastering huge amounts of data information, but in professional processing of these meaningful data. In other words, if big data is compared to an industry, then the key to profitability of this industry lies in improving the "processing ability" of data and achieving "value-added" of data through "processing"; from a technical point of view, the relationship between big data and cloud computing is as inseparable as the front and back of a coin. Big data cannot be processed by a single computer and must adopt a distributed architecture. Its feature is distributed data mining of massive data. But it must rely on distributed processing, distributed databases, cloud storage and virtualization technologies of cloud computing; with the advent of the cloud era, big data has also attracted more and more attention. The analyst team believes that big data is often used to describe a large amount of unstructured and semi-structured data created by a company, which takes too much time and money to download to a relational database for analysis. Big data analysis is often associated with cloud computing because real-time analysis of large data sets requires a framework like MapReduce to distribute work to dozens, hundreds or even thousands of computers.
[0003] The Chinese patent application with publication number CN107332869A discloses a communication big data processing method for bidirectional rapid reception, processing, and forwarding of data, which is particularly suitable for a three-layer structure of multiple terminals, a communication server, and multiple monitoring centers. In the implementation, the database information is read and recorded in the memory, and the query memory is substituted for the query database. In addition, the HASH hashing technology is used when recording information in the memory, which greatly improves the query speed. Without adding any hardware equipment to the original communication server, the processing performance can be greatly improved by only updating the software implementation method, simplifying the system structure and saving system costs.
[0004] However, existing technologies are not convenient for predicting future network traffic and resource requirements by monitoring historical network traffic and device resource utilization. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention provides a communication big data processing method, which solves the problem that the prior art is inconvenient to predict future network traffic and resource requirements by monitoring historical network traffic and device resource utilization.
[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a communication big data processing method, comprising the following steps: obtaining the processed data about the historical network traffic and resource requirements of the device to be processed and preprocessing the processed data, wherein the processed data includes broadband utilization, CPU utilization and memory utilization; processing the processed data based on a trained prediction model, predicting the predicted value of the device to be processed, and adjusting the future network traffic and resource requirements of the device to be processed based on the predicted value, wherein the predicted value is used to represent the future network traffic and resource requirements of the device to be processed; obtaining the actual value of the network traffic and resource requirements of the device to be processed, and comparing the actual value with the predicted value, and determining whether the prediction model needs to be retrained and verified based on the comparison result.
[0007] Furthermore, preprocessing of the data to be processed includes processing missing values, and the specific process is as follows: obtaining the timestamp corresponding to the missing value, determining the preceding data of several time points before the timestamp corresponding to the missing value and the following data of several time points after the timestamp corresponding to the missing value; processing the preceding data and the following data to obtain the insertion value, matching the insertion value with the timestamp corresponding to the missing value, and filling in the insertion value.
[0008] Furthermore, the formula for calculating the inserted value is as follows: In the formula, X c is the insertion value, i=1,2,3,...,n is the number of previous data, x i is the i-th data in the previous sequence data, j=1,2,3,...,m is the number of previous sequence data, x j is the jth data in the subsequent data.
[0009] Furthermore, the training process of the prediction model is as follows: the pre-processed data to be processed is divided into a training group and a validation group; feature processing is performed on the training group and the validation group respectively; the feature-processed training group data is input into the prediction model; and the prediction model is trained according to a specified time step.
[0010] Furthermore, after the prediction model is trained based on the data in the training group, the trained prediction model is verified using the data in the validation group. The process is as follows: the data in the validation group is input into the trained prediction model to obtain the prediction value to be verified; the prediction value to be verified is compared with the true value in the validation group, and the trained prediction model is evaluated to obtain an evaluation value; the evaluation value is compared with the set evaluation threshold to judge the performance of the prediction model. If the evaluation value is lower than the set evaluation threshold, the prediction model is retrained.
[0011] Furthermore, the prediction model is established based on a long short-term memory network, and the prediction model includes an input layer, an LSTM layer and an output layer.
[0012] Furthermore, the calculation formula of the evaluation value is as follows: In the formula, Λ y is the evaluation value, a=1,2,3,...,A is the number of predicted values to be verified, RJ a is the ath predicted value to be verified, RY a For RJ a The corresponding a-th true value in the validation group, ΔR is the set allowable error value of the prediction model, and σ is the error adjustment factor.
[0013] Furthermore, actual values of network traffic and resource requirements of the device to be processed are obtained, and the actual values are compared with the predicted values. The process of determining whether the prediction model needs to be adjusted based on the comparison results is as follows: the actual values of network traffic and resource requirements of the device to be processed are obtained according to the set period, and the predicted value predicted by the prediction model corresponding to each actual value is obtained; the deviation value between each actual value and the corresponding predicted value is judged, and if the deviation value is greater than the set first deviation threshold, the reset index is increased by one; if the reset index is equal to the set reset threshold, the actual values of network traffic and resource requirements of the device to be processed obtained are used to retrain and verify the prediction model, and the reset index is used to represent the number of times the prediction value of the prediction model is wrong.
[0014] Furthermore, in the process of determining the deviation between each actual value and the corresponding predicted value, if the deviation value is continuously greater than the set deviation threshold for a set number of consecutive times, the total of the deviation values within the consecutive times is calculated; if the total of the deviation values is greater than or equal to the second deviation threshold, the actual values of the network traffic and resource requirements of the device to be processed are used to retrain and verify the prediction model.
[0015] Furthermore, the calculation formula of the total deviation value is as follows: Where Γ is the total deviation value, b=1,2,3,...,B is the number of consecutive times, YU bThe bth predicted value that is continuously greater than the set deviation threshold, SHI b For YU b The corresponding b-th actual value, e is a natural constant.
[0016] The present invention has the following beneficial effects:
[0017] (1) This communication big data processing method realizes the comprehensive management and optimization of the prediction model by comprehensively considering the deviation between the actual value and the predicted value, the dynamic adjustment mechanism, and multiple aspects of the model performance, providing a practical, flexible, and adaptive solution to the problem of network traffic and resource demand prediction.
[0018] (2) The communication big data processing method introduces a mechanism for continuous deviation judgment and deviation value sum calculation, making the system more adaptive and able to more stably judge whether the prediction model needs to be adjusted, which helps prevent the model from being overly sensitive to noise in the short term and improves the stability of the system. Through the calculation formula of the total deviation value, the deviation situation within a consecutive number of times is comprehensively considered, making the adjustment of the prediction model more comprehensive and accurate, helping to reduce error accumulation and reduce the impact of sudden noise.
[0019] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a flow chart of the communication big data processing method of the present invention.
[0021] Figure 2 This is a flow chart of processing missing values in the communication big data processing method of the present invention.
[0022] Figure 3 This is a flow chart of the training process of the prediction model of the communication big data processing method of the present invention.
[0023] Figure 4 The present invention is a flow chart of the communication big data processing method for determining whether the prediction model needs to be adjusted. DETAILED DESCRIPTION
[0024] The embodiment of the present application solves the problem that it is inconvenient to predict future network traffic and resource requirements by monitoring historical network traffic and device resource utilization in the prior art through a communication big data processing method.
[0025] The overall idea of the problem in the embodiment of this application is as follows:
[0026] First, obtain the historical network traffic and resource utilization data of the device to be processed, which includes but is not limited to information such as bandwidth utilization, CPU utilization, and memory utilization. The acquired data needs to be preprocessed to ensure data quality and consistency. Use machine learning or other predictive modeling techniques to establish a model that can learn from historical data and predict future network traffic and resource requirements. Use the trained model to process the data to be processed to obtain the predicted value, that is, the future network traffic and resource requirements of the device to be processed. Based on these predicted values, resource adjustment strategies can be implemented to ensure that future network needs are met, involving measures such as dynamically allocating bandwidth and adjusting CPU and memory allocation.
[0027] Monitor the actual network traffic and resource demand over a period of time. Obtain the actual values and compare them with the predicted values to evaluate the accuracy and performance of the model. Based on the comparison results, decisions can be made to determine whether the prediction model needs to be adjusted. If a large deviation is found between the predicted values and the actual values, the model needs to be retrained to improve accuracy. The feedback mechanism incorporates these experiences into the cycle of model improvement.
[0028] Through this overall concept, we can better understand and adapt to changes in network traffic and resource utilization, so as to make adjustments in advance, optimize system performance and improve network efficiency. This makes network resource management more intelligent and adaptive, and suitable for the ever-changing communication environment.
[0029] See also Figure 1 The embodiment of the present invention provides a technical solution: a communication big data processing method, comprising the following steps: obtaining the data to be processed about the historical network traffic and resource requirements of the device to be processed and preprocessing the data to be processed, wherein the data to be processed includes the broadband utilization, CPU utilization and memory utilization; processing the data to be processed based on the trained prediction model, predicting the predicted value of the device to be processed, and adjusting the future network traffic and resource requirements of the device to be processed based on the predicted value, wherein the predicted value is used to represent the future network traffic and resource requirements of the device to be processed; obtaining the actual value of the network traffic and resource requirements of the device to be processed, and comparing the actual value with the predicted value, and determining whether the prediction model needs to be retrained and verified based on the comparison result.
[0030] Specifically, Figure 2 As shown, preprocessing the data to be processed includes processing missing values, and the specific process is as follows: obtaining the timestamp corresponding to the missing value, determining the preceding data of several time points before the timestamp corresponding to the missing value and the following data of several time points after the timestamp corresponding to the missing value; processing the preceding data and the following data to obtain the insertion value, matching the insertion value with the timestamp corresponding to the missing value, and filling in the insertion value.
[0031] The formula to calculate the inserted value is as follows: In the formula, X c is the insertion value, i=1,2,3,...,n is the number of previous data, x i is the i-th data in the previous sequence data, j=1,2,3,...,m is the number of previous sequence data, x j is the jth data in the subsequent data.
[0032] In this embodiment, the method for processing missing values helps to improve the integrity of data. In the prediction modeling of network traffic and resource requirements, ensuring the integrity of data is very critical because missing values may lead to inaccurate predictions of future network states.
[0033] The interpolation calculation formula based on the preceding and succeeding data takes into account the weights of the preceding and succeeding data, making the interpolated values more reasonable. This interpolation method can better reflect the trends and changes of data when filling missing values, better preserve the information in the time dimension, and help to perform reasonable data interpolation in time.
[0034] The calculation logic is divided into the following steps: Calculate the average of the sum of squares of the preceding data and then the sum of squares of the succeeding data, add the averages obtained in the first two steps, take the square root of the above results, and get the insertion value. The purpose is to consider the average of the sum of squares of the preceding and succeeding data, and combine the change trend of the data in these two time periods to generate an insertion value to fill the missing value. The calculation of the insertion value helps to comprehensively consider the overall change of the data by averaging the sum of squares of the preceding and succeeding data.
[0035] Specifically, Figure 3 As shown in the figure, the training process of the prediction model is as follows: divide the preprocessed data to be processed into a training group and a validation group; perform feature processing on the training group and the validation group respectively; input the feature-processed training group data into the prediction model; and train the prediction model according to the specified time step.
[0036] After the prediction model is trained based on the data in the training group, the trained prediction model is verified using the data in the validation group. The process is as follows: the data in the validation group is input into the trained prediction model to obtain the prediction value to be verified; the prediction value to be verified is compared with the true value in the validation group, and the trained prediction model is evaluated to obtain the evaluation value; the evaluation value is compared with the set evaluation threshold to judge the performance of the prediction model. If the evaluation value is lower than the set evaluation threshold, the prediction model is retrained.
[0037] The prediction model is established based on a long short-term memory network, and the prediction model includes an input layer, an LSTM layer and an output layer.
[0038] In this implementation, dividing the data to be processed into a training group and a validation group, and performing feature processing on these two groups of data, helps to establish a prediction model with generalization ability. The prediction model based on the long short-term memory network can better capture the long-term dependencies in time series data. The LSTM model has good modeling capabilities for processing time series data, especially data with long-term memory dependencies, which helps to improve the accuracy of the prediction model.
[0039] Specifying a time step to train the prediction model can help the model better understand the patterns and trends in the time series data, and can improve the model's prediction performance for future time points. By using the data in the validation group to verify the trained model and then evaluating it by comparing the predicted values with the true values, the performance of the model can be effectively tested. By setting the evaluation threshold and making judgments, it can be automatically determined whether the model needs to be retrained, thereby ensuring the robustness and sustainability of the model. By setting the evaluation threshold and implementing real-time performance monitoring, it is possible to promptly detect the degradation or insufficiency of the model's performance, so that appropriate measures can be taken, such as retraining the model, to maintain the accuracy and effectiveness of the prediction model.
[0040] In general, the benefit of this part is that it provides a complete training and validation process, combined with the use of LSTM models, so that the prediction model can better adapt to time series data, and maintain the high performance and robustness of the model through real-time performance monitoring mechanisms. This has practical application value for solving future prediction problems based on historical network traffic and resource requirements, especially in communication systems.
[0041] The calculation formula of the evaluation value is as follows: In the formula, Λ y is the evaluation value, a=1,2,3,...,A is the number of predicted values to be verified, RJ a is the ath predicted value to be verified, RY a For RJ a The corresponding a-th true value in the validation group, ΔR is the set allowable error value of the prediction model, and σ is the error adjustment factor.
[0042] In this embodiment, Λ in the formula y The error between all predicted values to be verified and their corresponding true values is considered comprehensively. This comprehensive error assessment helps to understand the overall performance of the model more comprehensively, rather than just the accuracy of a single predicted value. The error adjustment factor helps to consider the adjustment of errors in the evaluation, so as to measure the performance of the model more reasonably. The error adjustment factor is a setting that is used to adjust the error so that the sensitivity of the evaluation value to the model performance can be adjusted by adjusting σ.
[0043] ΔR, as the set error tolerance of the prediction model, indicates the tolerance for errors in the evaluation, which makes the evaluation value flexible and allows the error tolerance to be adjusted according to the specific application scenario to meet different performance requirements.
[0044] Calculate the square difference between each predicted value to be verified and its corresponding true value, add all the square differences, and take the square root of the above result. The calculation logic of this evaluation value is based on the accumulation of square differences, and comprehensively considers the error adjustment factor, the number of predicted values to be verified, and the set prediction model error allowance to quantitatively evaluate the performance of the prediction model.
[0045] In general, the benefit of this part is that it provides a comprehensive and flexible evaluation method that takes multiple factors into consideration, making the evaluation value more interpretable and practical, helping to monitor the performance of the prediction model in real time, promptly identify the deficiencies of the model, and retrain the model when necessary to ensure the robustness and accuracy of the model.
[0046] Specifically, Figure 4 As shown, the actual values of network traffic and resource requirements of the device to be processed are obtained, and the actual values are compared with the predicted values. The process of determining whether the prediction model needs to be adjusted based on the comparison results is as follows: the actual values of network traffic and resource requirements of the device to be processed are obtained according to the set period, and the predicted value predicted by the prediction model corresponding to each actual value is obtained; the deviation value between each actual value and the corresponding predicted value is judged, if the deviation value is greater than the set first deviation threshold, the reset index is increased by one, if the reset index is equal to the set reset threshold, the actual values of network traffic and resource requirements of the device to be processed are used to retrain and verify the prediction model, and the reset index is used to represent the number of times the prediction value of the prediction model is wrong.
[0047] In this implementation, the actual values of the network traffic and resource requirements of the device to be processed are obtained according to the set period to form a time series of the actual values. For each actual value, a prediction model is used to perform a prediction to obtain a corresponding predicted value.
[0048] Calculate the deviation between each actual value and the corresponding predicted value. If the deviation is greater than the set first deviation threshold, further dynamic adjustment is triggered. When the deviation exceeds the set threshold, the reset index is increased by one. If the reset index is equal to the set reset threshold, the prediction model is retrained and verified using the obtained actual values to adapt to the current network traffic and resource demand characteristics.
[0049] The goal of this part of the concept is to obtain the actual value in real time, compare the deviation between the predicted value and the actual value, and set thresholds and reset mechanisms, so that the prediction model can adapt to the changing network environment in a timely manner and improve accuracy and stability. Overall, through automation and dynamic adjustment, the system's sensitivity and response capabilities to actual conditions are enhanced, which helps to optimize the prediction model of network traffic and resource requirements.
[0050] Specifically, in the process of judging the deviation value between each actual value and the corresponding predicted value, if the deviation value is continuously greater than the set deviation threshold for a set number of consecutive times, the total deviation value within the consecutive times is calculated; if the total deviation value is greater than or equal to the second deviation threshold, the actual values of network traffic and resource requirements of the device to be processed are used to retrain and verify the prediction model.
[0051] The calculation formula for the total deviation value is as follows: Where Γ is the total deviation value, b=1,2,3,...,B is the number of consecutive times, YU b The bth predicted value that is continuously greater than the set deviation threshold, SHI b For YU b The corresponding b-th actual value, e is a natural constant.
[0052] In this implementation scheme, the actual values of the network traffic and resource requirements of the device to be processed are obtained according to the set period to form a time series of the actual values. For each actual value, a prediction model is used to predict and obtain the corresponding predicted value, and the deviation value between each actual value and the corresponding predicted value is determined. If the deviation value is continuously greater than the set deviation threshold for a set number of consecutive times, the next step is executed.
[0053] Calculate the total deviation value within the consecutive times, use the calculation formula of the total deviation value, if the total deviation value is greater than or equal to the set second deviation threshold, execute the next step, use the actual values of network traffic and resource requirements of the device to be processed to retrain and verify the prediction model to adapt to the current network traffic and resource demand characteristics.
[0054] The goal of the concept is to obtain actual values in real time, compare the deviation between predicted and actual values, and set thresholds and reset mechanisms so that the prediction model can adapt to the changing network environment in a timely manner and improve accuracy and stability. Overall, this concept enhances the system's sensitivity and response capabilities to actual conditions through automation and dynamic adjustment, which helps optimize the prediction model of network traffic and resource requirements.
[0055] In summary, this application has at least the following effects:
[0056] By periodically acquiring actual values, a continuous deviation judgment mechanism, and calculating the sum of deviation values, this method realizes real-time performance monitoring of the prediction model and can dynamically adjust the model to adapt to changes in network traffic and resource requirements, thereby improving the real-time performance and flexibility of the system.
[0057] The introduction of a mechanism for continuous deviation judgment and total deviation value calculation makes the system more adaptive and able to more stably judge whether the prediction model needs to be adjusted, which helps prevent the model from being overly sensitive to short-term noise and improves the stability of the system.
[0058] The calculation formula of the total deviation value is used to comprehensively consider the deviations within a continuous number of times, making the adjustment of the prediction model more comprehensive and accurate, which helps to reduce error accumulation and reduce the impact of sudden noise.
[0059] By regularly retraining and validating the prediction model, and adjusting the model based on actual values, this method helps to improve the robustness and performance of the model, so that the model can better adapt to complex network environments and changes in resource requirements.
[0060] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0061] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0064] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0065] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A communication big data processing method, characterized in that: The following steps are involved: Obtaining the data to be processed about the historical network traffic and resource requirements of the device to be processed and preprocessing the data to be processed, wherein the data to be processed includes bandwidth utilization, CPU utilization, and memory utilization; Processing the data to be processed based on the trained prediction model, predicting the predicted value of the device to be processed, and adjusting the future network traffic and resource requirements of the device to be processed based on the predicted value, wherein the predicted value is used to represent the future network traffic and resource requirements of the device to be processed; Obtain the actual values of network traffic and resource requirements of the device to be processed, compare the actual values with the predicted values, and determine whether the prediction model needs to be retrained and verified based on the comparison results; Preprocessing the data to be processed includes processing missing values. The specific process is as follows: Obtain the timestamp corresponding to the missing value, determine the pre-order data of several time points before the timestamp corresponding to the missing value and the post-order data of several time points after the timestamp corresponding to the missing value; Process the pre-order data and post-order data to obtain the inserted value, match the inserted value with the timestamp corresponding to the missing value, and fill in the inserted value; The formula to calculate the inserted value is as follows: Where, X c is the insertion value, i=1,2,3,...,n is the number of previous data, x i is the i-th data in the previous sequence data, j=1,2,3,...,m is the number of previous sequence data, x j is the jth data in the post-sequence data; The training process of the prediction model is as follows: Divide the preprocessed data into a training group and a validation group; Perform feature processing on the training group and validation group respectively; Input the feature-processed training group data into the prediction model; Train the prediction model according to the specified time step; After the prediction model is trained based on the data in the training group, the trained prediction model is verified using the data in the validation group. The process is as follows: Input the data in the validation group into the trained prediction model to obtain the prediction value to be verified; Compare the predicted value to be verified with the true value in the verification group, evaluate the trained prediction model, and obtain an evaluation value; The evaluation value is compared with the set evaluation threshold to judge the performance of the prediction model. If the evaluation value is lower than the set evaluation threshold, the prediction model is retrained; The prediction model is established based on a long short-term memory network, and the prediction model includes an input layer, an LSTM layer and an output layer; The calculation formula of the evaluation value is as follows: In the formula, Λ y is the evaluation value, a=1,2,3,...,A is the number of predicted values to be verified, RJ a is the ath predicted value to be verified, RY a For RJ a The corresponding a-th true value in the validation group, ΔR is the set error allowance of the prediction model, and σ is the error adjustment factor; The process of obtaining the actual values of network traffic and resource requirements of the device to be processed, comparing the actual values with the predicted values, and determining whether the prediction model needs to be adjusted based on the comparison results is as follows: Obtaining actual values of network traffic and resource requirements of the device to be processed according to a set period, and obtaining predicted values obtained by the prediction model corresponding to each actual value; Determine the deviation between each actual value and the corresponding predicted value. If the deviation is greater than a set first deviation threshold, the reset index is incremented by one. If the reset index is equal to the set reset threshold, the prediction model is retrained and verified using the actual values of the network traffic and resource requirements of the device to be processed. The reset index is used to indicate the number of prediction errors of the prediction model. In the process of determining the deviation value between each actual value and the corresponding predicted value, if the deviation value is continuously greater than the set deviation threshold for a set number of consecutive times, the sum of the deviation values within the consecutive times is calculated, and if the sum of the deviation values is greater than or equal to the second deviation threshold, the prediction model is retrained and verified using the actual values of the network traffic and resource requirements of the device to be processed; The calculation formula for the total deviation value is as follows: Where Γ is the total deviation value, b=1,2,3,...,B is the number of consecutive times, YU b For the bth predicted value that is continuously greater than the set deviation threshold, SHI b For YU b The corresponding b-th actual value, e is a natural constant.
Citation Information
Patent Citations
Communication big data processing method
CN107332869A
Method and device for updating prediction model
CN105095614A
Cloud service resource dynamic prediction management method
CN117076882A