Efficiency evaluation method based on big data machine learning

Abstract

The invention discloses an efficiency evaluation method based on big data machine learning, and the method comprises the steps: 1, constructing an efficiency evaluation index system, and carrying out the evaluation task deduction simulation of the system, and obtaining a simulation data sample; and 2, training a simulation data sample by adopting a machine learning algorithm, fitting an efficiency evaluation meta-model of each node of the network, and carrying out model training optimization by adopting a particle swarm algorithm and a self-adaptive nearest distance threshold algorithm to obtain a final efficiency evaluation model so as to carry out efficiency evaluation. According to the method, the accuracy of model training can be continuously improved on the basis of model operation, an objective modeling method is provided for efficiency evaluation modeling of a network link formed by overall networking or multiple nodes so as to know the overall network condition and the quality condition of a data transmission link, meanwhile, the calculation efficiency of a machine learning algorithm is optimized, and the efficiency of the data transmission link is improved. The computing resources are saved, and the accuracy of networking and network link efficiency evaluation results is improved.

Description

Technical Field

[0001] The present invention belongs to the field of smart network technology, and specifically relates to a method for performance evaluation based on big data machine learning. Background Art

[0002] BP (Back Propagation) neural network algorithm is a widely used method in neural network. It was proposed by a scientific research team led by Rumelhart and McCelland in 1986. It is a multi-layer feedforward network trained by the error back propagation algorithm, namely BP neural network. The most commonly used structure is the three-layer BP neural network. BP neural network stores a large number of input / output relationships through memory function, and does not need to determine the mathematical calculation model of the input / output relationship in advance. When there are input variables of the same nature, BP neural network will use the existing corresponding relationship to automatically obtain the output value.

[0003] The BP neural network is trained continuously by using the gradient descent algorithm, and the training error results are fed forward to continuously adjust the weights and thresholds of the network units that affect the input / output relationship until the error is within an acceptable range. The corresponding relationship will be stored in the network, and the accurate output value can be calculated by relying on the existing memory of the re-input variable. Therefore, it is also called a multi-layer feedforward network.

[0004] The problem of whether the performance evaluation of many complex network systems is accurate is a difficulty faced by BP neural networks. Although the existing analytical methods have solved the problem of network performance evaluation to a certain extent, there are obvious subjective factors in the evaluation process, especially the problem of non-objective weighting of evaluation indicators. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for performance evaluation based on big data machine learning in view of the deficiencies of the above-mentioned prior art.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0007] A method for performance evaluation based on big data machine learning, comprising:

[0008] Step 1: Construct an effectiveness evaluation index system, and simulate the evaluation task to obtain simulation data samples;

[0009] Step 2: Use machine learning algorithms to train simulation data samples, fit the performance evaluation meta-model of each node in the network, and use particle swarm algorithm and adaptive nearest neighbor distance threshold algorithm to optimize model training to obtain the final performance evaluation model for performance evaluation.

[0010] To optimize the above technical solutions, the specific measures taken also include:

[0011] The performance evaluation index system described in the above step 1 includes performance evaluation input indicators and corresponding node performance; the performance evaluation input indicators include the memory, CPU, light attenuation value and computing power quantization value of the network node.

[0012] In the above step 1, the evaluation equipment and evaluation scenario are determined according to the evaluation task, and then a simulation deduction system is designed. Simulation experiments are designed in combination with the performance evaluation index system, and simulation data samples are obtained through the simulation deduction system and simulation experiments.

[0013] In the above step 2, a machine learning algorithm is used to train the simulation data sample to fit the performance evaluation meta-model of each node in the network, including:

[0014] Divide the simulation data samples to obtain a training sample set and a test sample set;

[0015] The machine learning algorithm is used to train the data of the training sample set, and the performance evaluation meta-model of each node in the network is obtained by fitting;

[0016] The sample prediction results of the meta-model are tested using the data of the test sample set. If the preset requirements are not met, additional training sample points are added, the machine learning algorithm is reselected to change the meta-model type, and re-training is performed until the sample prediction results of the meta-model meet the preset requirements.

[0017] The above-mentioned machine learning algorithms include BP algorithm, SVR algorithm, GMDH algorithm and DBN algorithm.

[0018] The above step 2 uses a particle swarm algorithm and an adaptive nearest neighbor distance threshold algorithm to perform model training optimization, including:

[0019] The particle swarm algorithm is used to optimize the network and optimize the computational efficiency of the machine learning algorithm;

[0020] The adaptive nearest neighbor distance threshold algorithm is used to dynamically set the threshold for clustering the performance evaluation input indicators of the nodes to filter out the training data and save computing resources.

[0021] The standard evolution formula used by the above particle swarm algorithm is as follows:

[0022]

[0023] Where: w is the inertia weight; c 1 、c 2 is the acceleration factor; r 1 , r 2 ∈rand[0,1]; are the speed, position, individual extremum optimal position and group extremum optimal position of the j-th dimension variable of parameter i in the k-th iteration respectively;

[0024] The particle swarm algorithm enhances the local optimization ability in the late stage of evolution based on the following formula:

[0025] w=w max -(w max -w min )k / T

[0026] C 1 =C max -(C max -C min )k / T

[0027] C 2 =C max +(C max -C min )k / T

[0028] Where: w max , w min are the maximum and minimum values ​​of the initial inertia weight respectively; C max , C min are the maximum and minimum values ​​of the initial acceleration factor respectively; w, C 1 , C 2 are the inertia weight, the first acceleration factor value, and the second acceleration factor value of the kth selection generation respectively; T is the optimization coefficient.

[0029] The fitness calculation method of the particle swarm algorithm is as follows: first, the difference between the output value of the sample observation data trained by the machine learning algorithm model and the true value is calculated, and then the squares of all the differences are summed and averaged.

[0030] The above-mentioned adaptive nearest neighbor distance threshold algorithm uses the following formula to dynamically set the threshold:

[0031]

[0032] In the formula, max is the maximum value of the distance between samples; min is the minimum value of the distance between samples; a is the adjustment parameter; is the average distance between samples.

[0033] The present invention has the following beneficial effects:

[0034] The present invention proposes to evaluate the performance of network nodes in the network, and to combine the performance evaluation model based on the metamodel to perform network evaluation, so as to understand the overall network situation and the quality of the data transmission link; the performance evaluation modeling process based on the metamodel of the present invention can continuously improve the accuracy of model training on the basis of model calculation; by using a machine learning algorithm to fit the performance evaluation metamodel of each node in the network, the correlation relationship of the evaluation index system can be excavated, and the evaluation index is given an objective weight, which provides an objective modeling method for the overall network or the network link performance evaluation modeling composed of multiple nodes. The present invention adopts a particle swarm (PSO) algorithm to optimize the initial weight and threshold of the network to optimize the calculation efficiency of the machine learning algorithm, and filters out some training data of little value through an adaptive nearest neighbor distance threshold algorithm to save computing resources and improve the accuracy of the network and network link performance evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of the method for performance evaluation based on big data machine learning of the present invention;

[0036] Figure 2 This is a schematic diagram of the method for performance evaluation based on big data machine learning of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.

[0039] like Figure 1-2 As shown, a method for performance evaluation based on big data machine learning of the present invention comprises:

[0040] Step 1: Construct an effectiveness evaluation index system, and simulate the evaluation task to obtain simulation data samples;

[0041] In the embodiment, the performance evaluation index system includes performance evaluation input indicators and corresponding node performance; the performance evaluation input indicators include the memory, cpu, light attenuation value and computing power quantization value of the network node, and the node performance refers to the node idle status. Construct a BP-based neural network to conduct network evaluation on the indicators of network nodes in the network in combination with the performance evaluation model based on the metamodel, so as to understand the overall network situation and the quality of the data transmission link. It mainly includes four indicators (memory, cpu, light attenuation and computing power quantization). The transmission data of the current node (four indicator data sets) is evaluated to obtain the current node status (resource idle status). The present invention also proposes a metamodel-based performance evaluation modeling process, which can continuously improve the accuracy of model training on the basis of model calculations compared to the prior art that only provides model calculations to obtain results. The specific introduction is as follows:

[0042] 1) The indicators of BP neural network training data are classified and aggregated, and each indicator generates a data set with four data sets. This is to facilitate the acquisition of the memory, CPU, light attenuation value and computing power quantization value of each network node in the log data (how much computing power the current traffic requires for the network node to perform logical processing).

[0043] The optical attenuation value adopts the industry-wide standard (-25DB) as the critical value of the network quality standard. The normal value of optical fiber attenuation is around 25DB. Generally speaking, the maximum optical attenuation allowed by optical fiber is -40DB, but in order to achieve a stable effect, it is recommended that the optical loss should not be greater than -25DB, because -25DB is the critical value for the normal and stable operation of optical terminal equipment.

[0044] Map the historical or current transmission data into a computing task. Each computing task has four data sets to calculate the total computing power and obtain the computing power quantification value of the computing task. Assuming there are n logic operation chips, m parallel computing chips and p neural network acceleration chips, the computing power requirements of the business can be uniformly described as the following computing power quantification model:

[0045]

[0046] In the formula, C br is the idle computing power, n is the number of logic operation chips, i is the index of n, f(a i ) is the mapping function of logical operation, α i is the mapping ratio coefficient of the i-th logic operation chip, q 1 (TOPS) is the redundant computing power of logical operations; m is the number of parallel computing chips, j is the index of m, f(b j ) is the mapping function for parallel computing, β j is the mapping ratio coefficient of the jth parallel computing chip, q 2(FLOPS) is the redundant computing power of parallel computing; p is the number of neural network acceleration chips, k is the index of p, f(c k ) is the mapping function of neural network acceleration, γ k is the mapping ratio coefficient of the kth neural network acceleration chip, q 3 (FLOPS) is the redundant computing power for neural network acceleration.

[0047] Taking parallel computing capability as an example, assuming there are three different types of parallel computing chip resources: b1, b2, and b3, then f(b j ) represents the mapping function of the parallel computing capability that the j-th parallel computing chip b can provide, and q2 represents the redundant computing power of parallel computing.

[0048] 2) Performance evaluation model, which evaluates the transmission data of the current node (four indicator data sets) to obtain the current node status (resource idleness).

[0049] The essence of the performance evaluation process is to solve a nonlinear equation with multiple objective inputs, which can be described by a mathematical model:

[0050]

[0051] Where: is the evaluation index set; is an n-dimensional real vector space: F(X) is a nonlinear objective function in an n-dimensional index space. i (X), h j (X) are the inequality constraints and equality constraints of the objective function respectively.

[0052] For neural networks, the objective function space F(X) of performance evaluation is very complex and usually cannot be described by specific mathematical expressions. It is a typical "black box system". Traditional mathematical analytical methods cannot accurately describe this "black box system", and other scientifically feasible and highly reliable methods need to be considered.

[0053] 3) Metamodel-based performance evaluation modeling process

[0054] Metamodel technology is a simplified model that uses the I / O data of the problem to fit the algorithm, thereby obtaining the internal laws of the problem. It provides a means of describing the internal mechanism of a complex system. The performance evaluation metamodel construction process of the present invention mainly includes the following four steps:

[0055] ① Design simulation scenarios.

[0056] According to the performance evaluation object, evaluation task and combat scenario requirements, combat simulation scenarios are designed to simulate the performance of a combat mission of the equipment to be evaluated, laying the foundation for obtaining performance evaluation data.

[0057] ② Establish an evaluation index system.

[0058] The significance of the evaluation index system is to guide the experimental design of simulation evaluation input / output indicators. By analyzing the evaluation tasks, the objects to be evaluated and the mechanism of the network confrontation process, the index elements that affect the network performance under confrontation conditions (memory, CPU, light attenuation and computing power quantization) and the output form of the performance results are determined, thus constructing a hierarchical evaluation index system.

[0059] ③Simulation experiment design.

[0060] The purpose of the experimental design is to obtain a combination of evaluation index parameters that is evenly distributed and can traverse all possible situations in the performance evaluation process, and to provide training samples and test sample data that contain general rules for the construction of the performance evaluation metamodel.

[0061] ④Select the meta-model and perform regression analysis.

[0062] According to the characteristics of the performance evaluation problem and the evaluation index system structure, and considering the efficiency of the metamodel algorithm for regression analysis, a suitable metamodel is selected. There are many methods for constructing performance evaluation metamodels, including artificial neural networks, support vector regression machines, GMDH algorithms, deep belief network models, etc.

[0063] Step 2: Use machine learning algorithms to train simulation data samples, fit the performance evaluation meta-model of each node in the network, and use particle swarm algorithm and adaptive nearest neighbor distance threshold algorithm to optimize model training to obtain the final performance evaluation model for performance evaluation.

[0064] Based on the specific introduction of step one above, the present invention is a method for constructing a BP network performance evaluation model based on a machine learning algorithm. By using a machine learning algorithm (such as a BP algorithm, an SVR algorithm, a GMDH algorithm, and a DBN algorithm), the performance evaluation meta-model of each node in the network is fitted, the correlation of the evaluation index system is excavated, and objective weights are given to the evaluation indexes, providing an objective modeling method for the overall networking or network link performance evaluation modeling composed of multiple nodes. Innovation First, the particle swarm (PSO) algorithm is used to optimize the initial weights and thresholds of the BP neural network to optimize the calculation efficiency of the BP algorithm. Secondly, the adaptive nearest neighbor distance threshold and the threshold for clustering the four indicators of the current node are dynamically set to filter out some training data of little value, thereby saving computing resources and improving the accuracy of the networking and network link performance evaluation results.

[0065] Among them, the specific method of using particle swarm (PSO) algorithm to optimize the initial weights and thresholds of BP neural network is as follows:

[0066] Taking the BP model as an example, when faced with large-scale data calculations, its computational efficiency is low and it is easy to fall into local optimality. The particle swarm optimization (PSO) algorithm is used to optimize the initial weights and thresholds of the BP neural network to optimize the computational efficiency of the BP algorithm. The PSO optimization algorithm is derived from the foraging behavior of simulated bird flocks, and the standard evolution formula is as follows:

[0067]

[0068] Where: w is the inertia weight; c 1 、c 2 is the acceleration factor; r 1 , r 2 ∈rand[0,1]; They are respectively the speed, position, individual extremum optimal position and group extremum optimal position of the j-th dimension variable of parameter i in the k-th iteration.

[0069] The improved particle swarm algorithm is used to solve the initial value and threshold of the BP model, and (Formula 3) is used to enhance the local optimization ability in the later stage of evolution, which can give full play to the search ability of the particles themselves and the cognitive ability of all particle groups.

[0070] w=w max -(w max -w min )k / T

[0071] C 1 =C max -(C max -C min )k / T

[0072] C 2 =C max +(C max -C min )k / T (Formula 3)

[0073] Where: w max , w min are the maximum and minimum values ​​of the initial inertia weight respectively; C max , C min are the maximum and minimum values ​​of the initial acceleration factor respectively; w, C 1 , C 2 are the inertia weight, the first acceleration factor value, and the second acceleration factor value of the kth generation respectively; k is the number of iterations.

[0074] The fitness function of the particle swarm algorithm is defined as the error obtained by BP neural network training, that is,

[0075]

[0076] Where: N, m are the number of samples and the dimension of observation data respectively; pij , t ij They are respectively the output value and true value of the j-dimensional observation data of the i-th sample trained by the BP algorithm model.

[0077] Adaptive nearest neighbor distance threshold, dynamically set the threshold for clustering the four indicators of the current node, filter out some training data of little value to save computing resources and improve the accuracy of networking and network link performance evaluation results.

[0078] When setting the nearest neighbor distance threshold in the target reduction method, if the threshold is too small, there will be too many optimization targets in a cluster, and if the threshold is too large, outliers will be mistakenly absorbed into a certain class. For this reason, an adaptive threshold calculation method is designed here to determine the value size according to the current sample point status. The following is the threshold calculation formula:

[0079]

[0080] In the formula: max is the maximum value of the distance between samples; min is the minimum value of the distance between samples; is the adjustment parameter; is the average distance between samples.

[0081] The value a in the formula is adaptively adjusted by the current clustering situation. If the current number of (network node) clusters (number of growing trees) is greater than celling (M / 3), the number of optimization targets that have been clustered is small and there are more clusters. At this time, a smaller value a is set to reduce the nearest neighbor distance threshold, promote further clustering of the algorithm, and reduce the number of final clusters so that the target can be further simplified; otherwise, the threshold is increased to increase the nearest neighbor distance threshold and increase the number of clusters. Adding the submean value to the nearest neighbor distance threshold formula can better consider the distribution of samples and avoid errors caused by the aggregation of distance values ​​between samples.

[0082] In summary, the present invention first constructs a BP-based neural network to evaluate the network indicators of network nodes in the network in combination with a meta-model-based performance evaluation model, so as to understand the overall network situation and the quality of the data transmission link. It mainly includes four indicators (memory, cpu, light attenuation and computing power quantization). Secondly, the transmission data of the current node (four indicator data sets) is evaluated to obtain the current node state (resource idle situation). Then the performance evaluation modeling process based on the meta-model is executed, which can continuously improve the accuracy of model training on the basis of model operation compared with the prior art that only provides model operation to obtain results. Finally, a BP network performance evaluation model construction method based on a machine learning algorithm is used to fit the performance evaluation meta-model of each node in the network by adopting a machine learning algorithm (such as a BP algorithm, an SVR algorithm, a GMDH algorithm, and a DBN algorithm), and the correlation of the evaluation index system is excavated, and objective weights are given to the evaluation index, which provides an objective modeling method for the overall networking or network link performance evaluation modeling composed of multiple nodes. The particle swarm (PSO) algorithm is innovatively used to optimize the initial weights and thresholds of the BP neural network to optimize the computational efficiency of the BP algorithm, and the nearest neighbor distance threshold is adaptively set. The threshold for clustering the four indicators of the current node is dynamically set to filter out some training data of little value, thereby saving computing resources and improving the accuracy of networking and network link performance evaluation results.

[0083] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0084] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for performance evaluation based on big data machine learning, characterized in that: include: Step 1: Construct an effectiveness evaluation index system, and simulate the evaluation task to obtain simulation data samples; Step 2: Use machine learning algorithms to train simulation data samples, fit the performance evaluation meta-model of each node in the network, and use particle swarm algorithm and adaptive nearest neighbor distance threshold algorithm to optimize model training to obtain the final performance evaluation model for performance evaluation.

2. The method for performance evaluation based on big data machine learning according to claim 1, characterized in that: The performance evaluation index system in step 1 includes performance evaluation input indicators and corresponding node performance; the performance evaluation input indicators include the memory, CPU, light attenuation value and computing power quantization value of the network node.

3. The method for performance evaluation based on big data machine learning according to claim 1, characterized in that: The step 1 determines the evaluation equipment and evaluation scenario according to the evaluation task, and then designs a simulation deduction system. At the same time, a simulation experiment is designed in combination with the performance evaluation index system, and simulation data samples are obtained through the simulation deduction system and simulation experiments.

4. The method for performance evaluation based on big data machine learning according to claim 1, characterized in that: Step 2 uses a machine learning algorithm to train simulation data samples and fit a meta-model for evaluating the performance of each node in the network, including: Divide the simulation data samples to obtain a training sample set and a test sample set; The machine learning algorithm is used to train the data of the training sample set, and the performance evaluation meta-model of each node in the network is obtained by fitting; The sample prediction results of the meta-model are tested using the data of the test sample set. If the preset requirements are not met, additional training sample points are added, the machine learning algorithm is reselected to change the meta-model type, and re-training is performed until the sample prediction results of the meta-model meet the preset requirements.

5. The method for performance evaluation based on big data machine learning according to claim 1, characterized in that: The machine learning algorithms include BP algorithm, SVR algorithm, GMDH algorithm and DBN algorithm.

6. The method for performance evaluation based on big data machine learning according to claim 1, characterized in that: Step 2 uses a particle swarm algorithm and an adaptive nearest neighbor distance threshold algorithm to optimize model training, including: The particle swarm algorithm is used to optimize the network and optimize the computational efficiency of the machine learning algorithm; The adaptive nearest neighbor distance threshold algorithm is used to dynamically set the threshold for clustering the performance evaluation input indicators of the nodes to filter out the training data and save computing resources.

7. The method for performance evaluation based on big data machine learning according to claim 6, characterized in that: The standard evolution formula used by the particle swarm algorithm is as follows: Where: w is the inertia weight; c1, c2 are acceleration factors; r1, r2∈rand[0,1]; They are respectively the speed, position, individual extremum optimal position and group extremum optimal position of the j-th dimension variable of parameter i in the k-th iteration.

8. The method for performance evaluation based on big data machine learning according to claim 6, characterized in that: The particle swarm algorithm enhances the local optimization ability in the late stage of evolution based on the following formula: ω=w max -(w max -w min )k / T C1=C max -(C max -C min )k / T C2=C max +(C max -C min )k / T Where: w max , w min are the maximum and minimum values ​​of the initial inertia weight respectively; C max , C min are the maximum and minimum values ​​of the initial acceleration factor respectively; ω, C1, C2 are the inertia weight, the first acceleration factor value, and the second acceleration factor value of the kth selection respectively; T is the optimization coefficient.

9. The method for performance evaluation based on big data machine learning according to claim 6, characterized in that: The fitness calculation method of the particle swarm algorithm is as follows: first, the difference between the output value of the sample observation data trained by the machine learning algorithm model and the true value is calculated, and then the squares of all the differences are summed and averaged.

10. The method for performance evaluation based on big data machine learning according to claim 7, characterized in that: The adaptive nearest neighbor distance threshold algorithm uses the following formula to dynamically set the threshold: In the formula, max is the maximum value of the distance between samples; min is the minimum value of the distance between samples; a is the adjustment parameter; is the average distance between samples.

Images