Scenarized network performance evaluation method

Abstract

The invention provides a scenarized network efficiency evaluation method, which comprises the following steps of: acquiring an actual network efficiency parameter of a certain scene, and determining a plurality of first-level evaluation indexes and a plurality of second-level evaluation indexes which influence the network efficiency of the scene; establishing an analytic hierarchy model, and constructing a judgment matrix for all first-level evaluation indexes and second-level evaluation indexes; obtaining weight coefficients of all first-level evaluation indexes and second-level evaluation indexes according to the constructed judgment matrix; calculating a weighted score of each secondary evaluation index according to the actual network efficiency parameter, a preset scoring standard and a weight coefficient of the secondary evaluation index; according to the weight coefficient and the evaluation score of the first-level evaluation index, calculating a comprehensive evaluation score of the network efficiency of the scene under different scoring standards; according to the invention, the problems that the actual use effect of the network signal cannot be comprehensively and truly reflected and the network quality in different scenes cannot be accurately and comprehensively monitored and evaluated in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of network performance evaluation, and in particular to a scenario-based network performance evaluation method. Background Art

[0002] In today's rapid development of the communications industry, network performance evaluation has become a core task to ensure service levels and improve user satisfaction. The traditional evaluation system focuses too much on key performance indicators (KPIs). Although these indicators have shown a continuous increase in numbers, they may not be in sync with the real feelings of users. Although communication operators have conducted preliminary research based on KPIs, key quality indicators (KQIs) and user perception systems, and established evaluation standards for some common scenarios, it is still difficult to achieve accurate and comprehensive network quality monitoring and evaluation when dealing with changing and specific scenarios, which directly affects the applicability and accuracy of the evaluation results.

[0003] In addition, existing evaluation standards often use coverage as the main measurement indicator, fail to fully consider the actual load carried by the network and the user's usage behavior, and ignore the multi-dimensional characteristics of signal quality, including key factors such as signal strength, stability and interference level. The limitations of this evaluation method make it unable to fully capture the complexity of network quality, and the reliance on a single indicator makes the evaluation results unable to fully reflect the actual user experience. Even if the coverage is wide, the actual user experience may still be greatly reduced due to signal quality issues.

[0004] Therefore, existing network performance evaluation methods urgently need to be improved to achieve more targeted quality improvement measures and more accurately reflect users' needs and expectations in real-world usage scenarios. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a scenario-based network performance evaluation method, which solves the problems in the prior art that it is unable to fully and truly reflect the actual usage effect of network signals, and is unable to accurately and comprehensively monitor and evaluate the network quality in different scenarios, so as to accurately guide the direction of network quality improvement.

[0006] An embodiment of the present invention provides a scenario-based network performance evaluation method, comprising the following steps:

[0007] Obtaining actual network performance parameters of a certain scenario, and determining a number of primary evaluation indicators that affect the network performance of the scenario and a number of secondary evaluation indicators that affect each primary evaluation indicator;

[0008] Establish a hierarchical analysis model and construct a judgment matrix for all first-level evaluation indicators and second-level evaluation indicators;

[0009] Obtain the weight coefficients of all primary evaluation indicators and secondary evaluation indicators according to the constructed judgment matrix;

[0010] Calculate the weighted score of each secondary evaluation indicator according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation indicator;

[0011] Sum the weighted scores of the second-level evaluation indicators under each first-level evaluation indicator to obtain the evaluation score of the first-level evaluation indicator;

[0012] Based on the weight coefficient and evaluation score of the first-level evaluation indicator, the comprehensive evaluation score of the network performance of the scenario under different scoring standards is calculated.

[0013] As a further solution, the hierarchical analysis model includes a decision-making target layer, a criterion layer, a sub-criterion layer and a measure layer; wherein the decision-making target layer is the network performance in a certain scenario, the criterion layer is a number of first-level evaluation indicators that affect the network performance in a certain scenario, the sub-criterion layer includes a number of corresponding second-level evaluation indicators that affect each of the first-level evaluation indicators, and the measure layer is the corresponding strategy taken to achieve the decision-making target.

[0014] As a further solution, the several first-level evaluation indicators include network failure, network resources, coverage performance, access performance, retention performance, interference performance and user perception.

[0015] As a further solution, the construction of a judgment matrix for all first-level evaluation indicators and second-level evaluation indicators includes: constructing a judgment matrix A for all first-level evaluation indicators in the measurement layer according to the established hierarchical analysis model:

[0016]

[0017] Among them, a ij >1, a ii =1,i,j=1,2,3...n;a ij Indicates a i to a j The relative importance value of n represents the number of evaluation indicators in the judgment matrix; similarly, the judgment matrix B is constructed for all the secondary evaluation indicators in the sub-criteria layer corresponding to the primary evaluation indicators of the quasi-measurement layer.

[0018] As a further solution, the weight coefficients of all primary evaluation indicators and secondary evaluation indicators are obtained according to the constructed judgment matrix:

[0019] Step 301: Use Saaty 1-9 scaling method to determine the relative importance of each evaluation index in the quasi-measurement layer and the sub-quasi-measurement layer, and fill in the judgment matrix;

[0020] Step 302: Calculate the maximum eigenvalue and the corresponding eigenvector for each constructed judgment matrix;

[0021] Step 303: normalize the calculated feature vector to obtain a weight vector;

[0022] Step 304: Perform a consistency check on each constructed judgment matrix, and calculate the consistency index CI and the test coefficient CR; if CR<01, it is considered that the judgment matrix passes the consistency check, and the normalized weight vector can be used as the weight coefficient of the judgment matrix; if CR≥01, it is considered that the judgment matrix fails the consistency check, and the judgment matrix is ​​reconstructed until the test coefficient CR of each judgment matrix is ​​less than 01;

[0023] Step 305: Repeat steps 302 to 304 for each judgment matrix until the weight coefficients of all evaluation indicators that pass the consistency test in the criterion layer and the sub-criterion layer are obtained.

[0024] As a further solution, the weighted score of each secondary evaluation indicator is calculated according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation indicator, including the following steps:

[0025] Step 401, collect and obtain actual network performance parameters in a certain scenario; wherein the network performance parameters include PRB average utilization parameter, high load ratio parameter, CQI excellent rate parameter and user average downlink perceived rate parameter;

[0026] Step 402: Determine the benchmark score of each secondary evaluation indicator according to the obtained actual network performance parameters and the preset scoring criteria; wherein the preset scoring criteria include the daily mode scoring criteria and the holiday mode scoring criteria;

[0027] Step 403: multiply the weight coefficient of each secondary evaluation indicator by its corresponding benchmark score under the preset scoring standard to obtain a weighted score for the secondary evaluation indicator.

[0028] As a further solution, in step 301, the use of the Saaty 1-9 scaling method to determine the relative importance of the evaluation indicators in the quasi-measurement layer and the sub-quasi-measurement layer includes: comparing each evaluation indicator in the same layer with all other indicators in pairs, and quantifying the relative importance of each pair of evaluation indicators, and the quantification values ​​are as follows: the equal importance quantification value is 1, the slightly important quantification value is 3, the relatively strong important quantification value is 5, the strongly important quantification value is 7, the extremely important quantification value is 9, and the intermediate values ​​of two adjacent judgments are 2, 4, 6, and 8.

[0029] As a further solution, in step 304, the calculation formula of the consistency index CI is:

[0030]

[0031] The calculation formula of the test coefficient CR is:

[0032]

[0033] Among them, λmax is the maximum eigenvalue of the judgment matrix, which is calculated according to the weight vector of each evaluation index in the judgment matrix, n represents the number of evaluation indicators in the judgment matrix, and RI represents the random consistency index, which is obtained by simulating the randomly generated judgment matrix N times and statistically analyzing it. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of the steps of a scenario-based network performance evaluation method provided by the present invention;

[0035] Figure 2 A schematic diagram of a hierarchical analysis model provided by the present invention;

[0036] Figure 3 A flowchart of the steps for obtaining weight coefficients of all primary evaluation indicators and secondary evaluation indicators provided by the present invention;

[0037] Figure 4 A flow chart of the steps for calculating the weighted score of each secondary evaluation index provided by the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0039] A scenario-based network performance evaluation method aims to more accurately evaluate network performance by comprehensively considering multiple factors, and based on this, improve or optimize the corresponding communication network.

[0040] like Figure 1 As shown, this method mainly includes the following steps:

[0041] Obtaining actual network performance parameters of a certain scenario, and determining a number of primary evaluation indicators that affect the network performance of the scenario and a number of secondary evaluation indicators that affect each primary evaluation indicator;

[0042] Establish a hierarchical analysis model and construct a judgment matrix for all first-level evaluation indicators and second-level evaluation indicators;

[0043] Obtain the weight coefficients of all primary evaluation indicators and secondary evaluation indicators according to the constructed judgment matrix;

[0044] Calculate the weighted score of each secondary evaluation indicator according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation indicator;

[0045] Sum the weighted scores of the second-level evaluation indicators under each first-level evaluation indicator to obtain the evaluation score of the first-level evaluation indicator;

[0046] Based on the weight coefficient and evaluation score of the first-level evaluation indicator, the comprehensive evaluation score of the network performance of the scenario under different scoring criteria is calculated.

[0047] In this embodiment, first, the indicators with greater impact on network quality are extracted from each network platform system, and the evaluation values ​​of repeated indicators are merged. For example, the indicators of the scenario performance scoring system can be determined into seven categories, including network failure, network resources, coverage performance, access performance, retention performance, interference performance and user perception. Then, each first-level indicator is further analyzed to determine the second-level evaluation indicators that affect them, for example, the network speed is subdivided into data transmission rate and delay, etc. Then, a hierarchical structure model of evaluation indicators is constructed using the analytic hierarchy process (AHP). The model is implemented by constructing a judgment matrix for each first-level evaluation indicator and all the second-level evaluation indicators under its jurisdiction. In the process of constructing this judgment matrix, it is necessary to ensure that each indicator can be reasonably and systematically evaluated, and intuitively reflect the association between the first-level and second-level evaluation indicators and the relative priority or weight relationship. After obtaining the weight coefficients of all required first-level evaluation indicators and second-level evaluation indicators, the weights of each first-level evaluation indicator are combined with the weights of the corresponding second-level evaluation indicators by weighted summation to obtain the value that can ultimately represent the relative contribution of each component of the network performance evaluation system. Subsequently, the actual network performance data of a given application scenario is obtained, and the specific scores of each secondary evaluation indicator under the specific application scenario conditions are calculated based on the set scoring criteria. Finally, the scores of the secondary evaluation indicators under each primary evaluation indicator are summarized and weighted according to the weight coefficient of the primary indicator to obtain a comprehensive evaluation score of the network performance in the scenario. The final evaluation score can be used to identify the advantages and potential problems of network services and provide decision support for network optimization and resource allocation.

[0048] like Figure 2 As shown, Figure 2 The schematic diagram of the hierarchical analysis model is shown in FIG. The hierarchical analysis model includes a decision target layer, a criterion layer, a sub-criterion layer, and a measure layer; wherein the decision target layer is the network performance of a certain scenario, the criterion layer is a number of first-level evaluation indicators that affect the network performance of a certain scenario, the sub-criterion layer includes a number of corresponding second-level evaluation indicators that affect each of the first-level evaluation indicators, and the measure layer is the corresponding strategy adopted to achieve the decision target.

[0049] In this embodiment, the hierarchical analysis model structure defines the basic elements of network performance evaluation. The decision target layer represents the network performance evaluation in a specific scenario. The lower layer is the criterion layer, which contains multiple first-level evaluation indicators that affect the network performance of the target scenario. In order to analyze the internal composition and influencing factors of each first-level evaluation indicator in more detail, the sub-criterion layer is introduced. At this level, the second-level evaluation indicators corresponding to each first-level evaluation indicator are determined. As a specific measure for implementing the hierarchical analysis model, the measure layer is at the bottom. For all the above evaluation results, we can propose corresponding solutions and improvement measures to optimize the scenario-based network performance. Through such a top-down, layer-by-layer decomposition evaluation and optimization process, it is possible to more accurately find effective ways and specific implementation plans to improve network performance.

[0050] Specifically, when conducting a scientific and effective evaluation of scenario-based network performance, the present invention determines primary and secondary evaluation indicators to comprehensively consider multiple key factors affecting network efficiency and quality; wherein the primary evaluation indicators mainly include the following seven aspects: network failure, network resources, coverage performance, access performance, retention performance, interference performance and user perception; based on the primary evaluation indicators, the secondary evaluation indicators are divided in detail:

[0051] (1) There are four secondary evaluation indicators for network failure: cell availability (4G), warning cell (4G), cell availability (5G), and warning cell (5G).

[0052] (2) Network resources have four secondary evaluation indicators: PRB average utilization (4G), high load ratio (4G), PRB average utilization (5G), and high load ratio (5G).

[0053] (3) Coverage performance has five secondary evaluation indicators: 4G secondary evaluation indicators MR coverage rate, CQI quality rate (4G), 5G secondary evaluation indicators MR coverage rate, CQI quality rate (5G), 5G diversion ratio.

[0054] (4) Access performance has 17 secondary evaluation indicators: wireless connection success rate (4G), RRC connection establishment success rate (4G), E-RAB establishment success rate (4G), RRC reconstruction success rate (4G), VOLTE wireless connection rate (4G), QCI1 bearer establishment success rate (4G), QCI5 bearer establishment success rate (4G), S1 signaling connection establishment success rate (4G), RRC connection establishment success rate (NB_IOT), wireless establishment success rate (5G), RRC secondary evaluation indicator connection establishment success rate (5G), QOS secondary evaluation indicator flow establishment success rate (5G), NG signaling connection establishment success rate (5G), UE context establishment success rate (5G), RRC reconstruction success rate (5G), VONR secondary evaluation indicator access success rate (5G), VoNR voice establishment success rate (5G).

[0055] (5) There are 19 secondary evaluation indicators for maintenance performance: E-RAB drop rate (4G), intra-system switching success rate (4G), intra-frequency switching success rate (4G), inter-frequency switching success rate (4), QCI1 drop rate (4G), QCI5 drop rate (4G), RRC reconstruction ratio (4G), QosFlow drop rate (5G), intra-system switching success rate (5G), intra-site switching success rate (5G), Xn interface switching success rate (5G), Ng interface switching success rate (5G), inter-site switching success rate (5G), intra-frequency switching success rate (5G), inter-frequency switching success rate (5G), 5G down-cutting to 4G success rate (5G), RRC reconstruction ratio (5G), VoNR voice drop rate (5G), VONR down-cutting to VoLTE success rate (5G).

[0056] (6) Interference performance has four secondary evaluation indicators: average noise per RB (4G), proportion of interfering cells (4G), average noise per RB (5G), and proportion of interfering cells (5G).

[0057] (7) User perception has 8 secondary evaluation indicators: user average downlink perceived rate (4G), low perception cell ratio (4G), user average downlink perceived rate (5G), low perception cell ratio (5G), VOLTE voice uplink packet loss rate (4G), VOLTE voice downlink packet loss rate (4G), VoNR uplink packet loss rate (5G) and VoNR downlink packet loss rate (5G).

[0058] Figure 3 A flowchart of the steps for obtaining weight coefficients of all primary evaluation indicators and secondary evaluation indicators provided by the present invention.

[0059] refer to Figure 3 The present invention further discloses a specific method implementation step, namely, a specific means of constructing a judgment matrix for all primary evaluation indicators and secondary evaluation indicators.

[0060] Step 301: Use Saaty 1-9 scaling method to determine the relative importance of each evaluation index in the quasi-measurement layer and the sub-quasi-measurement layer, and fill in the judgment matrix;

[0061] Firstly, based on the constructed analytic hierarchy model, for all the first-level evaluation indicators of the criterion layer: network failure, network resources, coverage performance, access performance, retention performance, interference performance and user perception, the Saaty1-9 scaling method is used to determine the relative importance of each evaluation indicator in the criterion layer, and the judgment matrix A is constructed:

[0062]

[0063] Among them, a ij >0, a ii =1,i,j=1,2,3...n;a ij Indicates a i to a j The relative importance value of n represents the number of evaluation indicators in the judgment matrix;

[0064] Among them, the Saaty1-9 scaling method is used to determine the relative importance of each evaluation indicator in the quasi-measurement layer and the sub-quasi-measurement layer, including: comparing each evaluation indicator in the same layer with all other indicators in pairs, and quantifying the relative importance of each pair of evaluation indicators. The quantitative values ​​are as follows: the equal importance quantitative value is 1, the slightly important quantitative value is 3, the relatively important quantitative value is 5, the strongly important quantitative value is 7, the extremely important quantitative value is 9, and the intermediate values ​​of two adjacent judgments are 2, 4, 6, and 8.

[0065] Table A: Judgment matrix of primary evaluation indicators (criteria layer)

[0066] Network failure Online Resources Covering performance Access performance Maintaining performance Interference performance User perception Network failure 1.00 1.00 0.89 0.32 0.33 0.80 0.50 Online Resources 1.00 1.00 0.89 0.32 0.33 0.80 1.50 Covering performance 1.13 1.13 1.00 0.36 0.38 0.90 0.56 Access performance 3.13 3.13 2.78 1.00 1.04 2.50 1.56 Maintain system energy 3.00 3.00 2.67 0.96 1.00 2.40 1.50 Interference performance 1.25 1.25 1.11 0.40 0.42 1.00 0.63 User perception 2.00 2.00 1.78 0.64 0.67 1.60 1.00

[0067] The same operation flow can also be applied to all the secondary evaluation indicators of the criterion sub-layer corresponding to each primary indicator in the criterion layer, so as to form a corresponding judgment matrix B.

[0068] Specifically, the present invention uses the 1-9 scale system proposed by Saaty to evaluate the relative importance of each indicator in the quasi-measurement layer and the sub-quasi-measurement layer compared with other indicators, thereby filling in the judgment matrix. In the specific implementation process, it can be completed by adopting the advice of field experts or determining the specific quantitative values ​​of the relative importance of each indicator based on existing data.

[0069] Step 302: Calculate the maximum eigenvalue and the corresponding eigenvector for each constructed judgment matrix;

[0070] Eigenvectors of the judgment matrix The calculation formula is:

[0071]

[0072] The calculation formula of the maximum eigenvalue λmax of the judgment matrix is:

[0073]

[0074] In this embodiment, λ max =7.27.

[0075] Step 303: normalize the calculated feature vector to obtain a weight vector;

[0076] The feature vector obtained in step 302 Normalize the vectors so that the sum of the vectors is 1, that is, obtain the weight vector W, where W = (w1, w2, w3...w n ) T ;

[0077] In this embodiment, based on the calculated feature vector Then normalize the weights to get the weight vector W.

[0078] W=(0.08,0.08,0.09,0.25,0.24,0.1,0.16) T ;

[0079] Where T represents vector transpose.

[0080] Step 304: Perform a consistency check on each constructed judgment matrix, and calculate the consistency index CI and the test coefficient CR; if CR<01, it is considered that the judgment matrix passes the consistency check, and the normalized weight vector can be used as the weight coefficient of the judgment matrix; if CR≥01, it is considered that the judgment matrix fails the consistency check, and the judgment matrix is ​​reconstructed until the test coefficient CR of each judgment matrix is ​​less than 01;

[0081] In step 304, the calculation formula of the consistency index CI is:

[0082]

[0083] The calculation formula of the test coefficient CR is:

[0084]

[0085] Among them, λmax is the maximum eigenvalue of the judgment matrix, which is calculated according to the weight vector of each evaluation index in the judgment matrix, n represents the number of evaluation indicators in the judgment matrix, and RI represents the random consistency index, which is obtained by simulating the randomly generated judgment matrix 1000 times and by counting the RI value table.

[0086] Table 2: RI value table

[0087] Matrix order n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 RI 0 0 0.52 0.89 1.12 1.26 1.36 1.41 1.46 1.49 1.52 1.54 1.56 1.58 1.59

[0088] For example, the consistency index CI=0.05 is obtained by calculation, the order of the judgment matrix is ​​known to be 8, and RI=1.41 is selected according to Table 2, and then CR=0.03 is obtained according to the calculation formula of the test coefficient CR, and 0.03<0.1, indicating that the consistency degree of the judgment matrix A is considered to be within the allowable range.

[0089] Step 305: Repeat steps 302 to 304 for each judgment matrix until the weight coefficients of all evaluation indicators that pass the consistency check in the criterion layer and the sub-criterion layer are obtained.

[0090] Similarly, the relevant values ​​of all judgment matrices in the secondary evaluation indicators can be obtained by following the ascending steps.

[0091] Figure 4 A flow chart of the steps for calculating the weighted score of each secondary evaluation index provided by the present invention.

[0092] refer to Figure 4 , according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation index, the weighted score of each secondary evaluation index is calculated, including the following steps:

[0093] Step 401: Collect and obtain actual network performance parameters in a certain scenario; wherein the network performance parameters include a PRB average utilization parameter, a high load ratio parameter, a CQI quality rate parameter, and a user average downlink perceived rate parameter;

[0094] Step 402: Determine the benchmark score of each secondary evaluation indicator according to the obtained actual network performance parameters and the preset scoring criteria; wherein the preset scoring criteria include the daily mode scoring criteria and the holiday mode scoring criteria;

[0095] Step 403: multiply the weight coefficient of each secondary evaluation indicator by its corresponding benchmark score under the preset scoring standard to obtain a weighted score for the secondary evaluation indicator.

[0096] When implementing this scenario-based network performance evaluation method, we divided the evaluation into ten scenarios, including transportation hubs, scenic spots, rails, high-speed railways, highways, campuses, hospitals, residences, government centers, and business districts, to ensure that the evaluation covers a variety of network usage environments. For each scenario, actual network performance parameters that reflect network usage efficiency and performance status, such as PRB average utilization, high load ratio, CQI excellent rate, and user average downlink perceived rate, are collected to evaluate network performance in different scenarios based on preset scoring criteria for daily and holiday modes. These criteria can distinguish different needs and expectations of users for network services at different times or in specific environments; the weight coefficient of each secondary evaluation indicator is multiplied by the benchmark score determined according to the preset scoring criteria to obtain a weighted score, ensuring that the evaluation results not only reflect the performance of a single indicator, but also reflect the comprehensive effects of different indicators. Repeat the above steps to complete the score calculation of all secondary evaluation indicators in this scenario.

[0097] After completing the score calculation of each secondary evaluation indicator, add the weighted scores of all secondary evaluation indicators under the same primary evaluation indicator to obtain the summary score of the primary evaluation indicator, that is, the evaluation score. Each primary evaluation indicator will have a weight coefficient, which reflects the importance of the indicator in the entire network performance evaluation. Through the weight coefficient, the evaluation score of the primary evaluation indicator can be weighted and adjusted. Finally, the weighted first-level evaluation indicator evaluation scores are summarized to obtain the comprehensive evaluation score of the network performance of the scenario under different preset scoring standards. In addition, different scoring standards may be involved in the evaluation process, such as daily mode and holiday mode. According to the selected scoring standard, the calculation method of the evaluation score can be adjusted to meet the special needs of different scenarios.

[0098] The final comprehensive evaluation score can be used to analyze and compare network performance in different scenarios, and identify strengths and weaknesses in terms of network coverage, performance, and user satisfaction. This information can help network operators optimize network configuration, improve service quality, and meet user expectations and needs. Through this method, scenario-based network performance evaluation can not only provide detailed indicator analysis, but also give a comprehensive evaluation at the macro level, providing scientific and reasonable decision-making support for the management and optimization of communication networks.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A scenario-based network performance evaluation method, characterized in that: The steps include: Obtaining actual network performance parameters of a certain scenario, and determining a number of primary evaluation indicators that affect the network performance of the scenario and a number of secondary evaluation indicators that affect each primary evaluation indicator; Establish a hierarchical analysis model and construct a judgment matrix for all first-level evaluation indicators and second-level evaluation indicators; Obtain the weight coefficients of all primary evaluation indicators and secondary evaluation indicators according to the constructed judgment matrix; Calculate the weighted score of each secondary evaluation indicator according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation indicator; Sum the weighted scores of the second-level evaluation indicators under each first-level evaluation indicator to obtain the evaluation score of the first-level evaluation indicator; Based on the weight coefficient and evaluation score of the first-level evaluation indicator, the comprehensive evaluation score of the network performance of the scenario under different scoring standards is calculated.

2. A scenario-based network performance evaluation method according to claim 1, characterized in that: The hierarchical analysis model includes a decision-making target layer, a criterion layer, a sub-criterion layer and a measure layer; wherein the decision-making target layer is the network performance of a certain scenario, the criterion layer is a number of first-level evaluation indicators that affect the network performance of a certain scenario, the sub-criterion layer includes a number of second-level evaluation indicators that affect each first-level evaluation indicator, and the measure layer is the corresponding strategy taken to achieve the decision-making target.

3. A scenario-based network performance evaluation method according to claim 2, characterized in that: The several first-level evaluation indicators include network failure, network resources, coverage performance, access performance, retention performance, interference performance and user perception.

4. A scenario-based network performance evaluation method according to claim 2, characterized in that: The construction of a judgment matrix for all the first-level evaluation indicators and the second-level evaluation indicators includes: constructing a judgment matrix A for all the first-level evaluation indicators in the measurement layer according to the established hierarchical analysis model: Among them, a ij >0, a it =1,i,j=1,2,3...n;a ij Indicates a i to a j The relative importance value of n represents the number of evaluation indicators in the judgment matrix; similarly, the judgment matrix B is constructed for all the secondary evaluation indicators in the sub-criteria layer corresponding to the primary evaluation indicators of the quasi-measurement layer.

5. A scenario-based network performance evaluation method according to claim 2, characterized in that: The weight coefficients of all primary evaluation indicators and secondary evaluation indicators are obtained according to the constructed judgment matrix: Step 301: Use Saaty 1-9 scaling method to determine the relative importance of each evaluation index in the quasi-measurement layer and the sub-quasi-measurement layer, and fill in the judgment matrix; Step 302: Calculate the maximum eigenvalue and the corresponding eigenvector for each constructed judgment matrix; Step 303: normalize the calculated feature vector to obtain a weight vector; Step 304: Perform consistency check on each constructed judgment matrix and calculate consistency index CI and test coefficient CR; If CR<01, the judgment matrix is ​​considered to have passed the consistency test, and the normalized weight vector can be used as the weight coefficient of the judgment matrix; if CR≥01, the judgment matrix is ​​considered to have failed the consistency test, and the judgment matrix is ​​reconstructed until the test coefficient CR of each judgment matrix is ​​less than 1; Step 305: Repeat steps 302 to 304 for each judgment matrix until the weight coefficients of all evaluation indicators that pass the consistency check in the criterion layer and the sub-criterion layer are obtained.

6. A scenario-based network performance evaluation method according to claim 1, characterized in that: The step of calculating the weighted score of each secondary evaluation indicator according to the actual network performance parameter, the preset scoring standard and the weight coefficient of the secondary evaluation indicator comprises the following steps: Step 401: Collect and obtain actual network performance parameters in a certain scenario; Step 402: Determine the benchmark score of each secondary evaluation indicator according to the obtained actual network performance parameters and the preset scoring standard; Step 403: multiply the weight coefficient of each secondary evaluation indicator by its corresponding benchmark score under the preset scoring standard to obtain a weighted score for the secondary evaluation indicator.

7. A scenario-based network performance evaluation method according to claim 1, characterized in that: The actual network performance parameters include a PRB average utilization parameter, a high load ratio parameter, a CQI quality rate parameter and a user average downlink perceived rate parameter.

8. A scenario-based network performance evaluation method according to claim 1, characterized in that: The preset scoring criteria include daily mode scoring criteria and holiday mode scoring criteria.

9. A scenario-based network performance evaluation method according to claim 5, characterized in that: In step 301, the use of the Saaty 1-9 scaling method to determine the relative importance of the evaluation indicators in the quasi-measurement layer and the sub-quasi-measurement layer includes: comparing each evaluation indicator in the same layer with all other indicators in pairs, and quantifying the relative importance of each pair of evaluation indicators, and the quantification values ​​are as follows: the equal importance quantification value is 1, the slightly important quantification value is 3, the relatively strong important quantification value is 5, the strongly important quantification value is 7, the extremely important quantification value is 9, and the intermediate values ​​of two adjacent judgments are 2, 4, 6, and 8.

10. A scenario-based network performance evaluation method according to claim 5, characterized in that: In step 304, the calculation formula of the consistency index CI is: The calculation formula of the test coefficient CR is: Among them, λmax is the maximum eigenvalue of the judgment matrix, which is calculated according to the weight vector of each evaluation index in the judgment matrix, n represents the number of evaluation indicators in the judgment matrix, and RI represents the random consistency index, which is obtained by simulating the randomly generated judgment matrix N times and statistically analyzing it.

Images