Method and device for evaluating poor-quality cell of voice service

Abstract

The application discloses a kind of voice service quality poor cell evaluation method and device, the method is applied to the network side of 5G communication system, comprising: constructing voice MOS fitting model;In the statistical period of statistical period, the MR data generated when 5G user initiates VoNR voice service under 5G cell under target area is obtained, the MR data includes SS-RSRP data, SS-SINR data and the downlink PRB utilization of main service cell;SS-RSRP data, SS-SINR data in MR data and the downlink PRB utilization of main service cell are substituted into voice MOS fitting model, and the MOS prediction value of all cell user voice is obtained;Based on the MOS prediction value of all cell user voice, determine the VoNR voice quality poor 5G cell under target area in the statistical period of statistical period;VoNR voice quality poor 5G cell under target area in the statistical period of statistical period is sorted.The method can obtain the evaluation result of voice service quality poor cell comprehensively, truly and objectively.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication technology, and in particular to a method and device for evaluating a poor-quality voice service cell. BACKGROUND

[0002] Voice service is one of the important services of operators in the 5G era, and providing stable and high-quality voice service for users is an important part of improving 5G experience. In order to improve user perception, shorten call setup delay, and enhance the intelligibility of real-time voice communication, 3GPP standards introduce VoNR (Voice over New Radio) based on 5G and IMS (IP Multimedia Subsystem). VoNR is the target voice solution for operators. VoNR can fully utilize the advantages of 5G large bandwidth and high spectral utilization of new air interface / antenna technology, strong anti-fading characteristics, and provide users with shorter voice call access delay and ultra-high-definition voice experience by using EVS (Enhanced Voice Services) encoding mode. However, VoNR is completely carried by 5G NR network, and the voice quality is strongly related to network coverage and antenna transceiver performance. How to accurately evaluate the capability of 5G network carrying VoNR becomes a key link to guarantee VoNR service perception.

[0003] Since 4G VoLTE, the MOS (Mean Opinion Score) has become the mainstream solution for operation and maintenance personnel to evaluate voice quality. However, the existing technology still relies on the DT / CQT test of the existing network to obtain the voice MOS value. This method cannot be done all-weather, all-time, and all-region, and requires a large amount of manpower and material resources for long-term data collection, which not only has high cost, but also human operation errors or accidental events can affect the accuracy of VoLTE MOS, making it difficult to achieve comprehensive, true and objective evaluation of poor-quality voice service cells.

[0004] Under this background, however, the MOS value is still obtained in most cases by relying on the DT / CQT test of the existing network, which cannot be done all-weather, all-time, and all-region, and requires a large amount of manpower and material resources for long-term data collection, which not only has high cost, but also human operation errors or accidental events can affect the accuracy of voice MOS.

[0005] The voice service end-to-end perception usually adopts the method of voice MOS (Mean Opinion Score) scoring, which is divided into subjective evaluation and objective evaluation.

[0006] The subjective evaluation is completed by artificially receiving and perceiving the voice quality and scoring in a subjective manner. The method needs to consume a large amount of manpower and time, and the score has certain randomness, and therefore the objective evaluation method is recommended in the specification.

[0007] The objective evaluation has two modes of active mode and passive mode. The active mode usually initiates a test service on the terminal side by using a terminal and a test instrument, and evaluates based on the comparison between the original voice signal and the distorted voice signal. The objective evaluation in this mode usually uses a numerical distance or an auditory model describing how the auditory system perceives the quality to quantify the voice quality, and the main scoring algorithms include PSQM / PSQM+, PAMS, PESQ and POLQA, etc. The passive mode directly evaluates the voice quality by converting the delay, jitter and packet loss of the actual voice service output signal into MOS scores, and the main scoring algorithm is E-Model. The mode is mainly completed by collecting indexes of the voice service occurring in real time in the network. As shown in the following table, it is a corresponding table of MOS scores and user experience.

[0008] Table: Corresponding table of MOS scores and user experience

[0009]

[0010] Taking the VoLTE voice service as an example, for the same voice service, the MOS value of the VoLTE voice service is obviously different with the change of the wireless environment under different frequency points, different bandwidths and different device forms, and the difference of the corresponding MOS under the same RSRP can reach 0.5.

[0011] Similarly, the MOS value of the voice service also has differences under different network loads. Therefore, in order to reasonably evaluate the MOS quality of the VoNR voice service under the 5G network, it is necessary to innovatively design a method to collect data for a long time all day, all period and all region, and comprehensively, truly and objectively obtain the evaluation results of the voice service poor cell. SUMMARY

[0012] The technical problem to be solved by the present application is to solve the above-mentioned deficiencies of the prior art, and a voice service poor cell evaluation method and device are provided, which can collect data for a long time all day, all period and all region, reasonably evaluate the MOS quality of the VoNR voice service under the 5G network, and comprehensively, truly and objectively obtain the evaluation results of the voice service poor cell.

[0013] In a first aspect, the present application provides a voice service poor cell evaluation method applied to the network side of a 5G communication system, and the method comprises the following steps:

[0014] S1: a voice MOS fitting model is constructed, the voice MOS fitting model being constructed according to RSRP historical data, SINR historical data, downlink PRB utilization rate historical data of a main service cell and voice MOS historical data;

[0015] S2: MR data generated when a 5G user initiates a VoNR voice service in a 5G cell in a target area in a statistical period is obtained, the MR data including SS-RSRP data, SS-SINR data and downlink PRB utilization rate of a main service cell;

[0016] S3: the SS-RSRP data, the SS-SINR data and the downlink PRB utilization rate of the main service cell in the MR data are substituted into the voice MOS fitting model to obtain MOS prediction values of all cell users;

[0017] S4: based on the MOS prediction values of all cell users, a VoNR voice quality poor 5G cell in the target area in the statistical period is determined;

[0018] S5: the VoNR voice quality poor 5G cell in the target area in the statistical period is sorted.

[0019] Further, in the step S1, the voice MOS fitting model is constructed, specifically including:

[0020] A1: a large amount of voice service road test data is collected, each road test data including a time stamp, an SS-RSRP value, an SS-SINR value, downlink PRB utilization rate of a main service cell and a MOS value;

[0021] A2: a grid of road test data is built based on the time stamp;

[0022] A3: after the MOS value is grid resampled, the SS-RSRP value, the SS-SINR value, the downlink PRB utilization rate of the main service cell and the MOS value are associated and corresponded to the grid to obtain voice service road test grid data;

[0023] A4: a voice MOS fitting model is constructed according to the voice service road test grid data.

[0024] Further, in the step A4, the voice MOS fitting model is constructed according to the voice service road test grid data, specifically including:

[0025] According to the voice service road test grid data, a multiple nonlinear regression is used for fitting to determine a MOS first fitting model, and a corresponding relationship between MOS values, SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a main service cell is obtained according to the MOS first fitting model, the MOS first fitting model being:

[0026] MOS MNR = f(RSRP MNR , SINR MNR , downlink PRB utilization rate MNR )

[0027] In the formula, MOS MNR is an MOS value in multiple nonlinear regression fitting, RSRP MNR is an SS-RSRP value in multiple nonlinear regression fitting, SINR MNR is an SS-SINR value in multiple nonlinear regression fitting, and downlink PRB utilization rate MNR is a downlink PRB utilization rate of a main service cell in multiple nonlinear regression fitting.

[0028] Or,

[0029] According to the voice service road test grid data, an LSTM time series prediction is used for fitting to determine a MOS second fitting model, and a corresponding relationship between MOS values, SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a main service cell is obtained according to the MOS second fitting model, the MOS second fitting model being:

[0030] MOS LSTM = f(RSRP LSTM , SINR LSTM , downlink PRB utilization rate LSTM )

[0031] In the formula, MOS LSTM is an MOS value in LSTM time series prediction, RSRP LSTM is an SS-RSRP value in LSTM time series prediction, SINR LSTM is an SS-SINR value in LSTM time series prediction, and downlink PRB utilization rate LSTM is a downlink PRB utilization rate of a main service cell in LSTM time series prediction.

[0032] Or,

[0033] According to the voice service road test grid data, a MOS third fitting model is determined by using an artificial neural training algorithm for fitting, and a corresponding relationship between MOS values and SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a serving cell is obtained according to the MOS third fitting model, the MOS third fitting model being:

[0034] MOS AI = f(RSRP AI , SINR AI , downlink PRB utilization rate AI )

[0035] In the formula, MOS AI is an MOS value in the artificial neural training algorithm, RSRP AI is an SS-RSRP value in the artificial neural training algorithm, SINR AI is an SS-SINR value in the artificial neural training algorithm, and downlink PRB utilization rate AI is a downlink PRB utilization rate of a serving cell in the artificial neural training algorithm.

[0036] A voice MOS fitting model is obtained according to the MOS first fitting model, the MOS second fitting model, or the MOS third fitting model.

[0037] Further, the MOS first fitting model is determined by using multivariate nonlinear regression for fitting, specifically including the following steps:

[0038] B1.1: A MOS first fitting model is obtained by using multivariate nonlinear regression for fitting;

[0039] B1.2: A first regression evaluation is performed on the MOS first fitting model by using a first regression evaluation index to obtain an accuracy of the MOS first fitting model;

[0040] B1.3: The accuracy of the MOS first fitting model is compared with a first accuracy: if the accuracy of the MOS first fitting model is greater than or equal to the first accuracy, the MOS first fitting model is determined; if the accuracy of the MOS first fitting model is less than the first accuracy, steps B1.1 to B1.3 are repeated to re-fit and compare the accuracy until the MOS first fitting model is determined.

[0041] The MOS second fitting model is determined by using LSTM time series prediction for fitting, specifically including the following steps:

[0042] B2.1: A MOS second fitting model is obtained by using LSTM time series prediction for fitting;

[0043] B2.2: performing a second regression evaluation on the MOS second fitting model by a second regression evaluation index to obtain a precision of the MOS second fitting model;

[0044] B2.3: comparing the precision of the MOS second fitting model with the second precision: if the precision of the MOS second fitting model is greater than or equal to the second precision, the MOS second fitting model is determined; if the precision of the MOS second fitting model is less than the second precision, steps B2.1 to B2.3 are repeated to re-fit and compare the precision until the MOS second fitting model is determined;

[0045] The MOS third fitting model is determined by fitting using the artificial neural training algorithm, specifically including the following steps:

[0046] B3.1: obtaining the MOS third fitting model by fitting using the artificial neural training algorithm;

[0047] B3.2: performing a third regression evaluation on the MOS third fitting model by a third regression evaluation index to obtain a precision of the MOS third fitting model;

[0048] B3.3: comparing the precision of the MOS third fitting model with the third precision: if the precision of the MOS third fitting model is greater than or equal to the third precision, the MOS third fitting model is determined; if the precision of the MOS third fitting model is less than the third precision, steps B3.1 to B3.3 are repeated to re-fit and compare the precision until the MOS third fitting model is determined;

[0049] Wherein: the first precision, the second precision, and the third precision are all preset values; the first regression evaluation index, the second regression evaluation index, and the third regression evaluation index all include mean square error MSE, root mean square error RMSE, mean absolute error MAE, mean absolute percentage error MAPE, and determination coefficient R-Squared.

[0050] Further, in step S2, the MR data generated when the 5G user initiates the VoNR voice service in the 5G cell in the target area in the statistical period of the statistical period is obtained, specifically including the following steps:

[0051] S2.1: obtaining the voice bill generated by the 5G voice service active user in the target area in the statistical period;

[0052] S2.2: based on the start time and the end time in the voice bill, all SADPI bills generated by the same user are screened out, and the VoNR voice bill is determined from the SA DPI bill;

[0053] S2.3: Screen out the MR data reported by the 5G voice service active user to the resident main service cell during the VoNR voice service, wherein the MR data includes SS-RSRP data, SS-SINR data, and downlink PRB utilization rate of the main service cell.

[0054] Further, in the step S4, based on the MOS prediction value of the voice service of the cell user, the VoNR voice quality poor 5G cell in the target area in the statistical period of the statistical cycle is determined, specifically including:

[0055] Obtain the sum of the VoNR voice QoS Flow number and the E-RAB connection number in the target cell network management index in the target area in the statistical period of the statistical cycle, and obtain a plurality of MOS prediction values of the target cell;

[0056] When the sum of the VoNR voice QoS Flow number and the E-RAB connection number in the network management index of the target cell in the statistical cycle is greater than the set threshold value, and when the proportion of the VoNR voice MOS value sampling point greater than 3.0 generated in the target cell in the statistical cycle is less than 80%, the target cell is determined as a VoNR voice quality poor 5G cell; wherein the set threshold value is a value pre-set according to the statistical cycle.

[0057] Further, in the step S5, the VoNR voice quality poor 5G cells in the target area in the statistical period of the statistical cycle are sorted, specifically including the following steps:

[0058] S5.1: Obtain the VoNR voice quality poor user from the VoNR voice quality poor 5G cell in the target area in the statistical period of the statistical cycle;

[0059] S5.2: Statistically analyze the 5G cell in which the VoNR voice quality poor user resides when processing the VoNR voice service, and sort the 5G cell according to the proportion of the VoNR voice MOS value quality poor sampling point from high to low, to obtain a list of 5G cells in which the VoNR voice quality poor user commonly resides;

[0060] The method further includes:

[0061] S6: Perform VoNR voice service optimization on the VoNR voice quality poor 5G cell according to the priority.

[0062] In a second aspect, the present application provides an evaluation device for voice service quality poor cell, which is applied to the network side of the 5G communication system, and the device includes:

[0063] The acquisition unit is configured to acquire MR data generated when a 5G user initiates a VoNR voice service in a 5G cell in a target area in a statistical period of a statistical period, the MR data including SS-RSRP data, SS-SINR data, and downlink PRB utilization of a main service cell, and is further configured to acquire RSRP historical data, SINR historical data, downlink PRB utilization historical data of the main service cell, and voice MOS historical data;

[0064] The construction unit is connected with the acquisition unit and is configured to construct a voice MOS fitting model according to the RSRP historical data, the SINR historical data, the downlink PRB utilization historical data of the main service cell, and the voice MOS historical data.

[0065] The calculation unit is connected with the acquisition unit and the construction unit and is configured to substitute the SS-RSRP data, the SS-SINR data, and the downlink PRB utilization of the main service cell in the MR data into the voice MOS fitting model to obtain a MOS prediction value of a cell user voice.

[0066] The judgment unit is connected with the calculation unit and is configured to determine a VoNR voice quality difference 5G cell in the target area in the statistical period of the statistical period based on the MOS prediction value of the cell user voice.

[0067] The sorting unit is connected with the judgment unit and is configured to sort the VoNR voice quality difference 5G cell in the target area in the statistical period of the statistical period.

[0068] Further, the construction unit includes:

[0069] The collection module is connected with the acquisition unit and is configured to collect a large amount of voice service road test data, each road test data including a timestamp, an SS-RSRP value, an SS-SINR value, downlink PRB utilization of a main service cell, and a MOS value.

[0070] The first building module is connected with the collection module and is configured to build a grid of road test data based on the timestamp.

[0071] The second building module is connected with the collection module and the first building module and is configured to associate and correspond the SS-RSRP value, the SS-SINR value, the downlink PRB utilization of the main service cell, and the MOS value to the grid after grid resampling of the MOS value to obtain voice service road test grid data.

[0072] The fitting module is connected with the second building module and is configured to construct a voice MOS fitting model according to the voice service road test grid data.

[0073] Further, a multiple nonlinear regression fitting submodule is connected with the second building module, configured to determine a MOS first fitting model by fitting the voice service road test grid data by multiple nonlinear regression, and obtain a corresponding relationship between the MOS value, the SS-RSRP value, the SS-SINR value, and the downlink PRB utilization rate of the host cell according to the MOS first fitting model, wherein the MOS first fitting model is:

[0074] MOS MNR = f(RSRP MNR , SINR MNR , downlink PRB utilization rate MNR )

[0075] wherein, MOS MNR is the MOS value in the multiple nonlinear regression fitting, RSRP MNR is the SS-RSRP value in the multiple nonlinear regression fitting, SINR MNR is the SS-SINR value in the multiple nonlinear regression fitting, and downlink PRB utilization rate MNR is the downlink PRB utilization rate of the host cell in the multiple nonlinear regression fitting.

[0076] An LSTM time series prediction fitting submodule is connected with the second building module, configured to determine a MOS second fitting model by fitting the voice service road test grid data by LSTM time series prediction, and obtain a corresponding relationship between the MOS value, the SS-RSRP value, the SS-SINR value, and the downlink PRB utilization rate of the host cell according to the MOS second fitting model, wherein the MOS second fitting model is:

[0077] MOS LSTM = f(RSRP LSTM , SINR LSTM , downlink PRB utilization rate LSTM )

[0078] wherein, MOS LSTM is the MOS value in the LSTM time series prediction, RSRP LSTM is the SS-RSRP value in the LSTM time series prediction, SINR LSTM is the SINR value in the LSTM time series prediction, and downlink PRB utilization rate LSTM is the downlink PRB utilization rate of the host cell in the LSTM time series prediction.

[0079] The artificial neural network training algorithm fitting submodule, connected to the second construction module, is used to determine the third fitting model of MOS based on voice service drive test grid data using an artificial neural network training algorithm. The third fitting model of MOS is then used to obtain the correspondence between MOS values ​​and SS-RSRP values, SS-SINR values, and the downlink PRB utilization rate of the primary serving cell. The third fitting model of MOS is as follows:

[0080] MOS AI =f(RSRP) AI SINR AI Downlink PRB utilization rate AI )

[0081] Where: MOS AI RSRP is the MOS value in the artificial neural network training algorithm. AI SINR is the SS-RSRP value in the artificial neural network training algorithm. AI The SS-SINR value in the artificial neural network training algorithm represents the downlink PRB utilization rate. AI This represents the downlink PRB utilization rate of the primary serving cell in the artificial neural network training algorithm.

[0082] The beneficial effects achieved by this invention are:

[0083] (1) For the same voice service, the VoLTE voice service MOS value varies significantly with the change of wireless environment under different frequency points, different bandwidths and different equipment forms. This invention realizes a voice MOS prediction and fitting model based on RSRP, SINR and cell load data that distinguishes different frequency points, different bandwidths and different equipment forms, so as to realize accurate evaluation of voice services.

[0084] (2) Traditional voice service evaluation uses the MOS box DT method, which requires a lot of manpower and resources to conduct voice service testing, and cannot achieve a comprehensive, reasonable and efficient service evaluation; the MOS value of voice services also varies under different network loads. Therefore, in order to reasonably evaluate the MOS quality of VoNR voice services under 5G networks, this invention innovatively designs a voice quality evaluation method based on wireless environment and base station load;

[0085] (3) In the prior art, since the MR data cannot distinguish the service type of the user, further judgment needs to be made with the aid of the XDR data, at the same time, the IMS bill cannot distinguish the VoLTE / VoNR voice service temporarily, and needs to be judged according to the access type of the 5G user initiating the IMS voice service, the service start and end time information, and the SA DPI bill of the user in the same time period. The application innovatively designs a method and device for comprehensively judging the poor quality 5G cell of VoNR voice service, and realizes accurate positioning of the cell for VoNR voice service problems.

[0086] (4) Since the list of screened VoNR voice service poor quality 5G cells may contain many 5G cells, the optimization workload is large, the application further invents a method and device for screening the 5G cell where the VoNR voice service poor quality user resides, sorts the 5G cells to be optimized according to the priority, focuses on the 5G cell where the voice service is highly likely to deteriorate, and improves the efficiency of optimization and operation and maintenance work. BRIEF DESCRIPTION OF DRAWINGS

[0087] Figure 1 The evaluation process of the voice service poor quality cell in the embodiment of the application;

[0088] Figure 2 The MOS test schematic diagram in the embodiment of the application;

[0089] Figure 3 The evaluation device of the voice service poor quality cell in the embodiment of the application;

[0090] Figure 4 The construction unit in the evaluation device of the voice service poor quality cell in the embodiment of the application.

[0091] Wherein: 10, acquisition unit, 20, construction unit, 21, collection module, 22, first building module, 23, second building module, 24, fitting module, 30, calculation unit, 40, judgment unit, 50, sorting unit. DETAILED DESCRIPTION

[0092] In order for those skilled in the art to better understand the technical solutions of the application, the embodiments of the application will be further described in detail below with reference to the drawings.

[0093] It can be understood that the specific embodiments and drawings described herein are only used to explain the application, and not to limit the application.

[0094] It can be understood that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0095] It can be understood that, for the convenience of description, only the parts related to the present application are shown in the drawings of the present application, and the parts irrelevant to the present application are not shown in the drawings.

[0096] It can be understood that each unit and module involved in the embodiments of the present application can correspond to only one entity structure, or can be composed of multiple entity structures, or multiple units and modules can be integrated into one entity structure.

[0097] It can be understood that, in the case of no conflict, the functions and steps marked in the flowcharts and block diagrams of the present application can occur in an order different from that marked in the drawings.

[0098] It can be understood that, in the flowcharts and block diagrams of the present application, the architecture, functions and operations of the possible implementations of the system, device, equipment and method according to the embodiments of the present application are shown. Each block in the flowchart or block diagram can represent a unit, module, program segment, code, which contains executable instructions for realizing the specified functions. Moreover, each block or combination of blocks in the block diagram and flowchart can be realized by a hardware-based system for realizing the specified functions, or by a combination of hardware and computer instructions.

[0099] It can be understood that the units and modules involved in the embodiments of the present application can be realized in the form of software or hardware, for example, the units and modules can be located in a processor.

[0100] Embodiment 1:

[0101] As shown in Figure 1 and Figure 2 , the present embodiment provides a method for evaluating poor voice service cell, mainly aiming at the voice perception of VoNR voice service under 5G network, based on the MR data and XDR data reported by voice service users, the evaluation and positioning of the 5G cell with poor voice perception and the aggregation of users with poor voice perception are realized, which provides key basis and technical support for voice service optimization. The specific implementation scheme is as follows:

[0102] Step 1. Establishing MOS voice prediction and fitting model for VoNR voice service

[0103] Step 1-1. MOS raw data collection: Collect a large number of VoNR voice service road test data containing time stamp, SS-RSRP, SS-SINR and MOS value through voice MOS box collection under a well-optimized and continuously covered 5G network of different frequency bands (such as 900M / 2.1G / 3.5G NR), different bandwidths (900M NR 5M / 10M, 2.1G NR 20M / 40M, 3.5G NR 100M), and different device types (900M NR 4T4R, 2.1G 4T4R / 2T2R), and record the downlink PRB utilization rate of the host cell in real time.

[0104] Step 1-2. Data cleaning and processing: Since MOS value is generated every 8-9 seconds, and SS-RSRP and SS-SINR are generated at different periods according to the capabilities of the chip and MOS device, generally 200ms or less, which leads to MOS and SS-RSRP / SS-SINR values not being completely synchronized. Therefore, the RSRP / SINR and MOS values in the table need to be processed according to time. First, all sampling points in the table are processed according to a 1s time grid granularity, and each SS-RSRP, SS-SINR and MOS value is collected to the nearest 1 whole second. Secondly, since MOS value is generated every 8-9 seconds, there will be a large number of null values in the 1s grid data table. The generation of MOS value is closely related to the SS-RSRP / SS-SINR value of the previous 8-9 seconds, at which time the MOS value can be backfilled until the previous MOS value. At the same time, the cell information in the table is associated, and the downlink PRB utilization rate of the host cell is backfilled to the table. At this time, a data table containing SS-RSRP / SS-SINR / MOS value / downlink PRB utilization rate of the host cell can be obtained in order (as shown in the following table, the following table is a sample table).

[0105]

[0106]

[0107] Step 1-3. Voice MOS fitting model: Based on the data table obtained in step 1-2, a voice MOS fitting model can be established, and the specific method can use multivariate nonlinear regression fitting method, use LSTM-based time series prediction method, or use artificial neural training algorithm (such as RNN, DNN, etc.) to obtain the following formula.

[0108]

[0109] Where c is a constant, and a unique MOS value can be obtained whenever a set of SS-RSRP, SS-SINR and downlink PRB utilization of the primary serving cell is given.

[0110] The accuracy of the prediction model can be calculated using regression evaluation metrics such as MSE, RMSE, MAE, MAPE, and R2. When the model's accuracy meets the expected requirements, it can be considered that the MOS prediction model can obtain the corresponding MOS value based on SS-RSRP, SS-SINR, and the downlink PRB utilization rate of the primary serving cell, i.e., the expected MOS value under different wireless environments and network loads. It should be noted that, as mentioned earlier, the MOS prediction model differs for different frequency bands, bandwidths, and device types. Therefore, steps 1-1 to 1-3 need to be repeated separately to establish the voice MOS fitting model. Finally, voice MOS fitting models for different frequency bands, bandwidths, and device types can be obtained.

[0111] Specifically, constructing the speech MOS fitting model includes:

[0112] A1: Collect a large amount of voice service drive test data. Each drive test data includes timestamp, SS-RSRP value, SS-SINR value, downlink PRB utilization rate of the primary serving cell, and MOS value.

[0113] A2: Based on timestamps, construct a grid for the road test data;

[0114] A3: After grid resampling the MOS value, the SS-RSRP value, SS-SINR value, downlink PRB utilization rate of the primary serving cell and MOS value are associated with the grid to obtain the voice service drive test grid data;

[0115] A4: Based on the voice service drive test grid data, a voice MOS fitting model is constructed.

[0116] Specifically, constructing the speech MOS fitting model includes:

[0117] Based on voice service drive test grid data, a first fitting model for MOS was determined using multivariate nonlinear regression. The first fitting model for MOS then yielded the corresponding relationships between MOS values ​​and SS-RSRP, SS-SINR, and downlink PRB utilization of the primary serving cell. The first fitting model for MOS is as follows:

[0118] MOS MNR =f(RSRP) MNR SINR MNR Downlink PRB utilization rate MNR )

[0119] Where: MOSMNR RSRP is the MOS value in the multivariate nonlinear regression fitting. MNR SINR represents the SS-RSRP value in multivariate nonlinear regression fitting. MNR The SS-SINR value in the multivariate nonlinear regression fitting represents the downlink PRB utilization rate. MNR The downlink PRB utilization rate of the primary serving cell in the multivariate nonlinear regression fitting;

[0120] or,

[0121] Based on voice service drive test grid data, LSTM time-series prediction was used to fit and determine the second fitting model for MOS. The correlation between MOS value and SS-RSRP value, SS-SINR value, and downlink PRB utilization rate of the primary serving cell was obtained from the second fitting model. The second fitting model for MOS is as follows:

[0122] MOS LSTM =f(RSRP) LSTM SINR LSTM Downlink PRB utilization rate LSTM )

[0123] Where: MOS LSTM RSRP represents the MOS value in LSTM timing prediction. LSTM SINR represents the SS-RSRP value in LSTM time series prediction. LSTM SINR value in LSTM time series prediction; downlink PRB utilization rate LSTM The downlink PRB utilization rate of the primary serving cell in LSTM time-series prediction;

[0124] or,

[0125] Based on voice service drive test grid data, an artificial neural network training algorithm was used to fit and determine the third fitting model of MOS. The corresponding relationships between MOS values ​​and SS-RSRP, SS-SINR, and downlink PRB utilization of the primary serving cell were obtained from the third fitting model. The third fitting model of MOS is as follows:

[0126] MOS AI =f(RSRP) AI SINR AI Downlink PRB utilization rate AI )

[0127] Where: MOS AI RSRP is the MOS value in the artificial neural network training algorithm. AI SINR is the SS-RSRP value in the artificial neural network training algorithm. AIFor the SS-SINR value in the artificial neural training algorithm, the downlink PRB utilization rate AI For the downlink PRB utilization rate of the primary serving cell in the artificial neural training algorithm

[0128] According to the constructed MOS first fitting model, MOS second fitting model or MOS third fitting model, a voice MOS fitting model is obtained.

[0129] Specifically, a bias constant can be added after the MOS first fitting model, MOS second fitting model or MOS third fitting model, that is,

[0130] MOS MNR =f(RSRP MNR , SINR MNR , downlink PRB utilization rate MNR )+C MNR ;

[0131] MOS LSTM =f(RSRP LSTM , SINR LSTM , downlink PRB utilization rate LSTM )+C LSTM ;

[0132] MOS AI =f(RSRP AI , SINR AI , downlink PRB utilization rate AI )+C AI ;

[0133] C MNR is a bias constant of multivariate nonlinear regression fitting, the value range of C MNR is -0.5-0.5; C LSTM is a bias constant of LSTM time series prediction, the value range of C LSTM is -0.5-0.5; C AI is a bias constant of artificial neural training algorithm, the value range of C AI is -0.5-0.5.

[0134] Specifically, a multivariate nonlinear regression is used for fitting to determine the MOS first fitting model, and the specific steps include the following:

[0135] B1.1: A multivariate nonlinear regression is used for fitting to obtain the MOS first fitting model;

[0136] B1.2: The MOS first fitting model is evaluated by a first regression evaluation index to obtain the accuracy of the MOS first fitting model;

[0137] B1.3: comparing the accuracy of the MOS first fitting model with the first accuracy: if the accuracy of the MOS first fitting model is greater than or equal to the first accuracy, the MOS first fitting model is determined; if the accuracy of the MOS first fitting model is less than the first accuracy, steps B1.1 to B1.3 are repeated to re-fit and compare the accuracy until the MOS first fitting model is determined;

[0138] The MOS second fitting model is determined by fitting using the LSTM time series prediction, specifically including the following steps:

[0139] B2.1: the MOS second fitting model is obtained by fitting using the LSTM time series prediction;

[0140] B2.2: the MOS second fitting model is evaluated by the second regression evaluation index to obtain the accuracy of the MOS second fitting model;

[0141] B2.3: comparing the accuracy of the MOS second fitting model with the second accuracy: if the accuracy of the MOS second fitting model is greater than or equal to the second accuracy, the MOS second fitting model is determined; if the accuracy of the MOS second fitting model is less than the second accuracy, steps B2.1 to B2.3 are repeated to re-fit and compare the accuracy until the MOS second fitting model is determined;

[0142] The MOS third fitting model is determined by fitting using the artificial neural training algorithm, specifically including the following steps:

[0143] B3.1: the MOS third fitting model is obtained by fitting using the artificial neural training algorithm;

[0144] B3.2: the MOS third fitting model is evaluated by the third regression evaluation index to obtain the accuracy of the MOS third fitting model;

[0145] B3.3: comparing the accuracy of the MOS third fitting model with the third accuracy: if the accuracy of the MOS third fitting model is greater than or equal to the third accuracy, the MOS third fitting model is determined; if the accuracy of the MOS third fitting model is less than the third accuracy, steps B3.1 to B3.3 are repeated to re-fit and compare the accuracy until the MOS third fitting model is determined;

[0146] Wherein: the first accuracy, the second accuracy, and the third accuracy are all preset values; the first regression evaluation index, the second regression evaluation index, and the third regression evaluation index all include mean square error MSE, root mean square error RMSE, mean absolute error MAE, mean absolute percentage error MAPE, and determination coefficient R-Squared.

[0147] Step 2. Screen out the MR data generated by 5G users initiating VoNR voice services in 5G cells in the statistical period. Since MR data cannot distinguish the service types of users, XDR data is needed for further judgment. At the same time, VoLTE / VoNR voice services cannot be distinguished in IMS bills, and need to be judged according to the access type of 5G users initiating IMS voice services, the service start and end time information, and the SA DPI bill of the user in the same time period. The judgment method is as follows:

[0148] Step 2-1. Screen out all IMS bills of 5G users in the statistical period, whose service type (Service Type field) is "1: Voice call" and access type (SOURCE_ACCESS_TYPE field) is identified as "56: 3GPP-NR-TDD", "57: 3GPP-NR-FDD" or "58: 3GPP-NR". Further screen out the data target area 5G cell bills in the access cell NCGI of the bill according to the work parameters, and these bills are recorded as the voice bills generated by 5G users in the target area in the statistical period, and these 5G users are recorded as VoNR voice service active users in the target area in the statistical period;

[0149] Step 2-2. Further screen all IMS bills screened out in step 2-1, and calculate the call time period of the IMS voice service based on the start time (Procedure Start Time field) and the end time (Procedure End Time) in the IMS bill. Screen out all SA DPI bills generated by the same user in the time period. If there is a handover type (Handover_Type field) of "5GStoEPS" in the SA DPI N1 / N2 interface bill of the user and the procedure start time (procedure_start time field) is within 5s after the start time of the IMS bill, it is determined that the IMS bill is an EPS FB VoLTE voice bill, otherwise it is determined that the IMS bill is a VoNR voice bill;

[0150] Step 2-3. Based on the start time (START_T field) and end time (END_T field) in the SA DPI call record and the start and end time of the IMS voice call record determined as VoNR service in step 2-2, the time period during which the user initiates VoNR voice service and the corresponding host cell on the 5G cell are jointly determined. The MR data reported by the user to the host cell during the VoNR voice service period is screened out, and the MR data contains the SS-RSRP / SS-SINR information of the host cell. At the same time, the downlink PRB utilization rate of the host cell in the period is collected as the load characteristics of the cell. According to the voice MOS fitting model in step 1, the VoNR voice MOS value generated by the user per second in the host cell can be obtained.

[0151] Step 3. Collect all MOS value information and associated cell information generated by VoNR voice active users in the target area in the statistical period of the statistical period. VoNR voice quality difference 5G cells in the area can be further determined, and the determination conditions are as follows:

[0152] Condition 1: In the statistical period, the sum of the number of VoNR voice QoS flows and the number of E-RAB connections (5QI=1) in the network management index of the cell is greater than a set threshold 1 (adjusted according to the statistical period, if the statistical period is one week, the threshold is recommended to be set to 100);

[0153] Condition 2: The proportion of VoNR voice MOS value sampling points greater than 3.0 generated in the cell in the statistical period is less than 80%;

[0154] At the same time, the above two conditions are met, that is, the cell is determined as a VoNR voice quality difference 5G cell;

[0155] Step 4. Further, analyze each VoNR voice active user in the target area in the statistical period of the statistical period. If the number of VoNR voice service users in the statistical period in the area is greater than threshold 2 (adjusted according to the statistical period, if the statistical period is one week, the threshold is recommended to be set to 10), and the proportion of MOS value > 3.0 sampling points generated by the user is less than 80%, the 5G user is recorded as a VoNR voice quality difference user; the 5G cell and the occupation time of the VoNR voice quality difference user during the VoNR voice service are accumulated and counted, and the VoNR voice quality difference user commonly resides in the 5G cell list is sorted according to the VoNR voice MOS value quality difference sampling point proportion from high to low.

[0156] Step 5. Based on the VoNR voice quality poor user resident 5G cell list obtained in step 4, the VoNR voice quality poor 5G cell obtained in step 3 can be prioritized for VoNR voice service special optimization, accurately positioning the 5G cell with poor voice service perception, and improving the voice service perception and network performance of VoNR voice service.

[0157] Based on the above steps, the present patent realizes a voice MOS prediction and fitting model based on RSRP, SINR and cell load data for different frequency points, different bandwidths and different device forms. At the same time, based on the SA DPI call record, IMS call record and MR data, the VoNR voice service perception evaluation of the target area VoNR voice service user is realized, based on the fitted voice MOS value, the VoNR voice quality poor 5G cell is further determined, and the accurate positioning of the VoNR voice service problem cell is realized. At the same time, the present patent realizes a determination method based on the VoNR voice quality poor user resident 5G cell, which sorts the 5G cells to be optimized according to the priority, focuses on the 5G cells with high probability of voice service deterioration, and improves the efficiency of optimization and operation and maintenance work.

[0158] Embodiment 2

[0159] As shown in Figure 1 and Figure 3 The present embodiment provides a voice service quality poor cell evaluation device, applied to the network side of a 5G communication system, which comprises:

[0160] The acquisition unit is configured to acquire MR data generated when 5G users initiate VoNR voice service in a target area in a statistical period of a statistical period, the MR data comprising SS-RSRP data, SS-SINR data, and downlink PRB utilization rate of a host cell, and to acquire RSRP historical data, SINR historical data, downlink PRB utilization rate historical data of the host cell, and voice MOS historical data;

[0161] The construction unit is connected with the acquisition unit and is configured to construct a voice MOS fitting model based on the RSRP historical data, the SINR historical data, the downlink PRB utilization rate historical data of the host cell, and the voice MOS historical data;

[0162] The calculation unit is connected with the acquisition unit and the construction unit, and is configured to substitute the SS-RSRP data, the SS-SINR data, and the downlink PRB utilization rate of the host cell in the MR data into the voice MOS fitting model to obtain MOS prediction values of all users in the cell;

[0163] The judging unit is connected with the calculating unit and is used for judging the VoNR voice quality difference 5G cell under the target area in the statistical period based on the MOS prediction value of the cell user voice quality.

[0164] The sorting unit is connected with the judging unit and is used for sorting the VoNR voice quality difference 5G cell under the target area in the statistical period.

[0165] Specifically, as shown in the figure, Figure 4 The constructing unit includes:

[0166] The collecting module is connected with the acquiring unit and is used for collecting a large amount of voice service road test data, each road test data including a timestamp, an SS-RSRP value, an SS-SINR value, a downlink PRB utilization rate of a main service cell and a MOS value.

[0167] The first building module is connected with the collecting module and is used for building a grid of the road test data based on the timestamp.

[0168] The second building module is connected with the collecting module and the first building module respectively and is used for, after grid resampling of the MOS value, associating and corresponding the SS-RSRP value, the SS-SINR value, the downlink PRB utilization rate of the main service cell and the MOS value to the grid to obtain voice service road test grid data.

[0169] The fitting module is connected with the second building module and is used for constructing a voice MOS fitting model according to the voice service road test grid data.

[0170] Further, the multiple nonlinear regression fitting submodule is connected with the second building module and is used for fitting and determining a MOS first fitting model by using multiple nonlinear regression according to the voice service road test grid data, and obtaining the corresponding relationship between the MOS value and the SS-RSRP value, the SS-SINR value and the downlink PRB utilization rate of the main service cell according to the MOS first fitting model, the MOS first fitting model being:

[0171] MOS MNR = f(RSRP MNR , SINR MNR , downlink PRB utilization rate MNR )

[0172] In the formula, MOS MNR is the MOS value in the multiple nonlinear regression fitting, RSRP MNR is the SS-RSRP value in the multiple nonlinear regression fitting, SINR MNR is the SS-SINR value in the multiple nonlinear regression fitting, and downlink PRB utilization rate MNRThe downlink PRB utilization rate of the primary serving cell in the multivariate nonlinear regression fitting;

[0173] The LSTM timing prediction fitting submodule, connected to the second construction module, is used to determine the second MOS fitting model based on voice service drive test grid data using LSTM timing prediction. The second MOS fitting model is then used to obtain the correspondence between MOS values ​​and SS-RSRP, SS-SINR, and the downlink PRB utilization rate of the primary serving cell. The second MOS fitting model is as follows:

[0174] MOS LSTM =f(RSRP) LSTM SINR LSTM Downlink PRB utilization rate LSTM )

[0175] Where: MOS LSTM RSRP represents the MOS value in LSTM timing prediction. LSTM SINR represents the SS-RSRP value in LSTM time series prediction. LSTM SINR value in LSTM time series prediction; downlink PRB utilization rate LSTM The downlink PRB utilization rate of the primary serving cell in LSTM time-series prediction;

[0176] The artificial neural network training algorithm fitting submodule, connected to the second construction module, is used to determine the third fitting model of MOS based on voice service drive test grid data using an artificial neural network training algorithm. The third fitting model of MOS is then used to obtain the correspondence between MOS values ​​and SS-RSRP values, SS-SINR values, and the downlink PRB utilization rate of the primary serving cell. The third fitting model of MOS is as follows:

[0177] MOS AI =f(RSRP) AI SINR AI Downlink PRB utilization rate AI )

[0178] Where: MOS AI RSRP is the MOS value in the artificial neural network training algorithm. AI SINR is the SS-RSRP value in the artificial neural network training algorithm. AI The SS-SINR value in the artificial neural network training algorithm represents the downlink PRB utilization rate. AI This represents the downlink PRB utilization rate of the primary serving cell in the artificial neural network training algorithm.

[0179] In Examples 1 and 2:

[0180] (1) For the same voice service, the MOS value of VoLTE voice service is obviously different with the change of wireless environment under different frequency points, different bandwidths and different device forms, and the application realizes a voice MOS prediction and fitting model based on RSRP, SINR and cell load data for different frequency points, different bandwidths and different device forms, and realizes accurate evaluation of voice service;

[0181] (2) The traditional voice service evaluation adopts the MOS box DT method, which needs a large amount of manpower and material resources to invest in voice service testing, and cannot achieve comprehensive, reasonable and efficient service evaluation. The MOS value of voice service also has differences under different network loads. Therefore, in order to reasonably evaluate the MOS quality of VoNR voice service under 5G network, the application innovatively designs a voice quality evaluation method based on wireless environment and base station load;

[0182] (3) In the prior art, since the MR data cannot distinguish the service type of the user, it is necessary to further judge with the aid of XDR data, and at the same time, the VoLTE / VoNR voice service cannot be distinguished in the IMS bill, and it is necessary to judge according to the access type of the 5G user initiating the IMS voice service, the service start and end time information, and the SA DPI bill of the user in the same time period. The application innovatively designs a method and device for comprehensively judging VoNR voice service quality difference 5G cell based on multiple data sources, and realizes accurate positioning of VoNR voice service problem cell.

[0183] (4) Since the screened VoNR voice service quality difference 5G cell list may contain many 5G cells, the optimization workload is large, and the application further invents a method and device for screening the 5G cell where the VoNR voice service quality difference user resides, sorts the 5G cells to be optimized according to priority, focuses on the 5G cell with high probability of voice service deterioration, and improves the efficiency of optimization and operation and maintenance work.

[0184] It can be understood that the above embodiments are only exemplary embodiments for illustrating the principles of the application, but the application is not limited thereto. Those skilled in the art can make various modifications and improvements without departing from the spirit and essence of the application, and these modifications and improvements are also regarded as the protection scope of the application.

Claims

1. A method for evaluating cells with poor voice service quality, applied to the network side of a 5G communication system, characterized in that, The method comprises the steps of: S1: constructing a voice MOS fitting model, wherein the voice MOS fitting model is constructed according to RSRP historical data, SINR historical data, downlink PRB utilization rate historical data and voice MOS historical data; The voice MOS fitting model is constructed, and specifically comprises: A1: collecting a large amount of voice service road test data, wherein each road test data comprises a timestamp, an SS-RSRP value, an SS-SINR value, a downlink PRB utilization rate of a main service cell and a MOS value; A2: building a grid of road test data based on the timestamp; A3: after grid resampling the MOS value, the SS-RSRP value, the SS-SINR value, the downlink PRB utilization rate of the main service cell and the MOS value are associated and corresponded to the grid to obtain voice service road test grid data; A4: constructing a voice MOS fitting model according to the voice service road test grid data; S2: obtaining MR data generated when a 5G user initiates a VoNR voice service in a 5G cell in a target area in a statistical period of a statistical cycle, wherein the MR data comprises SS-RSRP data, SS-SINR data and a downlink PRB utilization rate of a main service cell; S3: substituting the SS-RSRP data, the SS-SINR data and the downlink PRB utilization rate of the main service cell in the MR data into the voice MOS fitting model to obtain all MOS prediction values of cell user voice; S4: determining a VoNR voice quality poor 5G cell in the target area in the statistical period of the statistical cycle based on all MOS prediction values of cell user voice; In the step S4, the VoNR voice quality poor 5G cell in the target area in the statistical period of the statistical cycle is determined based on all MOS prediction values of cell user voice, and specifically comprises: obtaining the sum of the number of VoNR voice QoS flows and the number of E-RAB connections in the target cell network management index in the target area in the statistical period of the statistical cycle, and obtaining a plurality of MOS prediction values of the target cell; when the sum of the number of VoNR voice QoS flows and the number of E-RAB connections in the network management index of the target cell in the statistical cycle is greater than a set threshold value, and when the proportion of VoNR voice MOS value sampling points greater than 3.0 generated in the target cell in the statistical cycle is less than 80%, the target cell is determined as a VoNR voice quality poor 5G cell; wherein the set threshold value is a value pre-set according to the statistical cycle; S5: sorting the VoNR voice quality poor 5G cells in the target area in the statistical period of the statistical cycle.

2. The voice service quality poor cell evaluation method according to claim 1, wherein In the step A4, the voice MOS fitting model is constructed according to the voice service road test grid data, and specifically comprises: According to the voice service road test grid data, a multiple nonlinear regression is used for fitting to determine a MOS first fitting model, and a corresponding relationship between MOS values, SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a main service cell is obtained according to the MOS first fitting model, the MOS first fitting model being: MOS MNR = f(RSRP MNR , SINR MNR , downlink PRB utilization MNR ) where: MOS MNR MOS values for the multiple non-linear regression fit, RSRP MNR SS-RSRP values for the multiple non-linear regression fit, SINR MNR SS-SINR values for the multiple non-linear regression fit, Downlink PRB Utilization MNR Downlink PRB Utilization for the primary serving cell for the multiple non-linear regression fit; or, According to the voice service road test grid data, an LSTM time sequence prediction is used for fitting to determine a MOS second fitting model, and a corresponding relationship between MOS values, SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a main service cell is obtained according to the MOS second fitting model, the MOS second fitting model being: MOS LSTM = f(RSRP LSTM , SINR LSTM , downlink PRB utilization LSTM ) wherein: MOS LSTM MOS value in LSTM time series prediction, RSRP LSTM SS-RSRP value in LSTM time series prediction, SINR LSTM SINR value in LSTM time series prediction; Downlink PRB utilization LSTM Downlink PRB utilization of the primary serving cell in LSTM time series prediction; or, According to the voice service road test grid data, an artificial neural training algorithm is used for fitting to determine a MOS third fitting model, and a corresponding relationship between MOS values, SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of a main service cell is obtained according to the MOS third fitting model, the MOS third fitting model being: MOS AI = f(RSRP AI , SINR AI , downlink PRB utilization AI ) where: MOS AI MOS value in artificial neural training algorithm, RSRP AI SS-RSRP value in artificial neural training algorithm, SINR AI SS-SINR value in artificial neural training algorithm, downlink PRB utilization AI Downlink PRB utilization of the primary serving cell in artificial neural training algorithm; A voice MOS fitting model is obtained according to the MOS first fitting model, the MOS second fitting model, or the MOS third fitting model.

3. The voice service quality poor cell evaluation method according to claim 2, characterized in that, the multiple nonlinear regression is used for fitting to determine the MOS first fitting model, and the steps are specifically as follows: B1.1: The MOS first fitting model is obtained by using the multiple nonlinear regression for fitting; B1.2: The MOS first fitting model is evaluated by using a first regression evaluation index to obtain a precision of the MOS first fitting model; B1.3: The precision of the MOS first fitting model is compared with a first precision: if the precision of the MOS first fitting model is greater than or equal to the first precision, the MOS first fitting model is determined; if the precision of the MOS first fitting model is less than the first precision, steps B1.1 to B1.3 are repeated to re-fit and compare the precision until the MOS first fitting model is determined; the LSTM time sequence prediction is used for fitting to determine the MOS second fitting model, and the steps are specifically as follows: B2.1: The MOS second fitting model is obtained by using the LSTM time sequence prediction for fitting; B2.2: The MOS second fitting model is evaluated by using a second regression evaluation index to obtain a precision of the MOS second fitting model; B2.3: The precision of the MOS second fitting model is compared with a second precision: if the precision of the MOS second fitting model is greater than or equal to the second precision, the MOS second fitting model is determined; if the precision of the MOS second fitting model is less than the second precision, steps B2.1 to B2.3 are repeated to re-fit and compare the precision until the MOS second fitting model is determined; the artificial neural training algorithm is used for fitting to determine the MOS third fitting model, and the steps are specifically as follows: B3.1: The MOS third fitting model is obtained by using the artificial neural training algorithm for fitting; B3.2: performing third regression evaluation on the MOS third fitting model by a third regression evaluation index to obtain a precision of the MOS third fitting model; B3.3: comparing the precision of the MOS third fitting model with the third precision: if the precision of the MOS third fitting model is greater than or equal to the third precision, the MOS third fitting model is determined; if the precision of the MOS third fitting model is less than the third precision, steps B3.1 to B3.3 are repeated to re-fit and compare the precision until the MOS third fitting model is determined; wherein the first precision, the second precision, and the third precision are all preset values; the first regression evaluation index, the second regression evaluation index, and the third regression evaluation index all include mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and determination coefficient R-Squared.

4. The voice service quality poor cell evaluation method according to any one of claims 1 to 3, characterized in that, in the step S2, the MR data generated when the 5G user initiates the VoNR voice service in the 5G cell in the target area in the statistical period of the statistical period is obtained, and the steps include: S2.1: obtaining the voice bill generated by the 5G voice service active user in the target area in the statistical period; S2.2: based on the start time and the end time in the voice bill, all SA DPI bills generated by the same user are screened out, and the VoNR voice bill is determined from the SA DPI bill; S2.3: screening out the MR data reported by the 5G voice service active user to the host cell during the VoNR voice service period, the MR data including SS-RSRP data, SS-SINR data, and downlink PRB utilization rate of the host cell.

5. The voice service quality poor cell evaluation method according to any one of claims 1 to 3, characterized in that, in the step S5, the VoNR voice quality poor 5G cells in the target area in the statistical period of the statistical period are sorted, and the steps include: S5.1: obtaining the VoNR voice quality poor user from the VoNR voice quality poor 5G cell in the target area in the statistical period of the statistical period; S5.2: counting the 5G cell in which the VoNR voice quality poor user resides when processing the VoNR voice service, and sorting the 5G cells in descending order of the proportion of VoNR voice MOS value quality poor sampling points to obtain a list of 5G cells in which the VoNR voice quality poor user commonly resides; the method further includes: S6: performing VoNR voice service optimization on the VoNR voice quality poor 5G cell according to the priority. 6.An apparatus for evaluating a poor-quality cell of a voice service, applied to a network side of a 5G communication system, comprising: including: The acquisition unit is configured to acquire MR data generated when a 5G user initiates a VoNR voice service in a 5G cell in a target area in a statistical period of a statistical period, the MR data including SS-RSRP data, SS-SINR data, and downlink PRB utilization of a main service cell, and to acquire RSRP historical data, SINR historical data, downlink PRB utilization historical data of the main service cell, and voice MOS historical data; The construction unit is connected with the acquisition unit and is configured to construct a voice MOS fitting model according to the RSRP historical data, the SINR historical data, the downlink PRB utilization historical data, and the voice MOS historical data; The construction unit includes: The collection module is connected with the acquisition unit and is configured to collect a large amount of voice service road test data, each road test data including a timestamp, an SS-RSRP value, an SS-SINR value, downlink PRB utilization of a main service cell, and a MOS value; The first building module is connected with the collection module and is configured to build a grid of road test data based on the timestamp; The second building module is connected with the collection module and the first building module and is configured to associate and correspond the SS-RSRP value, the SS-SINR value, the downlink PRB utilization, and the MOS value to the grid after grid resampling of the MOS value, to obtain voice service road test grid data; The fitting module is connected with the second building module and is configured to construct a voice MOS fitting model according to the voice service road test grid data; The calculation unit is connected with the acquisition unit and the construction unit and is configured to substitute the SS-RSRP data, the SS-SINR data, and the downlink PRB utilization of the main service cell in the MR data into the voice MOS fitting model to obtain all MOS prediction values of cell user voice; The judgment unit is connected with the calculation unit and is configured to determine a VoNR voice quality poor 5G cell in the target area in the statistical period based on all MOS prediction values of cell user voice; The judgment unit is connected with the calculation unit and is configured to determine a VoNR voice quality poor 5G cell in the target area in the statistical period based on all MOS prediction values of cell user voice; The judgment unit is connected with the calculation unit and is configured to determine a VoNR voice quality poor 5G cell in the target area in the statistical period based on all MOS prediction values of cell user voice; The sorting unit is connected with the judgment unit and is configured to sort the VoNR voice quality poor 5G cells in the target area in the statistical period. ​ 7.The voice service quality poor cell evaluation device of claim 6, characterized in that, a multiple nonlinear regression fitting sub-module, connected with the second building module, is configured to determine a MOS first fitting model by fitting using multiple nonlinear regression according to the voice service drive test grid data, and obtain a corresponding relationship between MOS values and SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of the hosting cell according to the MOS first fitting model, wherein the MOS first fitting model is: MOS MNR = f(RSRP MNR , SINR MNR , downlink PRB utilization MNR ) where: MOS MNR MOS values for the multiple non-linear regression fit, RSRP MNR SS-RSRP values for the multiple non-linear regression fit, SINR MNR SS-SINR values for the multiple non-linear regression fit, Downlink PRB Utilization MNR Downlink PRB Utilization for the primary serving cell for the multiple non-linear regression fit; an LSTM time series prediction fitting sub-module, connected with the second building module, is configured to determine a MOS second fitting model by fitting using LSTM time series prediction according to the voice service drive test grid data, and obtain a corresponding relationship between MOS values and SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of the hosting cell according to the MOS second fitting model, wherein the MOS second fitting model is: MOS LSTM = f(RSRP LSTM , SINR LSTM , downlink PRB utilization LSTM ) where: MOS LSTM MOS value in LSTM time series prediction, RSRP LSTM SS-RSRP value in LSTM time series prediction, SINR LSTM SINR value in LSTM time series prediction; Downlink PRB utilization LSTM Downlink PRB utilization of the primary serving cell in LSTM time series prediction; an artificial neural training algorithm fitting sub-module, connected with the second building module, is configured to determine a MOS third fitting model by fitting using an artificial neural training algorithm according to the voice service drive test grid data, and obtain a corresponding relationship between MOS values and SS-RSRP values, SS-SINR values, and downlink PRB utilization rates of the hosting cell according to the MOS third fitting model, wherein the MOS third fitting model is: MOS AI = f(RSRP AI , SINR AI , downlink PRB utilization AI ) where: MOS AI MOS value in artificial neural training algorithm, RSRP AI SS-RSRP value in artificial neural training algorithm, SINR AI SS-SINR value in artificial neural training algorithm, Downlink PRB utilization AI Downlink PRB utilization of the primary serving cell in artificial neural training algorithm.

Images