Terminal perception evaluation method, storage medium and electronic device

Abstract

The embodiment of the invention provides a terminal perception evaluation method, a storage medium and an electronic device, and the method comprises the steps: carrying out the perception evaluation of a business key index of a terminal according to the basic capability and frequency band information of the terminal; and evaluating the terminal according to the perception evaluation result of the service key index. Through the method and the device, the problems that information association is missing and association analysis with the service information cannot be carried out when terminal perception analysis is carried out in related technologies are solved, and the effects of carrying out association analysis on the terminal in combination with the service information and avoiding information association missing are achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communications, and in particular, to a terminal perception evaluation method, a storage medium, and an electronic device. Background Art

[0002] During the 5G user migration process, customers from all over the country have repeatedly reported that some of the 5G terminals that were previously connected to the network have problems with terminal and network compatibility, and there is a probability that they cannot access the Internet and the voice call quality is poor. Operator customers are in urgent need of effective terminal perception and comprehensive analysis tools. In related technologies, the basic capabilities of the terminal and the frequency band are usually associated with the terminal library maintenance table information, but there are shortcomings such as missing information association, untimely, and partially inaccurate information, and it is impossible to associate the terminal capabilities with business information for analysis. Summary of the invention

[0003] The embodiments of the present invention provide a terminal perception evaluation method, a storage medium and an electronic device to at least solve the problem of missing information association and being unable to perform association analysis with business information when performing terminal perception analysis in the related art.

[0004] According to an embodiment of the present invention, a terminal perception evaluation method is provided, comprising: performing perception evaluation on key service indicators of the terminal according to basic terminal capabilities and frequency band information of the terminal; and evaluating the terminal according to the perception evaluation result of the key service indicators.

[0005] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0006] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0007] The present invention provides a terminal perception evaluation method, which performs perception evaluation on the key service indicators of the terminal according to the basic terminal capabilities and frequency band information of the terminal; and evaluates the terminal according to the perception evaluation results of the key service indicators. The present invention solves the problem of missing information association and failure to perform association analysis with service information when performing terminal perception analysis in the related technology, and achieves the effect of performing association analysis on the terminal in combination with service information to avoid missing information association. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1It is a hardware structure block diagram of a computer terminal of a terminal perception evaluation method according to an embodiment of the present invention;

[0009] Figure 2 is a schematic diagram of a network architecture for running a terminal perception evaluation method according to an embodiment of the present invention;

[0010] Figure 3 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention;

[0011] Figure 4 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention;

[0012] Figure 5 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention;

[0013] Figure 6 is a diagram showing the principle of setting a data acquisition interface according to a scenario embodiment of the present invention;

[0014] Figure 7 is a schematic diagram of a terminal perception evaluation process according to a scenario embodiment of the present invention;

[0015] Figure 8 Schematic diagram of threshold and weight updating principle according to a scenario embodiment of the present invention. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0018] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a terminal perception evaluation method according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0019] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the terminal perception evaluation method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0020] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0021] Figure 2 is a schematic diagram of a network architecture for running a terminal perception evaluation method according to an embodiment of the present invention. The embodiment of the present application can be run on Figure 2 In the network architecture shown in Figure 2 As shown in the figure, the bottom layer is the existing architecture of the mobile communication system, that is, the terminal is connected to the Internet APP server and the voice IMS domain system through the wireless base station and the core network to conduct mobile Internet access and voice call activities. On the basis of the mobile communication architecture, it also includes a data collection layer, which is used to collect specific data of users' mobile Internet access and voice calls, among which the number of key interfaces of the control plane can be 25, the number of key wireless interfaces can be 4, and the number of key interfaces of the user plane can be 2. The data sharing layer is used to clean the data sent by the collection layer, store the list data into the hadoop cluster, and store the terminal multi-dimensional statistical type data into the gbase database, so that the application layer can call the data. The data application layer is used to display the terminal perception profile analysis, capability analysis, network perception and head application, and support the terminal perception evaluation and delimitation capabilities.

[0022] In this embodiment, a terminal perception evaluation method running on the above-mentioned computer terminal or network architecture is provided. Figure 3 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0023] Step S302, performing a perception evaluation on the key service indicators of the terminal according to the basic terminal capability and frequency band information of the terminal;

[0024] In an exemplary embodiment, a perception evaluation is performed on the key service indicators of the terminal based on the basic terminal capabilities and frequency band information of the terminal, including: a perception evaluation is performed on the key service indicators of the terminal's voice calls and Internet access services based on the basic terminal capabilities and frequency band information.

[0025] In the actual implementation process, the key service indicators of the terminal's voice call and Internet services are perceived and evaluated, where the weights of different indicators can be set according to actual conditions, or obtained by big data learning of historical data. In the actual implementation process, different methods can also be used to perceive and evaluate the key service indicators, which is only an example here without specific limitation.

[0026] In an exemplary embodiment, a perception evaluation is performed on key service indicators of voice calls and Internet access services of a terminal, including: setting a poor perception threshold and a good perception threshold for the key service indicators, and performing a linear score based on the correspondence between the key service indicators and the value ranges of the poor perception threshold and the good perception threshold.

[0027] In the actual implementation process, the average values ​​of business parameters of key business indicators can be obtained. The specific threshold values ​​of the poor perception threshold and the good perception threshold for the key business indicators can be set according to actual conditions, or obtained by big data learning of historical data. There is no restriction here.

[0028] In the actual implementation process, the correspondence between the average value of the business parameter of the key business indicator and the value range of the poor perception threshold and the good perception threshold is: the poor perception threshold and the good perception threshold will form a value range, and the average value of the business parameters of different key business indicators will be distributed in the above value range, that is, the average value of the business parameter is dynamically distributed in the above value range, and is not fixedly distributed in a certain part of the above value range or a specific value. Therefore, specific scoring can be performed based on different distribution situations.

[0029] Step S304: Evaluate the terminal according to the perceived evaluation result of the key business indicator.

[0030] In an exemplary embodiment, the terminal is evaluated based on the perception evaluation results of the key business indicators, including: evaluating the terminal based on the change characteristics of the key business indicators before and after the activation of different terminal business capabilities to identify the perception types of different terminal models.

[0031] In an exemplary embodiment, the terminal service capability includes at least one of the following: uplink and downlink carrier aggregation (Carrier Aggregation, CA); dynamic spectrum sharing (Dynamic Spectrum Sharing, DSS); R16 energy saving; multi-band part (Bandwidth Part, BWP).

[0032] In the actual implementation process, the above capabilities can be enabled one by one or several or all of them at the same time to more comprehensively count the terminal data changes. In the actual implementation process, based on the perceived evaluation results of key business indicators, logarithmic analysis can be used to evaluate the terminals to distinguish the types of terminals.

[0033] In an exemplary embodiment, terminals are evaluated to identify perception types of different terminal models, including: classifying the perception type of each terminal into a high-perception user terminal or a low-perception user terminal according to change characteristics; and evaluating the corresponding models of the terminals according to the number statistics of high-perception user terminals and low-perception user terminals.

[0034] In the actual implementation process, for each terminal, the change characteristics of its key business indicators can be expressed as changes in the linear score results. The type of the terminal will be determined based on its change characteristics, that is, a high linear score is a high-perception user terminal, and a low linear score is a low-perception user terminal. In the actual implementation process, the high and low linear scores can be set according to the actual situation. The high and low thresholds are compared. In addition, for each terminal, the result of the linear score can be the weighted result of different business key indicators. After completing the division of high-perception user terminals and low-perception user terminals for different multiple terminals, the terminals of this brand type are evaluated for different terminal brand types. For example, among the MM brand terminals, the number of low-perception user terminals is too large, and the MM brand terminals can be determined as problem terminals. Specifically, for the determination of the number of high-perception user terminals and low-perception user terminals, specific thresholds can also be set according to the actual situation, and no specific restrictions are made here.

[0035] In the actual implementation process, the corresponding models of the terminals are evaluated. Combined with the number of high-perception user terminals and low-perception user terminals, terminals of different models can be divided into optimized terminals; terminals to be optimized; degraded terminals; and problem terminals.

[0036] In the actual implementation process, in order to more intuitively represent the types of terminals of different models, the four-quadrant bubble chart analysis method can be used to classify terminals into different types, such as: star terminals, potential terminals, declining terminals and problem terminals, and classify terminal types to facilitate user decision-making. In the actual implementation process, logarithmic analysis can be used, with the number of users in logarithmic form as the vertical coordinate and the perception score as the horizontal label of the bubble chart, in which the bubble size is positively correlated with the number of users, which can intuitively display the evaluation results for easy observation.

[0037] In an exemplary embodiment, Figure 4 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0038] Step S402, obtaining basic terminal capabilities and frequency band information according to information elements in the original signaling sent by the terminal;

[0039] Step S404, performing a perception evaluation on the key service indicators of the terminal according to the basic terminal capability and frequency band information of the terminal;

[0040] Step S406: Evaluate the terminal according to the perceived evaluation result of the key business indicator.

[0041] In the actual implementation process, the basic capabilities and frequency band information of the terminal are obtained according to the information elements in the original signaling sent by the terminal. Different bits may correspond to different terminal capabilities. To obtain the basic capabilities and frequency band information of the terminal according to different terminal capability information elements, a specific acquisition method, i.e., an acquisition rule, needs to be set, which will not be elaborated here.

[0042] In an exemplary embodiment, Figure 5 is a flow chart of a terminal perception evaluation method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0043] Step S502, obtaining basic terminal capabilities and frequency band information according to information elements in the original signaling sent by the terminal;

[0044] Step S504, performing a perception evaluation on the key service indicators of the terminal according to the basic terminal capability and frequency band information of the terminal;

[0045] Step S506, evaluating the terminal according to the perceived evaluation result of the key business indicator;

[0046] Step S508, collect terminal basic capabilities and frequency band information, build data clusters and databases; perform data learning based on the data clusters, databases, perception evaluation results of key business indicators, and terminal evaluation results, obtain perception evaluation thresholds of key business indicators and terminals, and perform iterative calibration of perception evaluation based on the perception evaluation thresholds. The data clusters are collected data during the evaluation process, and the database includes collected data, evaluation data, and historical data. The data clusters are used in the evaluation process, and the database is used to store the above data.

[0047] Through the above steps, a terminal perception evaluation method is provided, which performs perception evaluation on the key business indicators of the terminal according to the basic terminal capabilities and frequency band information of the terminal; and evaluates the terminal according to the perception evaluation results of the key business indicators. This solves the problem of missing information association and inability to perform association analysis with business information when performing terminal perception analysis in related technologies, and achieves the effect of performing association analysis on the terminal in combination with business information to avoid missing information association.

[0048] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0049] In this embodiment, a terminal perception evaluation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0050] A terminal perception evaluation device provided by an embodiment of the present invention may include: an indicator evaluation module, which is used to perform perception evaluation on key business indicators of a terminal based on the terminal basic capabilities and frequency band information of the terminal; a terminal evaluation module, which is used to evaluate the terminal based on the perception evaluation results of the key business indicators; an information acquisition module, which is used to obtain the terminal basic capabilities and frequency band information based on the information elements in the original signaling sent by the terminal; an iterative update module, which is used to collect the terminal basic capabilities and frequency band information and build a data cluster and a database; data learning is performed based on the data cluster, the database, the perception evaluation results of the key business indicators, and the evaluation results of the terminal to obtain the perception evaluation thresholds of the key business indicators and the terminal, so as to perform iterative calibration of the perception evaluation according to the perception evaluation thresholds.

[0051] In the actual implementation process, the terminal perception evaluation device is used to execute the specific processes and steps of the terminal perception evaluation method in the above embodiment, wherein the terminal perception evaluation device can be divided into different modules or units, which are not specifically limited here.

[0052] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0053] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0054] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0055] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0056] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0057] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0058] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0059] In order to enable those skilled in the art to better understand the technical solution of the present invention, it is described below in conjunction with specific scenario embodiments.

[0060] Scenario Example 1

[0061] The scenario embodiment of the present invention performs data collection by setting a data collection layer, wherein: Figure 6 : is a diagram showing the principle of setting the data acquisition interface according to a scenario embodiment of the present invention, such as Figure 6 As shown, the number of control plane key interfaces may be 25, the number of wireless key interfaces may be 4, and the number of user plane key interfaces may be 2.

[0062] The description of 5GC key interface information is shown in Table 1:

[0063] Table 1 5GC key interface information comparison table

[0064]

[0065]

[0066] The key interface information of EPC is shown in Table 2:

[0067] Table 2 EPC key interface information comparison table

[0068]

[0069] The key IMS interface information is shown in Table 3:

[0070] Table 3 IMS key interface information comparison table

[0071]

[0072]

[0073] Data is collected according to the settings of the above interface, and then terminal perception evaluation is performed based on the collected data. Figure 7 is a schematic diagram of the terminal perception evaluation process according to a scenario embodiment of the present invention, such as Figure 7 As shown, the following steps are included:

[0074] 1. Real-time and accurate extraction of terminal basic capabilities and frequency band information: Real-time extraction of information elements in the original signaling, timely and accurate identification of terminal basic capabilities UE-NR-Capability (such as VoNR, EPS FB, 5GC) and terminal frequency bands supportedBandListNR (such as 78&41&79&77&28&38&1&3&40&80&84), support terminal perception analysis and precise network planning and construction, compared with the traditional import terminal library configuration method, more timely, more accurate, and higher matching rate.

[0075] Different bits in the original signaling may correspond to a basic terminal capability. In the scenario embodiment of the present invention, there are 7 bits in total, and each bit is described as follows:

[0076] Position 0: 5G voice call (Voice Over New Radio, VoNR); Position 1: 5G voice call VoNR in low-frequency band; Position 2: 5G voice call VoNR in millimeter wave high-frequency band; Position 3: The mobile phone supports voice calls through the 4G core network; Position 4: The mobile phone supports voice calls through the 5G core network; Position 5: Support 5G independent networking; Position 6: Support 5G non-independent networking.

[0077] When acquiring data in real time through the N1N2 interface, different acquisition rules are used for different data types or data in different frequency bands, as follows:

[0078] For 5G voice calls, the mobile terminal's 5G wireless access capability->non-critical extended information->voice parameters->voice band parameter network differences->5G voice call, that is, the non-critical extended information is obtained based on the mobile terminal's 5G wireless access capability, and then the voice parameters and voice band parameter network differences are obtained, and finally the 5G voice call identifier is obtained.

[0079] For 5G voice calls in low-frequency bands, mobile terminal 5G wireless access capabilities -> non-critical extended information -> 5G low-frequency band mobile terminal capabilities -> voice band parameter network differences -> 5G voice calls (low-frequency band);

[0080] For 5G voice calls in the millimeter wave high frequency band, the mobile terminal 5G wireless access capability -> non-critical extended information -> 5G high frequency band mobile terminal capability -> voice band parameter network difference -> 5G voice call (millimeter wave high frequency band);

[0081] For 4G core network support for voice calls, mobile terminal 5G wireless access capability -> non-critical extended information -> voice parameters -> voice common parameters -> mobile phone supports voice calls through 4G core network;

[0082] For 5G core network to support voice calls, mobile terminal 5G wireless access capability -> non-critical extended information -> voice parameters -> voice common parameters -> mobile phone supports voice calls through 5G core network;

[0083] For 5G voice calls in low-frequency bands, the mobile terminal 4G wireless access capability -> non-critical extended information -> ... -> 5G wireless parameters -> 5G voice calls (low-frequency band);

[0084] For 5G voice calls in the millimeter wave high frequency band, the mobile terminal 4G wireless access capability -> non-critical extended information -> ... -> 5G wireless parameters -> 5G voice calls (millimeter wave high frequency band);

[0085] Mobile terminal 4G wireless access capability -> non-critical extended information -> ... -> Mobile terminal 4G wireless access capability -> 5G wireless parameters -> Enhanced dual link capability (support 5G non-independent networking);

[0086] Mobile terminal 4G wireless access capability -> non-critical extended information ->... -> Mobile terminal 4G wireless access capability -> 5G wireless parameters -> 5G independent networking.

[0087] The terminal frequency band information is obtained through the N1N2 interface, and the acquisition rules are also set as follows:

[0088] Mobile terminal 5G wireless access capability -> RF parameters -> 5G supported frequency band list number -> 5G frequency band or mobile terminal 4G wireless access capability -> non-critical extended information -> ... -> 5G wireless access parameters -> 5G supported frequency band list number, that is, the frequency band information is obtained from the mobile terminal 5G wireless access capability first;

[0089] If the frequency band information is obtained through the 4G wireless access capability of the mobile terminal, the terminal supports the NSA 5G frequency point acquisition rule (through the N1N2 interface):

[0090] Mobile terminal 4G wireless access capability -> non-critical extended information -> ... -> 5G wireless access parameters -> 5G supported frequency band list.

[0091] Among them, the ellipsis part in the above acquisition rules represents different data information transmission levels, which may have different levels according to actual conditions and will not be elaborated here.

[0092] 2. Terminal Perception Linear Scoring

[0093] In the scenario embodiment of the present invention, the terminal perception linear scoring specifically includes: mobile Internet access (smooth Internet access, smooth business) terminal perception evaluation, voice call (occupying, dialing, answering quickly, hearing clearly, no dropped calls) terminal perception evaluation, and mobile Internet access + voice call terminal comprehensive perception evaluation.

[0094] In the actual implementation process, big data clustering builds a terminal perception evaluation system from two levels: 5G terminal customers' Internet perception (smooth Internet access and smooth business) and voice calls (available for call, connectable, fast answering, clear hearing, and no dropped calls); the terminals included in the TOP evaluation are based on a baseline of more than 500 terminals. Only by ensuring the number of terminals collected can the accuracy of data acquisition be guaranteed.

[0095] In the scenario embodiment of the present invention, the following scoring rules and proportions are set for the perception evaluation of the terminal. In the actual implementation process, the specific scoring settings and proportion divisions can be set according to the actual situation. This is only an example and is not a restrictive description.

[0096] (1) Big data terminal perception evaluation system: The data service weight is 70% and the voice service weight is 30%, which evaluates the overall situation of the terminal.

[0097] (2) Overall business scoring rule: Linear scoring is performed according to the dynamic distribution of the average value of the indicator in the (worst, best) interval, with poor perception (1-2), good perception (2-4), and good perception (4-5). Specifically, if the current indicator is lower than the poor perception threshold, the indicator score is 0; if the current indicator is higher than the good perception threshold, the indicator score is 5; if the current indicator is between the poor perception threshold and the good perception threshold, dynamic linear scoring is performed.

[0098] (3) Detailed business scoring algorithm: Set different weights for different key indicators, and then calculate the total perception score, UE_QoE (perception score) = key indicator 1_QoE*key indicator 1_weight+key indicator 2_QoE*key indicator 2_weight+key indicator 3_QoE*key indicator 3_weight+...+key indicator N_QoE*key indicator N_weight.

[0099]

[0100] Among them, score represents the quality score of a certain indicator, which refers to UE_QoE in the embodiment of the present invention; x represents the average value of the indicator. The average value of the composite indicator needs to be summed up to obtain the numerator and denominator respectively before calculation, and cannot be directly calculated by arithmetic average. i represents the label of the same key indicator. Worst and best are the cutoff thresholds of the indicator, but when the indicator is outside the (worst, best) interval, it is directly set to 0 or 5. Direction is the direction of the indicator. When it is equal to 1, the larger the indicator, the better, and when it is equal to -1, the smaller the indicator, the better.

[0101] (4) Thresholds and weights support flexible configuration

[0102] In the scenario embodiment of the present invention, the terminal indicator threshold, that is, the cutoff threshold and weight of the above indicator are configurable to enhance the applicability in different scenarios. An example is shown in the following table:

[0103] Table 4 Comparison table of thresholds and weight configurations of different indicators

[0104]

[0105] In the actual implementation process, it is also possible to (5) support the perception of poor-quality terminals for key businesses (such as Douyin, WeChat, Toutiao, etc.) to meet the actual needs of users. Head applications can be flexibly configured according to user needs.

[0106] In the process of data analysis and display, logarithmic analysis can be used, with the number of users in logarithmic form as the vertical coordinate and the perception score as the horizontal label of the bubble chart, where the bubble size is positively correlated with the number of users, and the evaluation results can be displayed intuitively for easy observation. The four-quadrant bubble chart analysis method divides terminals into star terminals, potential terminals, declining terminals, and problem terminals, and classifies terminal types to facilitate user decision-making.

[0107] 3. Correlation Analysis between Terminal Capabilities and Perception Indicators

[0108] Evaluate and compare the change characteristics of key indicators of voice calls and mobile Internet access after enabling R16 UE energy saving, uplink and downlink CA capabilities, DSS, and multi-BWP support capabilities on the terminals, and identify terminal models with poor perception; among them, the above capabilities can be enabled one by one, or several or all of them can be enabled at the same time to more comprehensively count the terminal data changes.

[0109] (1) Terminal capability statistics support downlink and uplink (DL / UL) CA;

[0110] SA networks are gradually maturing, and enhanced technologies such as carrier aggregation (CA) have been slowly being commercialized. Whether the activation of the CA function will have any impact on the use of ordinary terminals in the existing network, as well as the proportion of CA-supported terminals in the entire network, the distribution of chips and brands, and the indicators of CA terminals themselves in the existing network will help promote network maturity and increase the user experience of terminals of different levels.

[0111] (2) Terminal capability statistics support dynamic spectrum sharing (DSS);

[0112] Since 5G spectrum resources are relatively precious, DSS technology can balance the dynamic resource allocation of 4 / 5G users under the same spectrum, which is conducive to improving the utilization of network resources. It is necessary to monitor the performance of DSS-supported terminals in real commercial networks after the DSS function is enabled, as well as various network indicators, to ensure that resource allocation gains are not affected while ensuring that existing 4G and 5G users are not affected.

[0113] (3) Terminal capability statistics support R16 UE energy saving;

[0114] What consumers care most about is the use and standby capacity of 5G terminals. The introduction of new R16 energy-saving technologies will increase the terminal usage time, which will inevitably have some impact on the network, and the resource scheduling strategy needs to be adjusted in time. Therefore, it is necessary to identify terminals that support R16 UE energy saving, the proportion of terminals that support R16 UE energy saving in the entire network, the distribution of chips and brands, and the indicators of the terminals themselves in the existing network.

[0115] (4) Terminal capability statistics support multiple BWPs

[0116] In the R15 stage, the application of multi-BWP, a power-saving technology for SA network UEs, has already seen some problems that affect the re-establishment of indicators in the existing network. Whether the multi-BWP function will have an impact on the use of ordinary terminals in the existing network after it is enabled, as well as the proportion of terminals that support multi-BWP in the entire network, the distribution of chips and brands, and the indicators of multi-BWP terminals themselves in the existing network will help promote network maturity and increase the user experience of terminals of different levels.

[0117] 4. Terminal threshold self-intelligence.

[0118] In the actual implementation process, the settings of the above thresholds and weights can be automatically updated and adjusted through big data learning. The thresholds for judging the high and low quality of terminals are the results of self-learning of big data in the existing network. The rules support decoupling and adjustable thresholds, which are in line with the threshold characteristics of the coexistence of 4 / 5G in the existing network. Combined with the user's business behavior, it is supplemented according to the actual data rules and customer expert experience to realize the intelligent end-to-end collaborative quality analysis. Figure 8 Schematic diagram of threshold and weight update principle according to a scenario embodiment of the present invention. Figure 8As shown, after the DPI basic data is collected, the data is cleaned, and data learning is performed based on the collected data, including the mobile Internet terminal dimension perception indicators and the voice call terminal dimension perception indicators, to complete the self-learning of the benchmark values ​​and challenge values ​​of various indicators. Among them, the top 20% within 90 days can be selected as the challenge value, and the tail 20% can be selected as the benchmark value for data learning, so as to obtain new thresholds and weights for updating before performing terminal perception evaluation.

[0119] In summary, the present invention provides a terminal perception evaluation method, which selects key indicators in voice calls and mobile Internet access, adopts a weighted linear scoring mechanism, evaluates the comprehensive perception scores of various terminal models, and screens out the top typical terminals with good and poor perception in each price range, while supporting custom configuration of weights; real-time and accurate extraction of terminal basic capabilities and frequency band information: real-time extraction of information elements in the original signaling, timely and accurate identification of terminal basic capabilities UE-NR-Capability (such as VoNR, EPS FB, 5GC) and terminal frequency bands supportedBandListNR (such as 78&41&79&77&28&38&1&3&40&80&84), terminal-network collaborative analysis, and supports accurate network planning and construction. Analysis of the correlation between terminal capabilities and perception indicators: evaluate and compare the change characteristics of key indicators of voice calls and mobile Internet access after the terminal turns on R16 UE energy saving, uplink and downlink CA capabilities, and multi-BWP support capabilities, and identify terminal models with poor perception. Terminal threshold self-intelligence: The threshold for judging the high and low quality of terminals is the result of self-learning of the existing network big data. The rules support decoupling and the threshold is adjustable and configurable, which conforms to the threshold characteristics of the existing network when 4 / 5G coexist, and is supplemented by the user's business behavior to realize the intelligent end-to-end collaborative quality analysis. Through the present invention, the comprehensive perception of the terminal can be accurately evaluated, and the terminals with poor perception can be demarcated.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A terminal perception evaluation method, characterized in that: include: Performing a perception evaluation on key service indicators of the terminal according to terminal basic capabilities and frequency band information of the terminal; The terminal is evaluated according to the perceived evaluation result of the key business indicator.

2. The method according to claim 1, characterized in that Before performing perceptual evaluation on the key service indicators of the terminal according to the terminal basic capability and frequency band information of the terminal, the method further includes: The basic capability of the terminal and the frequency band information are acquired according to the information elements in the original signaling sent by the terminal.

3. The method according to claim 1, characterized in that The performing perceptual evaluation on the key service indicators of the terminal according to the terminal basic capability and frequency band information of the terminal includes: According to the basic capabilities of the terminal and the frequency band information, a perception evaluation is performed on the key service indicators of the voice call and Internet access services of the terminal.

4. The method according to claim 3, characterized in that The key service indicators of the voice call and Internet access services of the terminal are perceived and evaluated, including: A poor perception threshold and a good perception threshold are set for the key business indicator, and a linear score is performed according to the correspondence between the key business indicator and the value ranges of the poor perception threshold and the good perception threshold.

5. The method according to claim 1, characterized in that The terminal is evaluated according to the perceived evaluation result of the key service indicator, including: According to the change characteristics of the key service indicators before and after the activation of different terminal service capabilities, the terminal is evaluated to identify the perception types of different terminal models.

6. The method according to claim 5, characterized in that The terminal is evaluated to identify the perception type of different terminal models, including: Classifying the perception type of each terminal into a high-perception user terminal or a low-perception user terminal according to the change characteristics; According to the quantity statistics of the high-perception user terminals and the low-perception user terminals, the corresponding models of the terminals are evaluated.

7. The method according to claim 5, characterized in that in, The terminal service capability includes at least one of the following: Uplink and downlink carrier aggregation CA; Dynamic spectrum sharing DSS; R16 energy saving; Multi-band partial BWP.

8. The method according to claim 1, characterized in that Also includes: Collecting the basic capabilities of the terminal and the frequency band information, and building a data cluster and a database; Data learning is performed based on the data cluster, the database, the perception evaluation results of the key business indicators, and the evaluation results of the terminal to obtain the perception evaluation thresholds of the key business indicators and the terminal, so as to iteratively calibrate the perception evaluation according to the perception evaluation thresholds.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any one of claims 1 to 8 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method described in any one of claims 1 to 8 is implemented.

Images