Early warning user determination method, apparatus and device for 5g voice service, and storage medium

Abstract

The application provides a method and device for determining a pre-warning user of a 5G voice service, an apparatus, and a storage medium, relating to the technical field of communication. The method realizes the determination of a pre-warning user with poor network perception. The method comprises: a determining apparatus acquiring voice service data of a target user in each preset period; the determining apparatus determining a voice perception key indicator in each preset period according to the voice service data of the target user in each preset period; the determining apparatus determining an evaluation result of voice perception quality of the target user in each preset period according to the voice perception key indicator in each preset period, the evaluation result being used to represent whether the voice perception quality of the target user in the preset period meets the standard; and the determining apparatus determining whether the target user is a pre-warning user according to the evaluation results in the multiple preset periods. The application can be used in the process of 5G voice service operation and maintenance, and can be used to solve the problem of poor user perception.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method, apparatus, device and storage medium for early warning user identification of 5G voice services. Background Technology

[0002] With the increasing maturity of 5G technology and the rapid growth of 5G users, voice services are gradually moving towards VONR solutions after the commercialization of Voice over New Radio (VONR). VONR is a high-definition voice value-added service based on 5G networks. It utilizes the 5G access network, 5G core network, and IP multimedia subsystem (IMS) provided by 5G standalone (SA) networks to independently carry high-definition video calls, supporting simultaneous voice calls and high-speed internet access. Users can enjoy a higher quality voice service experience and a higher data service speed. Simultaneously, user perception of voice services has been elevated to a new level; 5G VONR voice perception is a crucial aspect of users' assessment of the quality of an operator's network.

[0003] However, with the construction and deployment of 5G networks, network complexity increases, and network operation pressure rises. Timely identification and optimization of users with poor network perception are crucial for operators. Currently, however, 5G VoNR voice services lack a digital and intelligent method for identifying and addressing these users. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for identifying early warning users of 5G voice services, which enables intelligent identification of early warning users with poor network awareness.

[0005] In a first aspect, this application provides a method for identifying early warning users of 5G voice services. The method includes: acquiring voice service data of a target user within each preset period, wherein the voice service data characterizes the target user's voice perception during 5G VoNR usage; determining key voice perception indicators for each preset period based on the target user's voice service data, wherein the key voice perception indicators characterize key parameters for analyzing the target user's 5G VoNR voice perception quality; determining evaluation results of the target user's voice perception quality for each preset period based on the key voice perception indicators, wherein the evaluation results characterize whether the target user's voice perception quality meets the standards within the preset period; determining whether the target user is an early warning user based on the evaluation results across multiple preset periods; and identifying early warning users as users requiring voice perception optimization.

[0006] The method for identifying early warning users of 5G voice services provided in this application involves a device that determines key voice perception indicators for each preset period based on voice service data of the target user acquired in each preset period. Then, based on these key indicators, it determines the evaluation result of the target user's voice perception quality. Furthermore, based on the evaluation results across multiple preset periods, it can determine whether the target user is an early warning user. This application achieves intelligent and automatic identification of early warning users with poor network perception by analyzing voice service data, enabling administrators to optimize the voice perception of these early warning users.

[0007] One possible implementation method is that the key indicators of voice perception include at least one of the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and the mean opinion score (MOS) value of VONR's real-time transport protocol (RTP).

[0008] Another possible implementation is as follows: the VONR initial registration success rate is used to represent the ratio of the number of successful VONR initial registrations of the target user to the total number of VONR initial registrations within a preset period; the VONR originating network connection rate is used to represent the ratio of the number of successful VONR originating network connections of the target user to the number of VONR originating network attempts within a preset period; the VONR final call connection rate is used to represent the ratio of the number of successful VONR final call connections of the target user to the number of VONR final call attempts within a preset period; and the VONR average call latency is used to represent the average time interval between the target user sending an invitation message and the called user receiving the invitation message within a preset period.

[0009] The VONR dropped call rate is used to characterize the ratio of the number of VONR dropped calls to the number of VONR answered calls for a target user within a preset period; the MOS value is used to characterize the average voice quality perceived by the target user during a VONR call within a preset period.

[0010] Another possible implementation is to determine the evaluation result of the target user's speech perception quality in each preset period based on the key speech perception indicators in each preset period. This includes: for each preset period, if each parameter in the key speech perception indicators meets the preset conditions, it is determined that the target user's speech perception quality meets the standard in the preset period; different parameters correspond to different preset conditions; otherwise, it is determined that the target user's speech perception quality does not meet the standard in the preset period.

[0011] Another possible implementation involves multiple preset periods consisting of N consecutive preset periods. The determination of whether a target user is a warning user is based on the evaluation results within these preset periods. This includes: if the target user's evaluation results fail to meet the standard for M preset periods within the N consecutive preset periods, the target user is determined to be a warning user; where N is a positive integer greater than or equal to 2, and M is a positive integer less than N; or, if the target user's evaluation results fail to meet the standard in every preset period within the N consecutive preset periods, the target user is determined to be a warning user.

[0012] Another possible implementation method, the above method also includes: inputting key indicators of voice perception into a voice service analysis model to obtain a solution; the voice service analysis model is used to analyze the causes of anomalies based on the key indicators of voice perception, and output corresponding solutions based on the causes of anomalies; and optimizing the voice perception of users who receive early warnings based on the solutions.

[0013] Secondly, this application provides a device for identifying early warning users of 5G voice services. The device includes an acquisition module and a determination module. The acquisition module is used to acquire voice service data of a target user within each preset period, wherein the voice service data characterizes the voice perception status of the target user during 5G VoNR usage. The determination module is used to determine key voice perception indicators for each preset period based on the voice service data of the target user within each preset period, wherein the key voice perception indicators characterize key parameters for analyzing the 5G VoNR voice perception quality of the target user. The determination module is also used to determine the evaluation result of the target user's voice perception quality for each preset period based on the key voice perception indicators, wherein the evaluation result characterizes whether the target user's voice perception quality meets the standard within the preset period. The determination module is further used to determine whether the target user is an early warning user based on the evaluation results within multiple preset periods, wherein the early warning user is a user requiring voice perception optimization.

[0014] One possible implementation method is that the key indicators of voice perception include at least one of the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and MOS value.

[0015] Another possible implementation is as follows: VONR initial registration success rate is used to characterize the ratio of the number of successful initial VONR registrations of the target user to the total number of initial VONR registrations within a preset period; VONR originating network connection rate is used to characterize the ratio of the number of successful initial VONR network calls of the target user to the number of attempted initial VONR network calls within a preset period; VONR ending network connection rate is used to characterize the ratio of the number of successful final VONR network calls of the target user to the number of attempted final VONR network calls within a preset period; VONR average call latency is used to characterize the average time interval between the target user sending an invitation message and the called user receiving the invitation message within a preset period; VONR dropped call rate is used to characterize the ratio of the number of dropped VONR calls of the target user to the number of VONR responses within a preset period; and MOS value is used to characterize the average perceived voice quality of the target user during a VONR call within a preset period.

[0016] Another possible implementation is to determine that, for each preset period, if each parameter in the key indicators of speech perception meets the preset conditions, the target user's speech perception quality meets the standard within the preset period; different parameters correspond to different preset conditions; otherwise, the target user's speech perception quality does not meet the standard within the preset period.

[0017] Another possible implementation is that multiple preset periods are N consecutive preset periods. The determination module is specifically used to determine the target user as a warning user if the evaluation results of the target user fail to meet the standard in M ​​preset periods within N consecutive preset periods; N is a positive integer greater than or equal to 2; M is a positive integer less than N; or, if the evaluation results of the target user fail to meet the standard in each preset period within N consecutive preset periods, the target user is determined as a warning user.

[0018] In another possible implementation, the above device further includes a processing module; the processing module is used to input key voice perception indicators into a voice service analysis model to obtain a solution; the voice service analysis model is used to analyze the causes of anomalies based on the key voice perception indicators and output corresponding solutions based on the causes of anomalies; the processing module is also used to optimize the voice perception of early warning users based on the solutions.

[0019] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0020] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; when the computer software instructions are executed in an electronic device, they cause the electronic device to implement the method described in the first aspect.

[0021] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the steps of the relevant method described in the first aspect above, so as to implement the method of the first aspect above.

[0022] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0023] Figure 1 A schematic diagram of the application environment for the method for determining early warning users of 5G voice services provided in this application;

[0024] Figure 2 A flowchart illustrating a method for determining early warning users for 5G voice services provided in this application;

[0025] Figure 3 A flowchart illustrating another method for determining early warning users for 5G voice services provided in this application;

[0026] Figure 4 A flowchart illustrating another method for determining early warning users for 5G voice services provided in this application;

[0027] Figure 5 A schematic diagram of a system architecture is provided for this application;

[0028] Figure 6 A schematic diagram illustrating the composition of a 5G voice service early warning user identification device provided in this application;

[0029] Figure 7 This is a schematic diagram of the composition of an electronic device provided in this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0032] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0033] In the early stages of 5G network deployment, operators' SA networks use the Evolved Packet System (EPS) fallback scheme to implement 5G voice services. After VoNR commercialization, a coexistence of VoNR and EPS fallback schemes will be used for a period to ensure user voice service availability. Once 5G SA deployment is sufficiently mature, voice services will ultimately be implemented using the VoNR scheme. As voice service is a fundamental service provided by operator networks, user perception of voice service is a crucial reference for evaluating operator network performance. Therefore, timely identification and optimization of users with poor network perception are fundamental to ensuring a good user voice service experience. However, currently, 5G VoNR voice services lack digital and intelligent methods for identifying and addressing these users.

[0034] In summary, there is an urgent need for an intelligent method to identify users with poor network perception. Based on this, this application provides a method for identifying users with poor network perception in 5G voice services. In this method, the determining device can determine key voice perception indicators for each preset period based on the voice service data of the target user acquired in each preset period. Then, it determines the evaluation result of the target user's voice perception quality based on the key voice perception indicators, and further determines whether the target user is a user with poor network perception based on the evaluation results over multiple preset periods. This application achieves intelligent and automatic identification of users with poor network perception by analyzing voice service data, enabling administrators to optimize the voice perception of these users.

[0035] The method for determining early warning users for 5G voice services provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes a 5G voice service early warning user identification device 101 (which can be simply referred to as the identification device) and a base station device 102. The identification device 101 and the base station device 102 can be connected via a wired network or a wireless network. The wired network or wireless network may include a router, a switch, or other devices that facilitate communication between the identification device 101 and the base station device 102; this embodiment does not limit the scope of the application.

[0036] In some embodiments, the determining device 101 may be a server cluster consisting of multiple servers, a single server, a computer, or a processor or processing chip in a server or computer, etc. This application does not limit the specific device form of the determining device 101. Figure 1 The example shown is that device 101 is a single server.

[0037] In some embodiments, the base station device 102 may be an evolved NodeB (eNB), a next-generation NodeB (gNB), a transmission receive point (TRP), a transmission point (TP), or some other access node. Depending on the size of the service coverage area provided, base stations can be further classified as macro base stations for providing macrocells, micro base stations for providing microcells, and femto base stations for providing femtocells. As wireless communication technology continues to evolve, future base stations may also adopt other names.

[0038] In some embodiments, when it is necessary to identify users with poor network perception, the determining device 101 can obtain the voice service data of the target user from the base station device 102, determine the key indicators of voice perception based on the voice service data, further determine the evaluation result of voice perception quality, and thus identify the users to be warned.

[0039] Figure 2 This is a flowchart illustrating a method for determining early warning users for 5G voice services, provided as an embodiment of this application. Figure 2 As shown, the method for determining early warning users for 5G voice services provided in this application can be implemented using the aforementioned determining device, and specifically includes the following steps:

[0040] S201. Within multiple preset periods, the device determines the voice service data of the target user within each preset period.

[0041] Among them, voice service data is used to characterize the voice perception of target users during the process of using 5G New Radio to carry voice VONR.

[0042] In some embodiments, within multiple preset periods, the determining device can obtain 5G voice service data of a target user from the base station for each preset period. The voice service data characterizes the voice perception of the target user during the process of using 5G New Radio (VoNR) to carry voice communication. The target user is any user within the target area who has activated 5G VoNR voice service.

[0043] It should be noted that the preset period can be any time period. For example, the determining device can select a time period of 1 day as the preset period. This application embodiment does not specifically limit this.

[0044] Optionally, the determining device can acquire voice service data of target users within the target area based on the target area configured by the administrator.

[0045] For example, the determining device can acquire voice service data of target users from base stations in the target area on a daily time granularity, and clean the data to remove useless data to ensure data availability. The voice service data includes external data representation (XDR) data of target users within the target area over a preset period.

[0046] It should be noted that XDR event logs are relatively raw data that comprehensively and objectively reflect the user experience during service usage. This data is a structured event-level record synthesized by collecting multi-interface signaling messages between devices and parsing the parameters within them. It covers all basic information of a user event. For example, VONR voice XDR data covers basic information of all interface signaling data such as N1, N2, and Mw in the 5GC domain and IMS domain. By mining event logs, the user experience of target users can be quickly and accurately analyzed. The main characteristics of XDR event log data are summarized as follows: a. Structured event-level records synthesized from multiple interfaces, containing all interface signaling data information of a single event, which can be used to build models for in-depth user behavior analysis; b. Rich field types, including user number, user's international mobile equipment identity (IMEI), service start and end time, service occurrence cell, etc., which can be combined with multiple fields for multi-dimensional joint analysis to solve problems that were previously difficult to solve; c. The data contains a large number of user and terminal information fields such as user number and user IMEI, which facilitates mining and analysis at the user and terminal dimensions when dealing with specific problems.

[0047] S202, The determining device determines the key voice perception indicators for each preset period based on the voice service data of the target user within each preset period.

[0048] Among them, the key indicators of voice perception are used to characterize the key parameters of the 5G VONR voice perception quality of the target user.

[0049] In some embodiments, after acquiring voice service data within each preset period, the determining device can determine voice service data related to key voice perception indicators based on the voice service data of the target user within each preset period, thereby determining the key voice perception indicators of the target user's voice service within each preset period. These key voice perception indicators are used to characterize and analyze the key parameters of the target user's 5G VONR voice perception quality.

[0050] The key indicators for voice perception include at least one of the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and the average voice opinion score (MOS) of VONR's real-time transport protocol RTP (referred to as VONR RTP average voice MOS).

[0051] S203. The determining device determines the evaluation result of the target user's speech perception quality in each preset period based on the key speech perception indicators in each preset period.

[0052] The evaluation results are used to characterize whether the target user's speech perception quality meets the standard within a preset period.

[0053] In some embodiments, after determining the key speech perception indicators of the target user within each preset period, the determining device determines that the speech perception quality of the target user meets the standard within each preset period, provided that each parameter in the key speech perception indicators meets preset conditions. Different parameters correspond to different preset conditions. Otherwise, it is determined that the speech perception quality of the target user does not meet the standard within the preset period. The evaluation result is used to characterize whether the speech perception quality of the target user meets the standard within the preset period.

[0054] S204. The device determines whether the target user is a warning user based on the evaluation results within multiple preset periods.

[0055] Among them, the users who are given a warning are those whose voice perception needs to be optimized.

[0056] In some embodiments, based on the above method, after determining the evaluation results of the voice perception quality of a target user within multiple preset periods, the determining device determines whether the target user is a warning user based on the evaluation results within the multiple preset periods. The warning user is a user whose voice perception needs to be optimized. The technical solution provided by the above embodiments brings at least the following beneficial effects: The 5G voice service warning user determination method provided in this application embodiment allows the determining device to determine key voice perception indicators for each preset period based on the acquired voice service data of the target user within each preset period, then determine the evaluation results of the target user's voice perception quality based on the key voice perception indicators, and further determine whether the target user is a warning user based on the evaluation results within the multiple preset periods. This application achieves intelligent and automatic identification of warning users with poor network perception by analyzing voice service data, so that managers can optimize the voice perception of warning users.

[0057] The following describes in detail the method for determining early warning users of 5G voice services provided in this application, with reference to specific embodiments and accompanying drawings.

[0058] like Figure 3 As shown, the method for determining early warning users for 5G voice services provided in this application may specifically include the following steps:

[0059] S301. Within multiple preset periods, the device determines the voice service data of the target user within each preset period.

[0060] Among them, voice service data is used to characterize the voice perception of target users during the process of using 5G New Radio to carry voice VONR.

[0061] The relevant description of S301 above can be found in the description of S201 above, and will not be repeated here.

[0062] S302. The determining device determines the key voice perception indicators for each preset period based on the voice service data of the target user within each preset period.

[0063] Among them, the key indicators of voice perception are used to characterize the key parameters of the 5G VONR voice perception quality of the target user.

[0064] In some embodiments, after the determining device acquires the voice service data of the target user within each preset period, for each preset period, the determining device can determine key voice perception indicators based on the XDR data in the voice service data. These key voice perception indicators include at least one of the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and VONR RTP average voice MOS value.

[0065] For example, when determining the key indicator of the initial registration success rate of the voice service of a target user, the determining device can obtain the number of successful initial registrations of VONR and the total number of initial registrations of VONR from the voice service data within a preset period, and then determine the initial registration success rate of VONR according to the following formula.

[0066]

[0067] It should be noted that the number of successful initial VONR registrations and the total number of initial VONR registrations can be determined based on the fields recorded in the XDR data. Specifically, the number of successful initial VONR registrations is the number of times the service-call session control function (S-CSCF) sends a 200 OK response to the initial registration request within a preset period. The total number of initial VONR registrations is the number of times the S-CSCF receives an initial registration request Register (buffer) within the preset period.

[0068] For another example, when determining the key indicator of the VONR originating network connection rate of the voice service of the target user, the determining device can obtain the number of VONR originating network connection attempts and the number of VONR originating network trial calls from the voice service data within a preset period, and then determine the VONR originating network connection rate according to the following formula, where 5G QoS Identifier (5QI) = 1.

[0069]

[0070] It should be noted that the number of VONR originating network connection successes and the number of VONR originating network attempt successes can be determined based on the fields recorded in the XDR data. Specifically, the number of VONR originating network connection successes refers to the number of times the calling party's session border control (SBC) successfully forwards 180 response messages, 183 (carrying P-Early Media), update (carrying P-Early Media), 403 response messages, 404 response messages, 405 response messages, 413 response messages, 414 response messages, 415 response messages, 416 response messages, 422 response messages, 423 response messages, 480 response messages, 486 response messages, 487 response messages, 488 response messages, 600 response messages, 603 response messages, 604 response messages, and 606 response messages to the calling user within a preset period after receiving a voice invitation request. The number of VONR initiating network call attempts is the number of times the calling SBC receives a voice invitation request within a preset period.

[0071] For another example, when determining the key indicator of the VONR final call network connection rate of the voice service of a target user, the determining device can obtain the number of VONR final call network connection attempts and the number of VONR final call network trial calls from the voice service data within a preset period, and then determine the VONR final call network connection rate according to the following formula, where 5QI = 1.

[0072]

[0073] It should be noted that the number of VONR final call network connections and the number of VONR final call network attempts can be determined based on the fields recorded in the XDR data. Specifically, the number of VONR final call network connections refers to the number of times the called party's SBC successfully forwards 180, 183 (carrying P-Early Media), update (carrying P-Early Media), 403, 404, 405, 413, 414, 415, 416, 422, 423, 480, 486, 487, 488, 600, 603, 604, and 606 response messages to the calling party after receiving a voice invitation request within a preset period. The number of VONR final call network attempts refers to the number of times the called party's SBC receives a voice invitation request within a preset period.

[0074] For another example, when determining the key indicator of the average VONR call latency of the target user's voice service, the determining device calculates the time interval between the target user sending all invitation messages and the called user receiving the invitation message within a preset period, and takes the average of all time intervals within the preset period as the average VONR call latency.

[0075] For another example, when determining the key indicator of the VONR call drop rate of a target user's voice service, the determining device can obtain the number of VONR call drops and the number of VONR responses from the voice service data within a preset period, and then determine the VONR call drop rate according to the following formula.

[0076]

[0077] It should be noted that the number of VONR dropped calls and the number of VONR responses can be determined based on the fields recorded in the XDR data. Specifically, the number of VONR dropped calls is the number of access service requests (ASRs) sent by the Policy and Charging Rules Function (PCRF) with voice or media type received by the SBC (regardless of calling or called domain) within a preset period, excluding ASRs with an abort cause of "PS to CS Handover". The number of VONR responses is the sum of the number of VONR originating network connections and the number of VONR terminating network connections within the preset period.

[0078] For another example, when determining the key indicator of the average voice MOS value of the VONR RTP for a target user's voice service, the determining device can obtain the total VONR RTP voice MOS value and the number of VONR RTP voice MOS counts from the voice service data within a preset period, and then determine the average voice MOS value of VONR RTP according to the following formula.

[0079]

[0080] It should be noted that the total MOS value and the number of MOS counts in the VONR RTP statistics can be directly determined from the fields recorded in the XDR data. The average MOS value in the VONR RTP is used to evaluate the average perceived voice quality during a VONR call over a period of time.

[0081] S303. The determining device determines the evaluation result of the target user's speech perception quality in each preset period based on the key speech perception indicators in each preset period.

[0082] The evaluation results are used to characterize whether the target user's speech perception quality meets the standard within a preset period.

[0083] In some embodiments, the device monitors each parameter of the key speech perception indicators of the target user in each preset period and determines whether each parameter meets preset conditions. For each preset period, if each parameter of the key speech perception indicators meets the preset conditions, the device determines that the evaluation result of the target user's speech perception quality in the preset period is satisfactory; otherwise, it determines that the evaluation result of the target user's speech perception quality in the preset period is unsatisfactory.

[0084] For example, the device statistically monitors each parameter of the key speech perception indicators for the target user within each preset period, compares them with preset conditions, and determines that parameters that do not meet the preset conditions are substandard. If one or more of the six parameters for the target user within a preset period are substandard, the evaluation result of the target user's speech perception quality for that preset period is determined to be substandard. The preset conditions are either default settings of the device or pre-set by administrators based on practical experience. As an example, the preset conditions for the key speech indicators are shown in Table 1 below.

[0085] Table 1

[0086]

[0087]

[0088] Table 1 contains six parameters of key voice perception indicators and their corresponding thresholds (i.e., preset conditions). The preset conditions for VONR initial registration success rate are "99%" and the target user's VONR initial registration success rate must be greater than 99% to be considered satisfactory. The preset conditions for VONR originating network connection rate are "98%" and the target user's VONR originating network connection rate must be greater than 98% to be considered satisfactory. The preset conditions for VONR final network connection rate are "98%" and the target user's VONR final network connection rate must be greater than 98% to be considered satisfactory. The preset condition for VONR average call latency is "3500ms" and the target user's VONR average call latency must be less than 3500ms to be considered satisfactory. The preset condition for VONR call drop rate is "0.1%" and the target user's VONR call drop rate must be less than 0.1% to be considered satisfactory. The preset condition for VONR RTP average voice MOS value is "3.5" and the target user's VONR RTP average voice MOS value must be greater than 3.5 to be considered satisfactory.

[0089] Specifically, as an example, the speech perception quality evaluation results of target users within a preset period are shown in Table 2.

[0090] Table 2

[0091]

[0092]

[0093] Table 2 includes "User Rating," "Date," "Province," "City," "User Number," and six key voice perception indicators along with their corresponding perception ratings. "User Rating" represents the evaluation result, indicating the voice perception quality of the target user within a preset period. "Date" indicates the start and end times of the preset period. "Province," "City," and "User Number" represent the target user's target region and their phone number, respectively. The "VONR Initial Registration Success Rate" is 99.5%, greater than 99%, therefore its perception rating is satisfactory. The "VONR Initial Call Network Connection Rate" is 97%, less than 98%, therefore its perception rating is unsatisfactory. The "VONR Final Call Network Connection Rate" is 98.5%, greater than 98%, therefore its perception rating is satisfactory. The "VONR Average Call Latency" is 3400ms, less than 3500ms, therefore its perception rating is satisfactory. The "VONR Call Drop Rate" is 0.05%, less than 0.1%, therefore its perception rating is satisfactory. The "VONR RTP average speech MOS value" is 3.8, which is greater than 3.5, so the perception rating for this parameter is "meets the standard". Therefore, one of the six key speech perception indicators for this target user is "not up to standard", hence the user rating for this target user is "not up to standard".

[0094] S304. The device determines whether a target user is a warning user based on the evaluation results within multiple preset periods.

[0095] Among them, the users who are given a warning are those whose voice perception needs to be optimized.

[0096] In some embodiments, if the evaluation results of the target user fail to meet the standard for M preset periods within N consecutive preset periods, the determining device determines the target user as a warning user; N is a positive integer greater than or equal to 2; M is a positive integer less than N; or, if the evaluation results of the target user fail to meet the standard in each preset period within N consecutive preset periods, the determining device determines the target user as a warning user. The warning user is the user to be optimized for voice perception.

[0097] For example, after determining the evaluation results of the target user's speech perception quality on a daily time granularity, the device obtains the target user's evaluation results for the current day and the evaluation results for the two most recent days. If there are two or more days of evaluation results that are "unsatisfactory" in these three days of evaluation results, the target user is determined to be a warning user. As an example, the target user's three-day evaluation results are shown in Table 3.

[0098] Table 3

[0099]

[0100]

[0101] Table 3 includes "Date", "Province", "City", "User Number", "Warning Status", and the target user's user rating for three consecutive days. "Date" indicates the day on which the target user was determined to be a warning user. "Province", "City", and "User Number" represent the target user's basic information. "Warning Status" indicates whether the target user is a warning user. "Date 1", "Date 1 User Rating", "Date 2", "Date 2 User Rating", "Date 3", and "Date 3 User Rating" represent the target user's user rating (i.e., evaluation results) over three consecutive days. Since the user rating was substandard for two days, the target user can be determined to be a warning user.

[0102] It should be noted that after the device identifies the user who is in the early warning, it can also determine the cause of the abnormality based on the user's abnormal voice perception key indicators and output the corresponding solution. The device can also display the user's information and the solution to the administrator through the front-end device so that the administrator can optimize the user's voice perception.

[0103] Therefore, as Figure 3 As shown, following S304, this application embodiment provides a method for determining early warning users for 5G voice services, which further includes the following S305-S306:

[0104] S305. The device inputs key indicators of voice perception into the voice service analysis model to obtain a solution.

[0105] Among them, the voice service analysis model is used to analyze the causes of anomalies based on key voice perception indicators and output corresponding solutions based on the causes of anomalies.

[0106] In some embodiments, the determining device determines the cause of the anomaly based on the abnormal voice perception key indicators of the warning user, and further determines the solution. The determining device inputs the voice perception key indicators of the warning user, the corresponding cause of the anomaly, and the solution into an experience base. The determining device establishes a voice service analysis model and performs machine learning based on the experience base, so that the determining device inputs the voice perception key indicators of the warning user into the voice service analysis model, the voice service analysis model analyzes the cause of the anomaly from multiple dimensions based on the abnormal voice perception key indicators, and outputs the corresponding solution based on the cause of the anomaly. The abnormal voice perception key indicators are used to characterize voice perception key indicators with a perception rating of "substandard". There can be multiple causes of anomalies and corresponding solutions. As an example, the format of the experience base of the voice service analysis model is shown in Table 4 below.

[0107] Table 4

[0108] province prefecture-level cities user Indicator Name threshold Current value Abnormal situation First failure characteristics Second failure characteristics Third failure characteristics Locating the root cause Demarcation Specialty Suggested solutions

[0109] Table 4 includes "Time", "Province", "City", "User", "Indicator Name", "Threshold", "Current Value", "Abnormal Situation", "First Failure Feature", "Second Failure Feature", "Third Failure Feature", "Location Reason", "Boundary Specialty", and "Suggested Solution". "Time" indicates the time when the target user was identified as a warning user. "Province", "City", and "User" represent the basic information of the warning user. "Indicator Name" indicates the name of the abnormal parameter among the key voice perception indicators of the warning user. "Threshold" indicates the preset condition for the abnormal key voice perception indicator. "Current Value" indicates the current value of the abnormal key voice perception indicator. "Abnormal Situation" indicates the abnormal situation of the key voice perception indicator of the warning user. "First Failure Feature", "Second Failure Feature", and "Third Failure Feature" represent multiple abnormal reasons for the warning user. "Location Root Cause" indicates the most critical abnormal reason among the multiple abnormal reasons for the warning user. "Boundary Specialty" indicates the business scope to which the location root cause belongs. "Suggested Solution" indicates the suggested solution for the warning user determined based on the location root cause and the boundary specialty.

[0110] S306. Determine if the device optimizes the user's voice perception based on the solution.

[0111] In some embodiments, the determining device presents the solution output by the voice service analysis model and the basic information of the warning user on the front-end device, so that the administrator can optimize the voice perception of the warning user according to the solution.

[0112] For example, the device will present the solution and warning user information in the customer service system. Customer service representatives can use different channels such as telephone calls, SMS messages, and mobile app to provide voice-based care to warning users, thereby improving their user experience.

[0113] The technical solutions provided by the above embodiments bring at least the following beneficial effects. The 5G voice service early warning user determination method provided in this application embodiment allows the determining device to determine key voice perception indicators for each preset period based on the acquired voice service data of the target user within each preset period. Then, it determines the evaluation result of the target user's voice perception quality based on the key voice perception indicators, and further determines whether the target user is an early warning user based on the evaluation results within multiple preset periods. This application achieves intelligent identification of early warning users with poor automatic network perception by analyzing voice service data, enabling administrators to optimize the voice perception of early warning users.

[0114] Furthermore, the key voice perception indicators determined by the determining device include VONR call average latency and VONR RTP average voice MOS value. Compared with existing technologies, the key voice perception indicators in this embodiment are more comprehensive and can more accurately reflect the evaluation results of target users' voice services, thereby more accurately identifying users subject to 5G VONR voice service warnings. In addition, the determining device can also input the key voice perception indicators of warning users into the voice service analysis model. The voice service analysis model analyzes the data and automatically outputs the causes of anomalies and solutions, enabling managers to optimize the voice perception of warning users based on the solutions, thus achieving intelligent management of 5G voice service users.

[0115] The following describes a specific embodiment of the method for determining early warning users for 5G voice services according to this application. The specific implementation process of this method is as follows: Figure 4 As shown.

[0116] The device first identifies the city (target area) set by management for pre-analysis and then determines a list of all user numbers (target users) within the analysis area of ​​that city. The device collects XDR data from all users on a daily basis, cleans the data, removes bad data, and ensures data usability. Based on the XDR data and the calculation method for key speech perception indicators, the device determines and statistically analyzes the key speech perception indicators for target users on a daily basis. It then compares these indicators with preset conditions to obtain an evaluation result of the target users' speech perception quality. Based on the evaluation results of the current day and the past two days, the device issues an early warning for deteriorating speech perception in target users (i.e., users whose speech perception quality evaluation results are substandard for two days or more are identified as warning users). The device inputs the early warning results into the business optimization system (i.e., inputs the warning user information and abnormal key speech perception indicators into the voice business analysis model), generates a business quality optimization solution, and provides pre-emptive care for customers who may file complaints based on the solution (i.e., optimizes the speech perception of warning users).

[0117] Figure 5 The system architecture diagram of this application embodiment includes: a data acquisition and distributed storage system 501, a data cleaning system 502, an application service system 503, a device management system 504, and a client-front-end presentation system 505.

[0118] The data acquisition and distributed storage system 501 is used to acquire and store the voice service data of the target user, and is applied in step S201 above. The data cleaning system 502 is used to clean the voice service data, removing useless data to ensure data availability, and is applied in step S201 above. The application service system 503 and device management system 504 are used to periodically monitor and statistically analyze key voice perception indicators, evaluate the voice perception quality of the target user, issue early warnings based on the voice perception quality of the target user, match business optimization solutions based on the early warning results, and provide proactive care for customer complaints, and are applied in steps S302-S306 above. The client-front-end presentation system 505 is used to present the information of the early warning user and the corresponding solutions on the front-end device for management personnel to view, and to optimize the voice perception of the early warning user, and is applied in step S306 above.

[0119] In an exemplary embodiment, this application also provides a device for determining users with early warning of 5G voice services. This device may include one or more functional modules for implementing the method for determining users with early warning of 5G voice services as described in the above embodiments.

[0120] For example, Figure 6 This is a schematic diagram of a 5G voice service early warning user identification device provided in an embodiment of this application. Figure 6 As shown, the 5G voice service early warning user determination device includes: an acquisition module 601 and a determination module 602.

[0121] The acquisition module 601 is used to acquire voice service data of the target user within each preset period, wherein the voice service data is used to characterize the voice perception status of the target user during the use of 5G VoNR. The determination module 602 is used to determine key voice perception indicators for each preset period based on the voice service data of the target user within each preset period, wherein the key voice perception indicators are used to characterize key parameters for analyzing the 5G VoNR voice perception quality of the target user. The determination module 602 is also used to determine the evaluation result of the target user's voice perception quality for each preset period based on the key voice perception indicators, wherein the evaluation result is used to characterize whether the target user's voice perception quality meets the standard within the preset period. The determination module 602 is also used to determine whether the target user is a warning user based on the evaluation results within multiple preset periods; warning users are users who need voice perception optimization.

[0122] In some embodiments, the key metrics for voice perception include at least one of the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and MOS value.

[0123] In other embodiments, the VONR initial registration success rate is used to characterize the ratio of the number of successful initial VONR registrations of a target user to the total number of initial VONR registrations within a preset period. The VONR originating network connection rate is used to characterize the ratio of the number of successful initial VONR network calls of a target user to the number of attempted initial VONR network calls within a preset period. The VONR ending network connection rate is used to characterize the ratio of the number of successful ending VONR network calls of a target user to the number of attempted ending VONR network calls within a preset period. The average VONR call latency is used to characterize the average time interval between the target user sending an invitation message and the called user receiving the invitation message within a preset period. The VONR dropped call rate is used to characterize the ratio of the number of dropped VONR calls of a target user to the number of VONR responses within a preset period. The MOS value is used to characterize the average perceived voice quality during a VONR call for a target user within a preset period.

[0124] In other embodiments, the determining module 602 is specifically used to determine, for each preset period, that the target user's speech perception quality meets the preset conditions if each parameter in the key speech perception indicators meets the preset conditions, with different parameters corresponding to different preset conditions; otherwise, determine that the target user's speech perception quality does not meet the preset conditions.

[0125] In other embodiments, the multiple preset periods are N consecutive preset periods. The determining module 602 is specifically used to determine the target user as a warning user if the evaluation results of the target user are not up to standard for M preset periods within N consecutive preset periods, where N is a positive integer greater than or equal to 2 and M is a positive integer less than N; or, if the evaluation results of the target user are not up to standard in each preset period within N consecutive preset periods, the target user is determined as a warning user.

[0126] In other embodiments, the user determination device for 5G voice services further includes a processing module 603.

[0127] The processing module 603 is specifically used to input key voice perception indicators into the voice service analysis model to obtain solutions. The voice service analysis model is used to analyze the causes of anomalies based on the key voice perception indicators and output corresponding solutions based on the causes of anomalies. The processing module 603 is also used to optimize the voice perception of users receiving early warnings based on the solutions.

[0128] In an exemplary embodiment, this application also provides an electronic device, which may be the 5G voice service early warning user determination device in the above method embodiments. Figure 7 This is a schematic diagram of the structure of a 5G voice service early warning user determination device provided in an embodiment of this application. Figure 7 As shown, the 5G voice service early warning user determination device may include: a processor 701 and a memory 702; the memory 702 stores instructions executable by the processor 701; when the processor 701 is configured to execute instructions, it causes the electronic device to implement the method described in the foregoing method embodiments.

[0129] In an exemplary embodiment, this application also provides a computer-readable storage medium storing computer program instructions thereon; when the computer program instructions are executed by a computer, the computer causes the computer to implement the method described in the foregoing embodiments. The computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0130] In an exemplary embodiment, this application also provides a computer program product that, when run on a computer, causes the computer to execute the aforementioned related method steps to implement the 5G voice service early warning user determination method in the above embodiments.

[0131] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining early warning users of 5G voice services using fifth-generation mobile communication technology, characterized in that, The method includes: Within multiple preset periods, voice service data of the target user is acquired in each preset period; the voice service data is used to characterize the voice perception of the target user during the process of using 5G New Radio to carry voice VONR; the voice service data includes external data representation (XDR) data of the target user in the target area within the preset period; the multiple preset periods are N consecutive preset periods; Based on the voice service data of the target user within each preset period, key voice perception indicators are determined for each preset period. These key voice perception indicators are used to characterize and analyze the key parameters of the 5G VONR voice perception quality of the target user. The key voice perception indicators include the following parameters: VONR initial registration success rate, VONR initial call network connection rate, VONR final call network connection rate, VONR average call latency, VONR call drop rate, and the average voice opinion score (MOS) value of the VONR real-time transport protocol RTP. For each preset period, if each parameter in the key indicators of speech perception meets the preset conditions, it is determined that the speech perception quality of the target user meets the standard within the preset period; different parameters correspond to different preset conditions. Otherwise, it is determined that the target user's voice perception quality is substandard within the preset period; If the evaluation results of the target user fail to meet the standards for M consecutive preset periods within the N consecutive preset periods, the target user is identified as a warning user; where N is a positive integer greater than or equal to 2; and M is a positive integer less than N. Alternatively, if the evaluation result of the target user fails to meet the standard in each of the N consecutive preset periods, the target user is identified as the warning user; the warning user is the user to be optimized for voice perception.

2. The method according to claim 1, characterized in that, The VONR initial registration success rate is used to characterize the ratio of the number of successful VONR initial registrations of the target user to the total number of VONR initial registrations within the preset period. The VONR originating network connection rate is used to characterize the ratio of the number of VONR originating network connections to the number of VONR originating network attempts for the target user within the preset period. The VONR final call network connection rate is used to characterize the ratio of the number of VONR final call network connections to the number of VONR final call attempts for the target user within the preset period. The VONR call average latency is used to characterize the average time interval between the target user sending the invitation message and the called user receiving the invitation message within the preset period. The VONR dropped call rate is used to characterize the ratio of the number of VONR dropped calls to the number of VONR answered calls for the target user within the preset period. The MOS value is used to characterize the average voice quality perceived by the target user during a VONR call within the preset period.

3. The method according to claim 1, characterized in that, The method further includes: The key indicators of voice perception are input into the voice service analysis model to obtain a solution; the voice service analysis model is used to analyze the causes of anomalies based on the key indicators of voice perception, and output the corresponding solutions based on the causes of anomalies. The solution described above optimizes the voice perception of the early warning user.

4. A device for identifying early warning users of 5G voice services, characterized in that, The device includes: an acquisition module and a determination module; The acquisition module is used to acquire voice service data of a target user within each preset period within a plurality of preset periods; the voice service data is used to characterize the voice perception of the target user during the use of 5G VONR; the voice service data includes external data representation (XDR) data of the target user within the target area within the preset period; the plurality of preset periods are N consecutive preset periods; The determining module is used to determine the key voice perception indicators for each preset period based on the voice service data of the target user within each preset period. The key voice perception indicators are used to characterize the key parameters for analyzing the 5G VONR voice perception quality of the target user. The key voice perception indicators include the following parameters: VONR initial registration success rate, VONR originating network connection rate, VONR terminating network connection rate, VONR average call latency, VONR call drop rate, and the average voice opinion score (MOS) value of the VONR real-time transport protocol RTP. The determining module is further configured to, for each preset period, determine that the voice perception quality of the target user meets the standard within the preset period if each parameter in the key voice perception indicators meets the preset conditions; different parameters correspond to different preset conditions; otherwise, determine that the voice perception quality of the target user does not meet the standard within the preset period. The determining module is further configured to, within the N consecutive preset periods, if the evaluation results of the target user fail to meet the standard for M preset periods, determine the target user as a warning user; where N is a positive integer greater than or equal to 2; and M is a positive integer less than N; or, within the N consecutive preset periods, if the evaluation results of the target user fail to meet the standard in each preset period, determine the target user as the warning user; the warning user is a user to be optimized for voice perception.

5. The 5G voice service early warning user identification device according to claim 4, characterized in that, The VONR initial registration success rate is used to characterize the ratio of the number of successful VONR initial registrations of the target user to the total number of VONR initial registrations within the preset period. The VONR originating network connection rate is used to characterize the ratio of the number of VONR originating network connections to the number of VONR originating network attempts for the target user within the preset period. The VONR final call network connection rate is characterized by the ratio of the number of VONR final call network connections to the number of VONR final call attempts for the target user within the preset period. The VONR call average latency is used to characterize the average time interval between the target user sending the invitation message and the called user receiving the invitation message within the preset period. The VONR dropped call rate is used to characterize the ratio of the number of VONR dropped calls to the number of VONR answered calls for the target user within the preset period. The MOS value is used to characterize the average voice quality perceived by the target user during a VONR call within the preset period.

6. The 5G voice service early warning user identification device according to claim 4, characterized in that, The device also includes a processing module; The processing module is used to input the key speech perception indicators into the speech service analysis model to obtain a solution; the speech service analysis model is used to analyze the causes of anomalies based on the key speech perception indicators and output the corresponding solutions based on the causes of anomalies. The processing module is also used to optimize the voice perception of the warning user based on the solution.

7. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory stores instructions that the processor can execute; When the processor is configured to execute the instructions, the electronic device performs the method as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: computer software instructions; When the computer software instructions are executed in an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-3.

Images