User perception prediction method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses a user perception prediction method and device, equipment, a medium and a product, and relates to the technical field of communication. According to the embodiment of the invention, the service layer data and the network layer data are combined, the associated sample KQI and sample KPI and the perception event prediction model trained by the sample perception event are utilized, the accuracy and the real-time performance of the data for predicting the user perception are ensured, and the accuracy and the real-time performance of the user perception prediction result are further ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and in particular to a user perception prediction method, device, equipment, medium and product. BACKGROUND

[0002] User perception is the subjective feeling of a user on the quality and performance of a device, network, required application and service. With the intensification of industry competition, focusing on user perception and improving user experience has become a powerful means for major operators to improve their competitiveness. Therefore, accurately and effectively predicting user perception on services is of great significance.

[0003] Currently, user perception is mainly predicted based on key quality indicators (KQI) or key experience indicators (KEI) of multimedia services, which has low accuracy and poor real-time performance.

[0004] Application Content

[0005] The embodiments of the present application provide a user perception prediction method, device, equipment, medium and product, which can solve the problems of low accuracy and poor real-time performance in predicting user perception in related technologies.

[0006] In a first aspect, the embodiments of the present application provide a user perception prediction method, comprising:

[0007] obtaining key quality indicators (KQI) of multimedia services and key performance indicators (KPI) of networks where the multimedia services are located within a preset time period;

[0008] inputting the KQI and the KPI into a perception event prediction model to output a perception event result, the perception event result comprising a perception anomaly event prediction result, the perception anomaly event prediction result being used to represent whether a perception anomaly event occurs; wherein the perception event prediction model is obtained by training sample KQI and sample KPI as input and sample perception events as output, and the sample KQI and the sample KPI are associated with the sample perception events;

[0009] in a case where it is determined according to the perception anomaly event prediction result that the perception anomaly event occurs, predicting user perception according to the number of perception anomaly events to obtain a prediction result.

[0010] In a second aspect, the embodiments of the present application provide a user perception prediction device, comprising:

[0011] an obtaining module configured to obtain key quality indicators (KQI) of multimedia services and key performance indicators (KPI) of networks where the multimedia services are located within a preset time period;

[0012] The prediction module inputs the KQI and the KPI into a perception event prediction model, and outputs a perception event result, wherein the perception event result is a perception anomaly event prediction result, and the perception anomaly event prediction result is used to represent whether a perception anomaly event occurs; the perception event prediction model is obtained by training sample KQI and sample KPI as input and sample perception events as output, and the sample KQI and the sample KPI are associated with the sample perception events; in a case where it is determined that the perception anomaly event occurs according to the perception anomaly event prediction result, user perception is predicted according to a number of perception anomaly events, and a prediction result is obtained.

[0013] In a third aspect, an electronic device is provided, including:

[0014] a processor;

[0015] a memory configured to store computer program instructions;

[0016] When the computer program instructions are executed by the processor, the method according to the first aspect is implemented.

[0017] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to the first aspect is implemented.

[0018] In a fifth aspect, a computer program product is provided, and the computer program product includes a computer program, and when the computer program is executed by a processor, the method according to the first aspect is implemented.

[0019] In the embodiments of the present application, the KQI and the KPI of the multimedia service in a preset time period and the KPI of the network in which the multimedia service is located are obtained; the KQI and the KPI are input into a perception event prediction model, and a perception event result is output, wherein the perception event result is a perception anomaly event prediction result, and the perception anomaly event prediction result is used to represent whether a perception anomaly event occurs; the perception event prediction model is obtained by training sample KQI and sample KPI as input and sample perception events as output, and the sample KQI and the sample KPI are associated with the sample perception events; in a case where it is determined that the perception anomaly event occurs according to the perception anomaly event prediction result, user perception is predicted according to a number of perception anomaly events, and a prediction result is obtained. That is, in the embodiments of the present application, the business layer data and the network layer data are combined, the perception event prediction model trained by the associated sample KQI and sample KPI and sample perception events is used, the accuracy and real-time performance of the data used for predicting user perception are ensured, and the accuracy and real-time performance of the user perception prediction result are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of a user perception prediction method provided in an embodiment of the present application;

[0022] Figure 2 A structural diagram of a perception event prediction model provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of another user perception prediction method provided in an embodiment of the present application;

[0024] Figure 4 A flowchart of another user perception prediction method provided in an embodiment of the present application;

[0025] Figure 5 A structural diagram of a user perception prediction device provided in an embodiment of the present application;

[0026] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0028] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0029] Before introducing the embodiments of the present application, the terms related to the embodiments of the present application are explained and described.

[0030] Key Quality Indicator, KQI for short, in the embodiments of the present application, KQI refers to a multimedia service quality index, for example, a multimedia service stall duration proportion.

[0031] Key Performance Indicator, KPI for short, in the embodiments of the present application, KPI refers to a network performance index, for example, a utilization rate of a cell physical resource block (Physical Resource Block, PRB), which reflects a capacity situation of a wireless resource.

[0032] Key Experience Indicator, KEI for short, KEI reflects a subjective feeling of a user when using a product service. In the embodiments of the present application, KEI can refer to a user experience of a multimedia service playing process, for example, whether a video stall situation is experienced.

[0033] Software Development Kit, SDK for short, in the embodiments of the present application, mainly refers to a soft probe function realized by a multimedia application program through built-in SDK code, and a user behavior, an abnormal event and the like of a user multimedia playing process are measured and reported.

[0034] HyperText Transfer Protocol, HTTP for short.

[0035] 4th Generation Mobile Communication, 4G for short, that is, Long Term Evolution (Long Term Evolution, LTE) technology or network.

[0036] 5th Generation Mobile Communication, 5G for short, which is a new generation of broadband mobile communication technology or network with characteristics of high rate, low delay and large connection.

[0037] X Detail Record, xDR for short, in the embodiments of the present application, xDR refers to signaling data generated by a collection and analysis device of an online log retention system, for example, xDR of a 4G S1-U or 5G N3 interface HTTP specific protocol, xDR of a video service composition and the like.

[0038] Serving Gateway, SGW for short, is a main core network element of a user plane of a 4G network.

[0039] User Plane Function, UPF for short, is a main core network element of a user plane of a 5G network.

[0040] Operation and Maintenance Center, OMC for short, usually refers to a management system of equipment, a network element, and a network.

[0041] User perception is a subjective feeling of a user on quality and performance of equipment, a network, and a required application and service. Accurate and effective prediction of user perception on service optimization is of great significance.

[0042] Currently, xDR based on multimedia service synthesis is mainly used to obtain KQI, and KQI is used to predict user perception; or, SDK built in multimedia service is used to obtain KEI from an application layer according to actual conditions of multimedia service, and KEI is used to predict user perception. The former has low accuracy, and the latter has poor real-time performance.

[0043] Therefore, an embodiment of the present application provides a user perception prediction method, device, equipment, medium, and product, which can solve the problem of low accuracy and poor real-time performance of related technologies in predicting user perception.

[0044] Figure 1 A flowchart of a user perception prediction method provided by an embodiment of the present application is shown in Figure 1 The user perception prediction method can include the following steps:

[0045] S110, KQI of multimedia service in a preset time period and KPI of a network in which the multimedia service is located are obtained.

[0046] S120, the KQI and the KPI are input into a perception event prediction model, and a perception event result is output.

[0047] The perception event result includes a perception abnormal event prediction result, and the perception abnormal event prediction result is used to represent whether a perception abnormal event occurs; the perception event prediction model is obtained by training sample KQI and sample KPI as input and sample perception events as output, and the sample KQI and the sample KPI are associated with the sample perception events.

[0048] S130, in a case where it is determined according to the perception abnormal event prediction result that the perception abnormal event occurs, user perception is predicted according to a number of the perception abnormal events, and a prediction result is obtained.

[0049] The embodiments of the present application combine service layer data and network layer data, utilize the associated sample KQI and sample KPI and the sample perception event trained perception event prediction model, ensure the accuracy and real-time performance of the data used for predicting user perception, and further ensure the accuracy and real-time performance of the user perception prediction result.

[0050] The above steps are described in detail as follows:

[0051] In S110, the start time point and the end time point of the preset time period can be set according to actual needs, for example, the preset time period can be 8:00-11:00 am, 2:00-5:00 pm, 7:00-9:00 pm, etc. The multimedia service may, for example, include at least one of a video service and an audio service.

[0052] The KQI is a key indicator at the service layer and can reflect the quality of different multimedia services. The KPI is a key indicator at the network layer and is usually an important parameter that can be monitored and measured at the network layer. In actual application, both the KQI and the KPI can have multiple indicators.

[0053] In the embodiments of the present application, for example, the xDR can be acquired, the multimedia service can be filtered according to the service type field of the xDR, and the KQI can be calculated according to the xDR of the multimedia service.

[0054] For example, the xDR can be divided into xDR based on a video composition service and xDR based on an HTTP protocol according to the service type field of the xDR.

[0055] For example, the KQI of the multimedia service at the application program can be determined according to the xDR of the video composition service, which may, for example, include but is not limited to the number of plays, the success rate of play, the initial buffering time, the proportion of the number of stalls, the proportion of the stall time, the download rate, the rate code rate ratio, etc. of the multimedia service in the preset time period.

[0056] For example, the KQI related to the HTTP protocol layer can be determined according to the xDR based on the HTTP protocol, which may, for example, include but is not limited to the total downlink traffic, the number of HTTP requests, the success rate of HTTP response, the HTTP response delay, the HTTP large package download rate, etc. of the multimedia service in the preset time period.

[0057] Exemplarily, the xDR according to the HTTP protocol can also determine a KQI related to a Transmission Control Protocol (TCP) layer, which can include, but is not limited to, a TCP connection success rate of multimedia service in a preset time period, a TCP connection delay, a TCP connection to multimedia service first response delay, a TCP uplink Round-Trip Time (RTT) delay, a TCP downlink RTT delay, a TCP uplink packet loss rate, a TCP downlink packet loss rate, a TCP uplink out-of-order rate, and a TCP downlink out-of-order rate.

[0058] The KPI of the network in which the multimedia service is located can include, but is not limited to, OMC performance indicators of 4G, 5G wireless cells, SGWs, and UPFs; fault alarm frequencies and fault alarm rates of 4G, 5G wireless cells, SGWs, and UPFs that affect the multimedia service; capacity indicators of core network elements, such as maximum utilization of a Protocol Data Unit (PDU) session; and capacity indicators of 4G, 5G wireless cells that affect the multimedia service, such as PRB utilization.

[0059] Embodiments of the present application combine KQIs at a service level and KPIs at a network level, and use data indicators of different dimensions to predict user perception, thereby ensuring the accuracy of the prediction result of user perception.

[0060] In S120, the perception event prediction model is used to predict whether a perception anomaly event occurs according to the KQI and the KPI. Embodiments of the present application do not limit the specific structure of the perception event prediction model, which can be, for example, a decision tree model such as a random forest model, a Light Gradient Boosting Machine (LightGBM) model, or the like.

[0061] Exemplarily, the perception event prediction model is trained by taking sample KQIs and sample KPIs as inputs and taking sample perception events as outputs, and the sample KQIs and the sample KPIs are associated with the sample perception events.

[0062] The sample perception event can be obtained according to user experience data obtained in real time during the multimedia service playing process. Embodiments of the present application train the perception event prediction model based on the associated KQI, KPI, and sample perception event, thereby ensuring the accuracy and real-time performance of the data used for predicting the prediction perception, reflecting the real user perception and service experience, and further ensuring the accuracy and real-time performance of the prediction result of user perception.

[0063] The structure and training process of the perception event prediction model can be referred to the following embodiments.

[0064] Exemplarily, the KQI and the KPI can be input into a pre-trained perception event prediction model, and a perception event result can be output, where the perception event result can include a perception abnormal event prediction result, and the perception abnormal event prediction result is used to represent whether a perception abnormal event occurs.

[0065] The perception abnormal event can include, for example, a play failure, a lag, a play initial buffering delay, and the like.

[0066] In S130, in actual application, a plurality of groups of KQI and KPI can be input into a pre-trained perception event prediction model, so as to obtain a plurality of perception abnormal event prediction results. According to each perception abnormal event prediction result, it can be determined whether the perception event corresponding to each group of KQI and KPI is a perception abnormal event.

[0067] In the case where it is determined that a perception abnormal event occurs, the number of perception abnormal events can be counted, and the user perception can be predicted according to the number of perception abnormal events, so as to obtain a prediction result.

[0068] Exemplarily, considering that the basic dimensions involved in predicting the user perception can be different, for example, the basic dimensions such as a cell, a network element, a content node, and the like can be involved, in order to accurately predict the user perception, exemplarily, the perception abnormal events can be divided according to a preset dimension, so as to obtain perception abnormal events corresponding to at least one dimension; for a first dimension in the at least one dimension, in the case where the number of perception abnormal events in the first dimension is greater than a reference number, it is determined that the user perception of the first dimension is degraded; and in the case where the number of perception abnormal events in the first dimension is less than or equal to the reference number, it is determined that the user perception of the first dimension is not degraded.

[0069] The preset dimension can be, for example, a cell, a network element, a content node, and the like, that is, the total perception abnormal events can be divided according to the dimension to which the perception abnormal events belong, so as to obtain perception abnormal events corresponding to each dimension.

[0070] For each dimension, the user perception of the user in the dimension can be predicted according to the number of perception abnormal events corresponding to the dimension, so as to obtain a prediction result.

[0071] The first dimension can be any one of the above dimensions, exemplarily, in the case where the number of perception abnormal events in the first dimension is greater than a reference number, it can be determined that the user perception of the first dimension is degraded, otherwise, it can be determined that the user perception of the first dimension is not degraded.

[0072] The reference quantity can be set according to actual needs, or can be determined according to a reference ratio of the number of the perception abnormal events in the first dimension to the total number of the perception events in the first dimension, and the total number of the perception events in the first dimension. For example, the reference quantity = the reference ratio * the total number of the perception events in the first dimension. The reference ratios of different dimensions can be different, and specific sizes can be set according to actual needs. For example, in some embodiments, the reference ratio of the cell dimension can be set to 0.5.

[0073] The embodiments of the present application can aggregate the number of perception abnormal events from different dimensions, and predict user perception based on the number of perception abnormal events aggregated from different dimensions, thereby realizing fine prediction of user perception and ensuring the accuracy of the prediction result.

[0074] In some embodiments, in the case where it is determined that the user perception is deteriorated, the user perception prediction method can further include the following steps:

[0075] The prediction result is prewarned in a preset manner.

[0076] The embodiments of the present application do not limit the prewarning manner, for example, one or more of the following manners can be used for prewarning: buzzer alarm, indicator light flickering, voice, telephone, short message, email, etc. For example, when prewarning by voice, telephone, short message, email, etc., in addition to pushing the prediction result of the user perception to the relevant personnel as user perception deterioration, the dimension of the user perception deterioration, the perception abnormal events involved, etc. can also be included, so as to facilitate the relevant personnel to quickly locate and solve the problem, and improve the user's perception as soon as possible, and improve the user's experience.

[0077] It can be understood that, in the training of the perception event prediction model, the embodiments of the present application need to associate the KQI, the KPI and the sample perception event, so as to ensure the accuracy and real-time performance of the user perception prediction result.

[0078] For example, before S110, the user perception prediction method can further include the following steps:

[0079] Obtaining candidate KQI, candidate KPI and sample perception event;

[0080] According to the association condition, determining the sample KQI and the sample KPI associated with the sample perception event from the candidate KQI and the candidate KPI;

[0081] The association condition includes at least one of the following: timestamp, user identity, domain name, and Internet protocol address of the server.

[0082] In the embodiments of the present application, the candidate KQI can be the KQI of the multimedia service in the historical time period, and similarly, the candidate KPI can be the KQI of the network in which the multimedia service is located in the historical time period. The sample perception event, also referred to as a sample KEI event, can be a user perception event in the historical time period, that is, each user perception event in the historical time period can be recorded as a sample KEI event. The sample KEI event can include a KEI abnormal event and a KEI non-abnormal event.

[0083] Exemplarily, the sample KEI event can be obtained according to the data reported by the SDK of single user and single multimedia service playing granularity, that is, soft probe data, in combination with the buffer type and the error code, the service behavior scene is screened, and then according to whether the multimedia service is successfully played, the first frame loading time, the playing jam frequency (times / hour), the playing error, the playing jam, the playing jam rate, the total number of playing jams and other index values in the service behavior scene, the sample KEI event is obtained. The sample KEI event obtained according to the soft probe data can reflect the perception event of the minimum user service granularity.

[0084] Exemplarily, for each sample KEI event, the candidate KQI and the candidate KPI can be associated with the sample KEI event according to the association condition, and the candidate KQI and the candidate KPI associated with the sample KEI event are recorded as a sample KQI and a sample KPI respectively.

[0085] Exemplarily, the association condition can include at least one of the following: a timestamp, a user identity, a domain name, an Internet Protocol Address (IP address) of a server.

[0086] For example, the candidate KQI, the candidate KPI and the sample KEI event with the same timestamp, the same user identity, the same domain name and the same IP address of the server can be associated. Exemplarily, each group of associated sample KQI, sample KPI and sample KEI event can be taken as a sample data.

[0087] It can be understood that data of different sources can have different data granularities, for example, one xDR of HTTP specific protocol represents one HTTP transaction interaction, and one multimedia service synthesis xDR represents one video playing process, which includes multiple HTTP transaction interactions. Therefore, when the KQI, the KPI and the sample KEI event are associated, the KQI, the KPI and the sample KEI event need to be unified to the same data granularity. For example, the unification can be performed to the multimedia service playing granularity first, and then the association is performed, so as to ensure the accuracy of the association result, and thus the training effect of the model can be improved.

[0088] The embodiment of the application associates the sample KQI, the sample KPI and the sample KEI event by associating conditions, and takes the associated sample KQI and sample KPI as input and the sample KEI event as output to train the perception event prediction model, thereby ensuring the accuracy and real-time performance of the data used for predicting user perception, and further ensuring the accuracy and real-time performance of the user perception prediction result.

[0089] In some embodiments, as shown in Figure 2 The perception event prediction model 200 includes a perception abnormal event prediction module 201 and an influence degree prediction module 202, wherein the perception abnormal event prediction module 201 is configured to predict the KEI abnormal event, and the influence degree prediction module 202 is configured to predict the influence degree of the KQI and KPI on the KEI event. Accordingly, the perception event result further includes the influence degree of the KQI and KPI on the KEI event.

[0090] Exemplarily, as shown in Figure 3 The user perception prediction method can include the following steps:

[0091] S310, acquiring KQI of a multimedia service and KPI of a network in which the multimedia service is located in a preset time period.

[0092] S320, inputting the KQI and KPI into a perception abnormal event prediction module to output a perception abnormal event prediction result.

[0093] S330, inputting the KQI, KPI and perception abnormal event prediction result into an influence degree prediction module to output the influence degree of the KQI and KPI on a perception event corresponding to the perception abnormal event prediction result.

[0094] S340, in a case where it is determined according to the perception abnormal event prediction result that a perception abnormal event occurs, predicting user perception according to the number of perception abnormal events to obtain a prediction result.

[0095] The processes of S310 and S340 can be referred to the above embodiments, and will not be described herein for brevity.

[0096] The other steps will be described in detail as follows:

[0097] In S320, the perception abnormal event prediction module takes the KQI and KPI as input and outputs a KEI abnormal event prediction result. The KEI abnormal event prediction result can include a KEI abnormal event or a KEI non-abnormal event. Exemplarily, the KEI non-abnormal event can be represented by “0”, and different KEI abnormal events can be represented by different identifiers, such as “1” representing a playing failure, “2” representing a playing lag, “3” representing a playing initial buffering delay, etc. Of course, other identifiers can also be used, and the embodiment of the application does not make specific limitations.

[0098] The service layer index and the network layer index acquired within the preset time period are input into the anomaly event prediction module, so as to predict whether the KEI anomaly event exists within the preset time period.

[0099] In S330, the influence degree prediction module takes the KQI and the KPI and the output of the anomaly event prediction module, i.e., the anomaly event prediction result, as input, and takes the influence degree of the KQI and the KPI on the KEI event as output.

[0100] For example, for a certain KEI anomaly event, the influence degree of the KQI and the KPI on the KEI anomaly event can be obtained by inputting the KQI, the KPI and the KEI anomaly event into the influence degree prediction module.

[0101] The influence degree is used to represent the influence degree of the KQI and the KPI on the KEI anomaly event. The greater the influence degree, the greater the influence degree of a certain index on the KEI anomaly event. According to the influence degree, the index with greater influence on the KEI anomaly event can be quickly determined, and the problem index can be quickly located, so as to improve the user perception as soon as possible.

[0102] For example, for each KEI event, the entropy weight method or other algorithms can be used to calculate the influence degree of the associated KQI and KPI on the KIE event.

[0103] The embodiments of the present application can not only predict the perception anomaly event by using the perception event prediction model, but also further determine the influence degree of the associated KQI and KPI on each KEI event, which helps to quickly locate the problem KQI and the problem KPI.

[0104] For example, the KQI and the KPI both include multiple KQIs and KPIs, and in the case of determining the occurrence of the perception anomaly event according to the perception anomaly event prediction result, as shown in FIG. 5, the user perception prediction method can include the following steps: Figure 4

[0105] S410, acquiring the KQI of the multimedia service and the KPI of the network where the multimedia service is located within a preset time period.

[0106] S420, inputting the KQI and the KPI into the anomaly event prediction module, and outputting the anomaly event prediction result.

[0107] S430, inputting the KQI, the KPI and the anomaly event prediction result into the influence degree prediction module, and outputting the influence degree of the KQI and the KPI on the perception event corresponding to the anomaly event prediction result.

[0108] S440, predicting the user perception according to the number of the anomaly events, and obtaining the prediction result.

[0109] ​S450, determining a first deviation degree of each KQI and a second deviation degree of each KPI.

[0110] The first deviation degree represents a degree of deterioration of each KQI relative to a reference KQI, and the second deviation degree represents a degree of deterioration of each KPI relative to a reference KPI.

[0111] S460, determining a first weight of each KQI according to the first deviation degree of each KQI and a first influence degree of each KQI on the KEI abnormal event, and determining a second weight of each KPI according to the second deviation degree of each KPI and a second influence degree of each KPI on the KEI abnormal event.

[0112] S470, determining a target KQI affecting the KEI abnormal event according to the first weight of each KQI, and determining a target KPI affecting the KEI abnormal event according to the second weight of each KPI.

[0113] The processes of S410-S440 can refer to the above embodiments, which will not be described herein for brevity.

[0114] The other steps will be described in detail as follows:

[0115] In S450, the first deviation degree represents a degree of deterioration of each KQI relative to a reference KQI, and the second deviation degree represents a degree of deterioration of each KPI relative to a reference KPI.

[0116] Similarly, the second deviation degree represents a degree of deterioration of each KPI relative to a reference KPI, and the second deviation degree represents a degree of deterioration of each KPI relative to a reference KPI.

[0117] The greater the deviation degree, the greater the degree of deterioration of the corresponding indicator, that is, the greater the degree of deviation from the reference indicator. For example, the greater the first deviation degree, the greater the degree of deterioration of the KQI, that is, the greater the gap between the KQI and the reference KQI.

[0118] In S460, the first weight of each KQI can be determined according to the first deviation degree of each KQI and the first influence degree of each KQI on the KEI abnormal event.

[0119] Exemplarily, a first product value of the first deviation degree of each KQI and the first influence degree of the KQI on the KEI abnormal event can be determined, and the first product value is determined as the first weight of each KQI. That is, α i = β i * γ im , where α i represents the first weight of the i-th KQI, β i represents the first deviation degree of the i-th KQI, γ im represents the first influence degree of the i-th KQI on the m-th KEI abnormal event, and m and i are both integers greater than or equal to 1.

[0120] According to the second deviation degree of each KPI and the second influence degree of each KPI on the perception abnormal event, the second weight of each KPI can be determined.

[0121] Exemplarily, a second product value of the second deviation degree of each KPI and the second influence degree of the KPI on the KEI abnormal event can be determined, and the second product value is determined as the second weight of each KPI. That is, α j = β j * γ jm , where α j represents the second weight of the j-th KPI, β j represents the second deviation degree of the j-th KPI, γ jm represents the second influence degree of the j-th KPI on the m-th KEI abnormal event, and j is an integer greater than or equal to 1.

[0122] For each KQI, the first weight of each KQI can be obtained by the above manner, and similarly, for each KPI, the second weight of each KPI can be obtained by the above manner, which provides a basis for subsequent bounding.

[0123] In S470, according to the first weight of each KQI, a target KQI affecting the KEI abnormal event can be determined; and according to the second weight of each KPI, a target KPI affecting the KEI abnormal event can be determined.

[0124] Exemplarily, the first weights of the KQIs can be arranged in descending order, and the second weights of the KPIs can be arranged in descending order.

[0125] Exemplarily, the first N KQIs can be determined as the target KQIs, and the first M KPIs can be determined as the target KPIs. Exemplarily, N = M = 3, of course, N and M can also be other integers, and the embodiments of the present application are not limited specifically.

[0126] The target KQI and the target KPI are the indicators that have a relatively great influence on a certain KEI abnormal event.

[0127] It should be noted that in actual application, the execution sequence of S440 and S450-S470 is not limited in the embodiments of the present application, for example, S440 can be executed first, and then S450-S470 can be executed; or S450-S470 can be executed first, and then S440 can be executed; or S440 and S450-S470 can be executed simultaneously.

[0128] The embodiments of the present application calculate the weight of each KQI and KPI according to the deviation degree and the influence degree, and realize the brightness of the deviation degree and the influence degree. Then, the target KQI and the target KPI that have a greater influence on each KEI abnormal event are determined from each KQI and each KPI according to the weight, which guarantees the objectivity and accuracy of the bounding result, so that the relevant personnel can quickly locate the problem KQI and the problem KPI, improve the user's perception as soon as possible, and optimize the service.

[0129] Exemplarily, after the target KQI and the target KPI are determined, other ways can be further combined to locate to a specific cell, a network element, a content node, etc.

[0130] Exemplarily, for the determined target KQI, error codes, segmentation, clustering, horizontal comparison, etc. can be further used to determine the degraded KQI from the target KQI and the specific network nodes corresponding to the degraded KQI, such as specific cells, SGW network elements, etc.

[0131] Exemplarily, for the determined target KPI, the topology information can be obtained from the xDR data according to the type of the network node associated with the target KPI, and then the specific network node can be located. For example, the specific cell with degraded PRB utilization rate and the specific SGW network element generating an alarm can be determined.

[0132] The embodiments of the present application combine the data associated with the service level, the data associated with the network level, and the soft probe data, and use a supervised learning method to train a perception event prediction model, which guarantees the accuracy and real-time performance of the data used for predicting the user perception, so that the real user perception and service experience can be reflected, and the real-time requirement of the perception degradation identification and early warning can be met. Meanwhile, the embodiments of the present application can further determine the influence degree of the KQI and the KPI on the KEI abnormal event by using the perception event prediction model, and determine the weight of the KQI and the KPI by combining the deviation degree of the KQI and the KPI, and then perform bounding according to the weight, which guarantees the objectivity and accuracy of the bounding result.

[0133] Based on the same inventive concept, the embodiments of the present application also provide a user perception prediction device, which will be described in detail below. Figure 5 The user perception prediction device provided by the embodiments of the present application will be described in detail.

[0134] Figure 5 A structural diagram of a user perception prediction device provided for an embodiment of the present application.

[0135] As shown in the figure, the user perception prediction device 500 can include: Figure 5

[0136] The acquisition module 501 is configured to acquire key quality indicators KQI of multimedia services and key performance indicators KPI of networks in which the multimedia services are located within a preset time period.

[0137] The prediction module 502 is configured to input the KQI and the KPI into a perception event prediction model, and output a perception event result, the perception event result including a perception anomaly event prediction result, the perception anomaly event prediction result being used to represent whether a perception anomaly event occurs; wherein the perception event prediction model is obtained by training with sample KQI and sample KPI as input and sample perception events as output, the sample KQI and the sample KPI being associated with the sample perception events; in a case where it is determined according to the perception anomaly event prediction result that the perception anomaly event occurs, the user perception is predicted according to a number of the perception anomaly events, and a prediction result is obtained.

[0138] In the embodiment of the present application, the service layer data and the network layer data are combined, the perception event prediction model trained with the associated sample KQI and sample KPI and sample perception events is used, and the accuracy and real-time performance of the data used for predicting the user perception are ensured, and thus the accuracy and real-time performance of the user perception prediction result are ensured.

[0139] In some embodiments, the acquisition module 501 is further configured to, before acquiring the KQI of the multimedia services and the KPI of the networks in which the multimedia services are located within the preset time period, acquire candidate KQI, candidate KPI and sample perception events.

[0140] The user perception prediction device 500 can further include:

[0141] The determination module is configured to determine, according to an association condition, sample KQI and sample KPI associated with the sample perception events from the candidate KQI and the candidate KPI.

[0142] The association condition includes at least one of the following: a same time stamp, a same user identity, a same domain name and a same Internet Protocol address of a server.

[0143] In some embodiments, the prediction module 502 is specifically configured to:

[0144] divide the perception anomaly events according to a preset dimension, and obtain perception anomaly events corresponding to at least one dimension;

[0145] ​For a first dimension in the at least one dimension, in a case where a number of the perception abnormal events in the first dimension is greater than a reference number, determining that user perception in the first dimension is degraded; in a case where the number of the perception abnormal events in the first dimension is less than or equal to the reference number, determining that user perception in the first dimension is not degraded.

[0146] In some embodiments, the perception event prediction model comprises a perception abnormal event prediction module and an influence degree prediction module, the perception event result further comprises an influence degree of the KQI and the KPI on the perception event, and the perception event comprises a perception abnormal event and a perception non-abnormal event.

[0147] The prediction module 502 is specifically configured to:

[0148] input the KQI and the KPI into the perception abnormal event prediction module, and output a perception abnormal event prediction result;

[0149] input the KQI, the KPI and the perception abnormal event prediction result into the influence degree prediction module, and output an influence degree of the KQI and the KPI on a perception event corresponding to the perception abnormal event prediction result.

[0150] In some embodiments, the KQI and the KPI each comprise a plurality of KQIs and KPIs.

[0151] The determination module is further configured to, in a case where it is determined according to the perception abnormal event prediction result that the perception abnormal event occurs, determine a first deviation degree of each KQI and a second deviation degree of each KPI, the first deviation degree representing a degradation degree of each KQI relative to a reference KQI, and the second deviation degree representing a degradation degree of each KPI relative to a reference KPI.

[0152] determine a first weight of each KQI according to the first deviation degree of each KQI and a first influence degree of each KQI on the perception abnormal event, and determine a second weight of each KPI according to the second deviation degree of each KPI and a second influence degree of each KPI on the perception abnormal event.

[0153] determine a target KQI affecting the perception abnormal event according to the first weight of each KQI, and determine a target KPI affecting the perception abnormal event according to the second weight of each KPI.

[0154] In some embodiments, the determination module is specifically configured to:

[0155] determine a first product value of the first deviation degree and the corresponding first influence degree of each KQI, and determine the first product value as the first weight of each KQI.

[0156] In some embodiments, the determination module is specifically configured to:

[0157] A second product value of the second deviation degree and the corresponding second influence degree of each KPI is determined, and the second product value is determined as a second weight of each KPI.

[0158] In some embodiments, the user perception prediction device 500 may further include:

[0159] The early warning module is used to predict user perception based on the number of abnormal perception events when the prediction result of the prediction module 502 is user perception degradation, and after obtaining the prediction result, issue an early warning for the prediction result in a preset manner.

[0160] Figure 5 Each module in the device shown has the function of realizing Figures 1-4 The functions of each step in the process can achieve the corresponding technical effects, so for the sake of brevity, they will not be described here in detail.

[0161] Based on the same inventive concept, the present application embodiment also provides an electronic device. Figure 6 The electronic device provided in the embodiments of the present application is described in detail.

[0162] like Figure 6 As shown, the electronic device 600 may include a processor 610 and a memory 620 for storing computer program instructions.

[0163] The processor 610 may include a central processing unit (CPU) or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0164] The memory 620 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In one example, the memory 620 may include a removable or non-removable (or fixed) medium, or the memory 620 may be a non-volatile solid-state memory. In one example, the memory 620 may be a read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0165] The processor 610 implements the method in the embodiments of the present application by reading and executing computer program instructions stored in the memory 620, and achieves the corresponding technical effects of the embodiments of the present application. Figures 1-4 The processor 610 implements the method in the embodiments of the present application by reading and executing computer program instructions stored in the memory 620, and achieves the corresponding technical effects of the embodiments of the present application. Figures 1-4 The processor 610 implements the method in the embodiments of the present application by reading and executing computer program instructions stored in the memory 620, and achieves the corresponding technical effects of the embodiments of the present application.

[0166] In one example, the electronic device 600 can further include a communication interface 630 and a bus 640. As shown in the figure, the processor 610, the memory 620, and the communication interface 630 are connected through the bus 640 and complete communication with each other. Figure 6

[0167] The communication interface 630 is mainly used to realize the communication between various modules, devices and / or equipment in the embodiments of the present application.

[0168] The bus 640 includes hardware, software or both to couple the components of the electronic device 600 to each other. By way of example, and without limitation, the bus 640 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, the bus 640 can include one or more buses. Although the present embodiments describe and show a particular bus, the present application contemplates any suitable bus or interconnect.

[0169] The electronic device can execute the user perception prediction method in the embodiments of the present application after obtaining the key quality indicator KQI of the multimedia service and the key performance indicator KPI of the network where the multimedia service is located within a preset time period, thereby realizing the user perception prediction method described in the embodiments of the present application and the user perception prediction device described in the embodiments of the present application. Figures 1-4 The electronic device can execute the user perception prediction method in the embodiments of the present application after obtaining the key quality indicator KQI of the multimedia service and the key performance indicator KPI of the network where the multimedia service is located within a preset time period, thereby realizing the user perception prediction method described in the present application and the user perception prediction device described in the present application. Figure 5 The electronic device can execute the user perception prediction method in the embodiments of the present application after obtaining the key quality indicator KQI of the multimedia service and the key performance indicator KPI of the network where the multimedia service is located within a preset time period, thereby realizing the user perception prediction method described in the present application and the user perception prediction device described in the present application.

[0170] ​In addition, in combination with the user perception prediction method in the above embodiments, an embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement any one of the user perception prediction methods in the above embodiments.

[0171] In addition, an embodiment of the present application further provides a computer program product, comprising a computer program, which is executed by at least one processor to implement various processes of the above user perception prediction method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0172] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted herein. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0173] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of the machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0174] It also needs to be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be executed simultaneously.

[0175] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0176] The above describes only specific implementation of the present application. For the convenience and brevity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described herein. It should be understood that the protection scope of the present application is not limited in this way. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A user perception prediction method, characterized by, The method comprises the following steps: obtaining key quality indicators (KQIs) of multimedia services and key performance indicators (KPIs) of networks in which the multimedia services are located within a preset time period; inputting the KQIs and the KPIs into a perception event prediction model to output a perception event result, wherein the perception event result comprises a perception abnormal event prediction result, and the perception abnormal event prediction result is used to represent whether a perception abnormal event occurs; the perception event prediction model is trained by taking sample KQIs and sample KPIs as inputs and taking a sample perception event as an output, and the sample KQIs and the sample KPIs are associated with the sample perception event; in a case where it is determined according to the perception abnormal event prediction result that the perception abnormal event occurs, predicting user perception according to a number of the perception abnormal events to obtain a prediction result.

2. The method of claim 1, wherein, Before the step of obtaining the KQIs of the multimedia services and the KPIs of the networks in which the multimedia services are located within the preset time period, the method further comprises the following steps: obtaining candidate KQIs, candidate KPIs, and a sample perception event; determining, according to an association condition, sample KQIs and sample KPIs associated with the sample perception event from the candidate KQIs and the candidate KPIs; the association condition comprises at least one of the following: a same time stamp, a same user identity, a same domain name, and a same Internet Protocol (IP) address of a server.

3. The method of claim 1, wherein, The step of predicting the user perception according to the number of the perception abnormal events to obtain the prediction result comprises the following steps: dividing the perception abnormal events according to a preset dimension to obtain perception abnormal events corresponding to at least one dimension; for a first dimension in the at least one dimension, in a case where a number of the perception abnormal events in the first dimension is greater than a reference number, determining that user perception in the first dimension is degraded; and in a case where the number of the perception abnormal events in the first dimension is less than or equal to the reference number, determining that the user perception in the first dimension is not degraded.

4. The method of claim 1, wherein, The perception event prediction model comprises a perception abnormal event prediction module and an influence degree prediction module, the perception event result further comprises an influence degree of the KQIs and the KPIs on a perception event, and the perception event comprises the perception abnormal event and a perception non-abnormal event; the step of inputting the KQIs and the KPIs into the perception event prediction model to output the perception event result comprises the following steps: inputting the KQIs and the KPIs into the perception abnormal event prediction module to output the perception abnormal event prediction result; inputting the KQIs, the KPIs, and the perception abnormal event prediction result into the influence degree prediction module to output an influence degree of the KQIs and the KPIs on a perception event corresponding to the perception abnormal event prediction result.

5. The method of claim 4, wherein, The KQIs and the KPIs each comprise a plurality of KQIs and KPIs; in a case where it is determined according to the perception abnormal event prediction result that the perception abnormal event occurs, the method further comprises the following steps: determining a first deviation degree of each of the KQIs and a second deviation degree of each of the KPIs, wherein the first deviation degree represents a degradation degree of each of the KQIs relative to a reference KQI, and the second deviation degree represents a degradation degree of each of the KPIs relative to a reference KPI. determine a first weight of each KQI according to the first deviation degree of each KQI and a first influence degree of each KQI on the perceived abnormal event; and determine a second weight of each KPI according to the second deviation degree of each KPI and a second influence degree of each KPI on the perceived abnormal event; determine a target KQI affecting the perceived abnormal event according to the first weight of each KQI; and determine a target KPI affecting the perceived abnormal event according to the second weight of each KPI.

6. The method of claim 5, wherein, The determining of the first weight of each KQI according to the first deviation degree of each KQI and the first influence degree of each KQI on the perceived abnormal event comprises: determining a first product value of the first deviation degree and the corresponding first influence degree of each KQI, and determining the first product value as the first weight of each KQI; The determining of the second weight of each KPI according to the second deviation degree of each KPI and the second influence degree of each KPI on the perceived abnormal event comprises: determining a second product value of the second deviation degree and the corresponding second influence degree of each KPI, and determining the second product value as the second weight of each KPI.

7. The method according to any one of claims 1 to 6, characterized in that, In a case where the prediction result is user perception degradation, after the predicting of the user perception according to the number of the perceived abnormal events to obtain the prediction result, the method further comprises: warning the prediction result in a preset manner.

8. A user perception prediction apparatus characterized by comprising: comprise: an acquisition module, configured to acquire key quality indicators (KQIs) of multimedia services and key performance indicators (KPIs) of a network in which the multimedia services are located in a preset time period; a prediction module, configured to input the KQIs and the KPIs into a perceived event prediction model, and output a perceived event result, the perceived event result comprising a perceived abnormal event prediction result, the perceived abnormal event prediction result being used to represent whether a perceived abnormal event occurs; wherein the perceived event prediction model is obtained by training sample KQIs and sample KPIs as inputs and sample perceived events as an output, the sample KQIs and the sample KPIs being associated with the sample perceived events; in a case where it is determined according to the perceived abnormal event prediction result that the perceived abnormal event occurs, the prediction module is further configured to predict user perception according to a number of the perceived abnormal events to obtain a prediction result.

9. An electronic device, comprising: comprise: a processor; a memory, configured to store computer program instructions; when the computer program instructions are executed by the processor, the method in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, when the computer program instructions are executed by the processor, the method in any one of claims 1-7 is implemented.

11. A computer program product, characterised in that, comprise a computer program, when the computer program is executed by the processor, the method in any one of claims 1-7 is implemented.

Images