Method, apparatus and device for evaluating users based on specific service

Abstract

The application provides a user evaluation method, device and equipment based on specific services. In the user evaluation method, specific service user groups are located by identifying specific services from mobile network access records, and the specific user groups are comprehensively evaluated in combination with wireless base station operation logs. The traditional passive index comprehensive evaluation of mobile wireless networks is changed to specific user-based wireless base station index evaluation, and the service type can be arbitrarily expanded. The required information is extracted in an uncertain environment, one step ahead, to make up for the short board of traditional perception evaluation, and to meet the demand for directional service perception evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication technology, in particular to a user evaluation method, device and equipment based on specific services. BACKGROUND

[0002] In today's network society, having users means having the market, and all industries are actively seizing the "Internet +" era of user entry. Analysis and evaluation of user perception are currently common and important research content.

[0003] In the prior art, the network quality is usually evaluated by using a single dimension of traditional network key performance indicators (KPI). The indicators involved include coverage indicators, call indicators, mobility management indicators, and quality indicators. For example, the call indicators include radio resource control (RRC) connection establishment success rate, radio access bearer (RAB) establishment success rate, wireless access rate, voice service wireless access rate, video phone wireless access rate, packet switch (PS) domain wireless access rate, and wireless circuit domain wireless access rate. The network quality is evaluated by calculating and aggregating the above indicators.

[0004] However, the traditional network quality evaluation cannot meet the actual production needs. In the event of an emergency, specific services cannot be identified, and the required information cannot be extracted in an uncertain environment, which may cause network problems after the emergency. SUMMARY

[0005] The present application provides a user evaluation method, device and equipment based on specific services to solve the problem that the traditional network key performance indicators cannot meet the specific service identification.

[0006] In a first aspect, the present application provides a user evaluation method based on specific services, comprising:

[0007] Generating an application detailed record XDR detailed list from the user information and location information reported by the base station, wherein each entry in the XDR detailed list includes user information, and source IP, destination IP, source port, destination port, and protocol type of the service accessed by the user.

[0008] Filtering detailed list information of a plurality of users corresponding to each type of service from the XDR detailed list;

[0009] Calculating the key performance indicators of each type of service based on the detailed list information of a plurality of users corresponding to each type of service;

[0010] obtaining a score of each service type based on a preset scoring rule of each type of service and a key performance indicator of each service type;

[0011] calculating a comprehensive perception evaluation result according to a preset evaluation factor of each service type and the score of each service type.

[0012] In combination with the first aspect, in some embodiments, the method further comprises:

[0013] determining the key performance indicator according to the base station information and the base station operation history in a time dimension, the key performance indicator including a key performance KPI indicator and a timely service performance KQI indicator;

[0014] The KPI indicator includes at least one of access performance, maintenance capability, mobile performance, service quality and perception performance; and the KQI indicator includes at least one of webpage latency, instant messaging latency, display latency, video latency, download rate and nucleic acid code latency.

[0015] In combination with the first aspect, in some embodiments, the evaluation factor of each service type includes a 4G data evaluation factor, a timely service evaluation factor and a perception rate evaluation factor.

[0016] In combination with the first aspect, in some embodiments, the obtaining of the score of each service type based on the preset scoring rule of each type of service and the key performance indicator of each service type includes:

[0017] for the 4G data service, the score of the 4G data service is calculated according to 4G connection rate*(1-4G drop rate)*intra-frequency handover success rate*inter-frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%);

[0018] for the timely service, the score of the timely service is calculated according to Min[100%-(webpage latency indicator value / 120-1), 100%]*Min[100%-(instant messaging indicator value / 120-1), 100%]*Min[100%-(nucleic acid code latency indicator value / 120-1), 100%]*Min[100%-(video latency indicator value / 120-1), 100%];

[0019] for the perception rate, the score of the perception rate is calculated according to perception rate*2.5 / 100.

[0020] In combination with the first aspect, in some embodiments, the method further comprises:

[0021] According to the comprehensive perception evaluation result obtained at each time, a long trend change graph of the comprehensive perception evaluation result is generated;

[0022] Based on the long trend change graph and the scoring range of different levels, network health level division is performed.

[0023] In a second aspect, the application provides a user evaluation device based on specific services, comprising:

[0024] A data generation module is configured to generate an application detailed record XDR detailed list according to user information and location information reported by a base station, wherein each entry of the XDR detailed list includes user information, and source IP, destination IP, source port, destination port, and protocol type of a service accessed by the user;

[0025] A service filtering module is configured to filter detailed list information of a plurality of users corresponding to each type of service from the XDR detailed list;

[0026] A first calculation module is configured to calculate a key performance indicator of the type of service based on the detailed list information of the plurality of users corresponding to each type of service;

[0027] A second calculation module is configured to obtain a score of each type of service based on a preset scoring rule of each type of service and the key performance indicator of each type of service;

[0028] A third calculation module is configured to calculate a comprehensive perception evaluation result according to a preset evaluation factor of each type of service and the score of each type of service.

[0029] In combination with the second aspect, in some embodiments, the device further comprises:

[0030] An indicator acquisition module is configured to determine the key performance indicator according to base station information and base station operation history, and to aggregate and filter according to a time dimension, wherein the key performance indicator includes a key performance KPI indicator and a timely service performance KQI indicator;

[0031] The KPI indicator includes at least one of access performance, retention capability, mobility performance, service quality, and perception performance; and the KQI indicator includes at least one of webpage delay, instant messaging delay, display delay, video delay, download rate, and nucleic acid code delay.

[0032] In combination with the second aspect, in some embodiments, the evaluation factor of each type of service includes a 4G data evaluation factor, a timely service evaluation factor, and a perception rate evaluation factor.

[0033] In combination with the second aspect, in some embodiments, the second calculation module comprises:

[0034] a data service unit configured to calculate a score of a 4G data service according to a 4G access rate *(1-4G drop rate)*intra-frequency handover success rate*inter-frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%) for the 4G data service;

[0035] a real-time service unit configured to calculate a score of a real-time service according to Min[100%-(web page latency index value / 120-1), 100%]*Min[100%-(instant messaging index value / 120-1), 100%]*Min[100%-(nucleic acid code latency index value / 120-1), 100%]*Min[100%-(video latency index value / 120-1), 100%] for the real-time service;

[0036] a perception rate unit configured to calculate a score of a perception rate according to perception rate*2.5 / 100 for the perception rate.

[0037] With reference to the second aspect, in some embodiments, the third calculation module comprises:

[0038] a calculation unit configured to multiply the score of each service type by an evaluation factor of the corresponding service type and then accumulate the scores to obtain the comprehensive perception evaluation result.

[0039] With reference to the second aspect, in some embodiments, the apparatus further comprises:

[0040] an image generation module configured to generate a long trend change graph of the comprehensive perception evaluation result according to the comprehensive perception evaluation result obtained at each time point;

[0041] a grade division module configured to divide the network health grade based on the long trend change graph and a score range of different grades.

[0042] A third aspect of the present application further provides an electronic device, comprising a processor, and a memory and a display connected to the processor in communication;

[0043] the memory stores computer execution instructions;

[0044] the processor executes the computer execution instructions stored in the memory to implement the method of the first aspect.

[0045] A fourth aspect of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method of the first aspect.

[0046] The application provides a user evaluation method, device and equipment based on specific services. For mobile network user evaluation, network perception guarantee of user aggregation area is completed through fast service identification, and network resources are flexibly allocated. Different from the prior art, the scheme identifies specific services through mobile network access records, locates mobile base station positions occupied by specific service user groups, and performs service comprehensive evaluation on specific user groups in combination with wireless base station operation logs. The traditional mobile wireless network passive index comprehensive evaluation is changed to wireless base station index evaluation based on specific users, and the service type can be arbitrarily expanded. Through specific service identification, the traditional base station perception evaluation method is integrated, the required information can be refined in an uncertain environment, the information is obtained one step ahead, the short board of the traditional perception evaluation is made up, and the directional service perception evaluation requirement is realized. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0048] Figure 1 An application scenario diagram of the user evaluation method based on specific services provided by the embodiments of the application is shown.

[0049] Figure 2 A flowchart of the user evaluation method based on specific services provided by the first embodiment of the application is shown.

[0050] Figure 3 A flowchart of the user evaluation method based on specific services provided by the second embodiment of the application is shown.

[0051] Figure 4 A flowchart of the user evaluation method based on specific services provided by the third embodiment of the application is shown.

[0052] Figure 5 A flowchart of the user evaluation method based on specific services provided by the fourth embodiment of the application is shown.

[0053] Figure 6 A flowchart of the comprehensive perception evaluation provided by the embodiments of the application is shown.

[0054] Figure 7 A flowchart of the LTE base station ID acquisition provided by the embodiments of the application is shown.

[0055] Figure 8 A flowchart of the LTE base station cell ID acquisition provided by the embodiments of the application is shown.

[0056] Figure 9 A flowchart of the LTE user service acquisition provided by the embodiments of the application is shown.

[0057] Figure 10 Structure diagram of the first embodiment of the user evaluation device based on specific service provided by the embodiments of the present application;

[0058] Figure 11 Structure diagram of the second embodiment of the user evaluation device based on specific service provided by the embodiments of the present application;

[0059] Figure 12 Structure diagram of the third embodiment of the user evaluation device based on specific service provided by the embodiments of the present application;

[0060] Figure 13 Structure diagram of the fourth embodiment of the user evaluation device based on specific service provided by the embodiments of the present application;

[0061] Figure 14 Structure diagram of the fifth embodiment of the user evaluation device based on specific service provided by the embodiments of the present application;

[0062] Figure 15 Structure diagram of the electronic device provided by the embodiments of the present application.

[0063] The specific embodiments of the present application have been shown by the above-mentioned drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application by any means, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] The exemplary embodiments will be described in detail herein below with reference to the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0065] User perception is born with service, and has a long history. In the era of "Internet+", it is promoted to the strategic level. Before the advent of the Internet era, users seek services, such as booking fixed telephone installation, queuing in shopping malls, etc. With the rapid development of mobile Internet, users can access global resources anytime and anywhere, cross-border competition is becoming increasingly fierce, and intra-industry competition is becoming more intense. The brand value, word-of-mouth effect, user stickiness and other enterprise soft power generated by user perception can help enterprises firmly grasp the user entrance and generate huge market value in the Internet era. Usually, the traditional network quality evaluation is evaluated by using the traditional network key performance indicator (KPI) indicator, but the traditional method cannot meet the actual production needs, cannot identify specific areas, and cannot realize the pre-deployment of network resources and real-time monitoring of dynamics.

[0066] To solve the above problems, the application provides a user evaluation method based on specific services, equipment and device. Through fast service identification, special user aggregation area network perception guarantee is completed, network resources are flexibly allocated, intelligent operation is realized, and market development is supported. Specifically, for user perception evaluation, the traditional network quality evaluation method is usually used. The inventor found in the research process that the traditional network KPI indicator cannot meet the actual production needs, and lacks user group perception evaluation based on specific services. Considering these problems, the inventor studied whether the user terminal and the base station can be associated, specific service types can be screened out, and network KPI indicators and timely service performance (Key Quality Indicators, KQI) indicators can be used to realize comprehensive evaluation of specific service user groups, so as to realize directional service perception evaluation, which is used for fine network management and supports market development. Based on this, the technical scheme of the application is proposed.

[0067] Figure 1 The application scenario diagram of the user evaluation method based on specific services provided by the embodiment of the application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the user evaluation method based on specific services provided by the embodiment of the application is applied to the actual scene of daily user group using mobile network, and the scene at least includes an operator core network server, a user group, and at least one base station. The user group can be any user, and the base station can be a large tower base station or an indoor suspended base station with only a router size. The user group and the base station are not limited in the present application.

[0068] The technical scheme of the application and how the technical scheme of the application solves the above technical problems will be described in detail in the following specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments. The embodiments of the application will be described in detail below with reference to the drawings.

[0069] Figure 2 The flowchart of the embodiment one of the user evaluation method based on specific service provided by the present application is shown in the figure. Figure 2 The specific steps include the following steps:

[0070] S101: generating XDR detailed list according to the user information and location information reported by the base station.

[0071] In this step, in order to quickly identify the specific service and the detailed information of the user group accessing the specific service, the signaling face of the base station is associated with the user face separation data of the user terminal when the user accesses the service, and finally the XDR detailed list is generated according to the user information and location information reported by the base station, wherein each entry of the XDR detailed list includes user information, source IP, destination IP, source port, destination port and protocol type of the service accessed by the user.

[0072] Specifically, after the user terminal is started, the user terminal will interact with the base station for signaling data, realize the association of the signaling face and the user face, generate an associated session, and obtain the location information of the user terminal through the management of the associated session. Since the location of each base station is fixed, the location information of the base station can be matched by obtaining the base station ID information. When the user moves, the location information of the user terminal is obtained in real time through the signaling face data update. When the user initiates the service, the user face itself does not carry the user information and the location information, and the base station fills back the user information and the location information through the associated signaling face session. When the service crosses multiple location areas, the last location information is recorded.

[0073] The field filling of the user plane bill service itself mainly depends on the protocol, adopts the IP+TEID combination, realizes the association of the user plane and the signaling plane and the analysis of the user information and the location information. The five-tuple of <source IP, destination IP, source port, destination port, protocol type> is used to realize the association between different data packets of the user plane. The analysis is that the information such as the uniform resource locator (URL) of the HyperText Transfer Protocol (HTTP) bill, Host and the like is parsed according to the standard protocol, the international mobile subscriber identity (IMSI), the mobile subscriber international ISDN / PSTN number (MSISDN), the international mobile equipment identity (IMEI), the IP address and the port number and the like are backfilled, so as to form an XDR detailed bill.

[0074] S102: Filtering the detailed bill information of multiple users corresponding to each type of service from the XDR detailed bill.

[0075] In this step, after obtaining the XDR detailed bill, the information such as the uniform resource locator (URL) of the HyperText Transfer Protocol (HTTP) bill, Host and the like is parsed, the international mobile subscriber identity (IMSI), the mobile subscriber international ISDN / PSTN number (MSISDN), the international mobile equipment identity (IMEI), the IP address and the port number and the like are backfilled, each type of service is filtered from the XDR detailed bill, and the detailed bill information of multiple users accessing the service is obtained through association. The detailed bill information of multiple users includes the information of the base station accessed by multiple users, each type of service can be selected as needed, and the present scheme is not limited.

[0076] S103: Calculating the key performance indicators of each type of service based on the detailed bill information of multiple users corresponding to each type of service.

[0077] In this step, according to the information of the base station in the detailed information of the users corresponding to each type of service and the historical running information reported by the base station, the key performance indicators of each type of service are calculated in the time dimension, wherein the key performance indicators include Key Performance Index (KPI) indicators and Key Quality Indicators (KQI) indicators.

[0078] Specifically, according to the network device type / wireless service type, the key performance indicators specified by the protocol are automatically aggregated and calculated in time sequence by using big data computing power. The KPI indicators include:

[0079] (1) Access performance: 4G connection rate, voice connection rate, etc.

[0080] (2) Maintenance capability: 4G drop rate, wireless drop rate, etc.

[0081] (3) Mobile performance: same frequency handover success rate, different frequency handover success rate, etc.

[0082] (4) Quality of service: 4G average PRB interference value, voice packet loss rate, etc.

[0083] (5) Perception performance: perception rate, etc.

[0084] The KQI indicators include: webpage delay, instant messaging delay, display delay, video delay, download rate, and nucleic acid code delay.

[0085] Each indicator is aggregated and calculated in time sequence, and is associated and managed according to the service type to obtain the real-time dynamic change value of each detailed indicator.

[0086] S104: Based on the preset scoring rules of each type of service and the key performance indicators of each service type, the score of each service type is obtained.

[0087] In this step, the scoring rules of each type of service are preset, and the real-time dynamic change value of each detailed indicator obtained in the above steps is calculated to obtain the score of each service type.

[0088] Specifically, the 4G data service scoring rule is: 4G connection rate*(1-4G drop rate)*same frequency handover success rate*different frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%).

[0089] Timely service scoring rule: Min [100%-(web page delay index value / 120-1), 100%] * Min [100%-(instant messaging index value / 120-1), 100%] * Min [100%-(nucleic acid code delay index value / 120-1), 100%] * Min [100%-(video delay index value / 120-1), 100%].

[0090] Perceived rate scoring rule: perceived rate * 2.5 / 100.

[0091] According to the above scoring rules, the real-time dynamic change value of each detailed index is respectively brought into the scoring rule formula to obtain the score of each service type.

[0092] S105: According to the preset evaluation factor of each service type and the score of each service type, a comprehensive perceived evaluation result is calculated.

[0093] In this step, after the score of each service type is calculated, in order to better know the perceived situation of the user group in a certain area, the evaluation factor of each service type is preset, and the comprehensive perceived evaluation result is calculated in combination with the score of each service type. The evaluation factor of each service type is dynamically adjusted according to the service type and the service proportion.

[0094] Specifically, the evaluation factor of each service type includes: 4G data evaluation factor y1, timely service evaluation factor y2, and perceived rate evaluation factor y3. By associating the score of each service type, the formula: comprehensive perceived evaluation result = 4G data service score * y1 + timely service score * y2 + perceived rate score * y3 is brought in. Finally, the comprehensive perceived evaluation result is obtained.

[0095] The user evaluation method based on specific services provided in this embodiment, according to the user information and location information reported by the base station, analyzes and generates an XDR detailed list, then filters out the detailed list information of each type of service and a plurality of corresponding users in the XDR detailed list, then screens out key performance indicators according to the detailed list information of the plurality of corresponding users, then calculates the score of each service type according to the preset scoring rule of each type of service, and finally calculates the comprehensive perceived evaluation result according to the formula. Through the screening of the key performance indicators, the specific service area is quickly identified, and the comprehensive perceived evaluation of the user group is made through the key performance indicators of the specific service type, ensuring the normal operation of the service and providing a basis for flexible allocation of network resources.

[0096] Figure 3 The flowchart of the user evaluation method based on specific services provided in Embodiment Two of the present application is shown in the above embodiment. Based on the above embodiment, the user evaluation method based on specific services further comprises:

[0097] S106: According to the base station information and the base station operation history log, the key performance indicators are determined by time dimension aggregation and screening.

[0098] In this step, the base station reports the operation log of the base station within the period set by the operator. In order to make perception evaluation for the user group dominated by a specific service, the traditional network performance indicators cannot meet the demand. Therefore, according to the base station information in the detailed bill of each type of service corresponding to multiple users screened out from the XDR detailed bill and the base station operation history log, the key performance indicators are screened out by time dimension aggregation.

[0099] Specifically, the key performance indicators include KPI indicators and KQI indicators, wherein the KPI indicators include:

[0100] (1) Access performance: 4G access rate, voice access rate, etc.

[0101] (2) Maintenance capability: 4G drop rate, wireless drop rate, etc.

[0102] (3) Mobile performance: Same frequency handover success rate, different frequency handover success rate, etc.

[0103] (4) Service quality: 4G average PRB interference value, voice packet loss rate, etc.

[0104] (5) Perception performance: Perception rate, etc.

[0105] The KQI indicators include: webpage delay, instant messaging delay, display delay, video delay, download rate, and nucleic acid code delay.

[0106] The user evaluation method based on a specific service provided in the embodiment aggregates and screens the key performance indicators according to the base station information and the base station operation history log, wherein the key performance indicators include KPI indicators and KQI indicators, the network quality is comprehensively evaluated by the KPI indicators and the KQI indicators, and the perception evaluation for the user group dominated by a specific service is realized.

[0107] In the user evaluation method based on a specific service provided in the embodiment, the evaluation factors of each type of service include: 4G data evaluation factor, instant service evaluation factor, and perception rate evaluation factor.

[0108] Figure 4 The flowchart of the user evaluation method based on a specific service provided in the embodiment is shown in FIG. 3. Based on the method embodiment one, as shown in FIG. 3, the score of each type of service is obtained in step S104 based on the preset score rule of each type of service and the key performance indicators of each type of service, including: Figure 4 the score of each type of service is obtained in step S104 based on the preset score rule of each type of service and the key performance indicators of each type of service, including:

[0109] S1041: 4G data service.

[0110] In this step, for the 4G data service, the key performance indicators involved include: 4G access rate in access performance, 4G drop rate in retention capability, same frequency handover success rate, inter-frequency handover success rate in mobility performance, and 4G average PRB interference value in quality of service.

[0111] Specifically, the 4G data service scoring rule formula is expressed as: 4G access rate * (1-4G drop rate) * same frequency handover success rate * inter-frequency handover success rate * min (100% * 4G average PRB interference value / -110, 100%), and the above key performance indicators are brought in to calculate the 4G data service score.

[0112] S1042: real-time service.

[0113] In this step, for the real-time service, the key performance indicators involved include: webpage delay indicator value, instant messaging indicator value, nucleic acid code delay indicator value, and video delay indicator value in KQI indicators.

[0114] Specifically, the real-time service scoring rule formula is expressed as: Min[100%-(webpage delay indicator value / 120-1), 100%] * Min[100%-(instant messaging indicator value / 120-1), 100%] * Min[100%-(nucleic acid code delay indicator value / 120-1), 100%] * Min[100%-(video delay indicator value / 120-1), 100%], and the above key performance indicators are brought in to calculate the real-time service score.

[0115] S1043: perception rate.

[0116] In this step, for the perception rate, the key performance indicators involved include: perception rate in perception performance.

[0117] Specifically, the perception rate scoring rule formula is expressed as: perception rate * 2.5 / 100, and the above key performance indicators are brought in to calculate the perception rate score.

[0118] The user evaluation method based on specific services provided in this embodiment pre-sets the scoring rules of each service type, calculates the score of each service type according to the scoring rules, and manages the specific services through scoring.

[0119] Figure 5 The flowchart of the user evaluation method based on specific services provided in Embodiment Four of the present application is shown. Based on the method embodiment one, the user evaluation method based on specific services further includes:

[0120] S107: generating a long trend change graph of the comprehensive perception evaluation result according to the comprehensive perception evaluation result obtained at each time point.

[0121] In this step, after obtaining the comprehensive perception evaluation result of the user, a long trend change graph of the comprehensive perception evaluation result is generated according to the historical result and the comprehensive perception evaluation result, and the change graph is displayed on a large screen for intuitive display, thereby providing a basis for realizing intelligent operation.

[0122] S108: dividing the network health level based on the long trend change graph and the scoring range of different levels.

[0123] In this step, in order to avoid abnormal judgment caused by fluctuations, the long trend change graph of the comprehensive perception evaluation result in the above step is combined with historical data to divide different states such as excellent+, excellent, excellent-, good, medium and poor. The specific scoring levels are shown in Table 1.

[0124] The user evaluation method based on specific services provided in this embodiment generates a long trend change graph of the comprehensive perception evaluation result by combining historical data, and divides it into different state levels, thereby realizing automatic monitoring and early warning of mobile wireless network sub-health and assisting front-line production personnel to ensure normal operation of the network.

[0125] Table 1: Scoring levels

[0126] Score Grade Score Grade 1 : Excellent + = [95%, 100%] Score Grade 2: Excellent = [90%, 95%] Score Grade 3: Excellent - = [85%, 90%] Score Grade 4: Good = [80%, 85%] Score Grade 5: Fair = [60%, 80%] Score Grade 6: Poor = [0%, 60%]

[0127] Figure 6 The flowchart of the comprehensive perception evaluation provided in this embodiment is shown in FIG. 1, and specifically includes: Figure 6

[0128] S201: unified management.

[0129] In this step, by associating the user plane with the signaling plane, unified management of various information of the base station and various information of the user is realized, and then the key performance indicators are calculated, the key performance indicators are classified by referring to the 3GPP protocol, and finally the key performance indicators are aggregated and screened according to the time dimension.

[0130] Specifically, after the user terminal is turned on, data interaction is performed with the base station, an associated session is generated by associating the user plane with the signaling plane, and the location information of the user terminal is obtained by managing the associated session. Since the location of each base station is fixed, the base station ID information can be matched to obtain the location information of the base station, Figure 7 The flowchart of the LTE for obtaining the base station ID provided in this embodiment is shown in FIG. 2. Figure 7 ​As shown, through a Long Term Evolution (LTE) network model, data interaction between a user terminal, a base station and a mobile management entity is realized, thereby realizing acquisition of a base station ID, Figure 8 A flowchart for acquiring a base station cell ID in LTE provided by the embodiment of the present application is shown in FIG. 2. Figure 8 As shown, through a Primary Synchronization Signal (PSS), a Secondary Synchronization Signal (SSS), a Physical Broadcast channel (PBCH) and a Physical Downlink Shared Channel (PDSCH), processing of the base station ID is realized, and finally the ID of the base station cell is obtained. When the user moves, real-time acquisition of the location information of the user terminal is realized through signaling plane data update.

[0131] When the user initiates a service, the user plane itself does not carry user information and location information, through an associated signaling plane session, using a device-fixed communication protocol, through reverse data filling, user behavior data is associated with base station information accessed by the user. The base station fills in the user information and the location information. When the service crosses multiple location areas, the last location information is recorded. The field filling of the user plane service itself mainly depends on the protocol, and an IP+TEID combination is used to realize association of the user plane and the signaling plane and analysis of the user information and the location information. Between different data packets of the user plane, a five-tuple of <source IP, destination IP, source port, destination port, protocol type> is used to realize association. The analysis is to analyze according to the standard protocol, for example, uniform resource locator URL, Host information and the like of the HTTP single bill, and backfill IMSI / MSISDN / IMEI, IP address and port number and the like, thereby forming an XDR detailed bill. Figure 9 A flowchart for acquiring a user service in LTE provided by the embodiment of the present application is shown in FIG. 3. Figure 9As shown, the user's service information is obtained through the LTE network architecture, wherein the network architecture is divided into four parts. Among them, UE represents the user terminal. LTE represents the base station, which can be a large tower base station or an indoor suspended base station with only router size. SAE represents the core network server of the operator, wherein the core network includes many servers, such as processing signaling, processing data, and processing charging policy. The signaling link S1-U obtains ENBID, CELLID, Server IP, Server Port, MSIP, MS Port, F-TEID, HTTP / DNS and other information, the link S11 obtains IMSI / IMEI / MSISDN and other information, the link S6a obtains IMSI information, and the link S1-MME obtains IMSI / IP-TMSI and other information. Finally, an XDR detailed list is generated, which includes Figure 9 the user's service information obtained in the middle.

[0132] According to the URL, Host and other information of the HTTP invoice, the IMSI / MSISDN / IMEI, IP address and port number and other information are backfilled, each type of service is filtered out from the XDR detailed list, and the detailed list information of multiple users accessing the above services is obtained through association.

[0133] S202: Large-screen BI presentation.

[0134] In this step, according to the XDR detailed list in the above steps, the key performance indicators are screened out, the evaluation factors are constructed according to the business proportion, the appropriate algorithm is selected to calculate the comprehensive perception evaluation result, and the comprehensive perception evaluation result is presented on the large screen, and the network monitoring is realized through grade division.

[0135] Specifically, according to the base station information in the detailed list information of multiple users corresponding to each type of service and the historical running information reported by the base station, the key performance indicators of each type of service are calculated according to the time dimension, wherein the key performance indicators include KPI indicators and KQI indicators.

[0136] Specifically, according to the network equipment type / wireless service type, the big data computing power is used for automatic analysis, the key performance indicators specified by the protocol are automatically aggregated and calculated according to the time sequence, and the KPI indicators include:

[0137] (1) Access performance: 4G connection rate, voice connection rate, etc.

[0138] (2) Retention capability: 4G drop rate, wireless drop rate, etc.

[0139] (3) Mobility performance: same frequency handover success rate, different frequency handover success rate, etc.

[0140] (4) Service quality: 4G average PRB interference value, voice packet loss rate, etc.;

[0141] (5) Perception performance: perception rate, etc.

[0142] KQI metrics include: webpage latency, instant messaging latency, display latency, video latency, download speed, and nucleic acid code latency.

[0143] By aggregating and calculating various indicators based on time series and managing them in association by business type, we can obtain the real-time dynamic change value of each detailed indicator.

[0144] The scoring rules for each type of business are pre-defined, and the score for each business type is calculated based on the real-time dynamic changes of each detailed indicator obtained from the above steps.

[0145] Specifically, the scoring rules for 4G data services are: 4G call success rate * (1 - 4G drop rate) * same-frequency handover success rate * different-frequency handover success rate * min(100% * 4G average PRB interference value / -110, 100%).

[0146] Timely Business Scoring Rules: Min[100% - (Webpage Latency Index Value)]

[0147] / 120-1),100%]*Min[100%-(Instant Messaging Index Value / 120-1),100%]*Min[100%-(Nucleic Acid Code Latency Index Value / 120-1),100%]*Min[100%-(Video Latency Index Value / 120-1),100%].

[0148] Perception rate scoring rules: Perception rate * 2.5 / 100.

[0149] Based on the above scoring rules, the real-time dynamic change value of each detailed indicator is substituted into the scoring rule formula to obtain the score for each business type.

[0150] After calculating the score for each business type, in order to better understand the perception of user groups within a certain area, evaluation factors for each business type are pre-set. Combined with the scores for each business type, a comprehensive perception assessment result is calculated. The evaluation factors for each business type are dynamically adjusted according to the business type and its proportion.

[0151] Specifically, the evaluation factors for each service type include: 4G data evaluation factor y1, real-time service evaluation factor y2, and perceived speed evaluation factor y3. By correlating the scores of each service type and substituting them into the formula: Comprehensive Perception Evaluation Result = 4G Data Service Score * y1 + Real-time Service Score * y2 + Perception Speed ​​Score * y3. The final comprehensive perception evaluation result is then obtained.

[0152] After obtaining the comprehensive perception evaluation result of the user, a long trend change graph of the comprehensive perception evaluation result is drawn according to the historical result and the comprehensive perception evaluation result, and the change graph is presented on a large screen for intuitive display, thereby providing a basis for realizing intelligent operation.

[0153] For the long trend change graph of the comprehensive perception evaluation result in the above steps, in order to avoid abnormal judgment caused by fluctuations, different states such as excellent +, excellent, excellent-, good, medium, and poor are divided in combination with historical data.

[0154] The user evaluation method based on specific services provided by the embodiments of the application realizes unified management of various information of the base station and various information of the user through the association of the user plane and the signaling plane, then calculates the key performance indicators through the unified management, and classifies the key performance indicators according to the 3GPP protocol, and finally aggregates and screens the key performance indicators according to the time dimension. According to the XDR detailed list, the key performance indicators are screened, and evaluation factors are constructed according to the business proportion, a suitable algorithm is selected to calculate the comprehensive perception evaluation result, the comprehensive perception evaluation result is presented on the large screen, and network monitoring is realized through grade division. The specific service area is quickly identified, the user group is comprehensively perceived through the key performance indicators, the normal operation of the service is ensured, and the network resources are flexibly allocated.

[0155] Figure 10 The structure diagram of the user evaluation device based on specific services provided by the embodiments of the application is shown in Figure 10 As shown in the figure, the user evaluation device based on specific services 300 includes:

[0156] The data generation module 301 is configured to generate an XDR detailed list according to the user information and the location information reported by the base station, and each entry in the XDR detailed list includes user information, and source IP, destination IP, source port, destination port, and protocol type of the service accessed by the user.

[0157] The service filtering module 302 is configured to filter detailed list information of a plurality of users corresponding to each type of service from the XDR detailed list.

[0158] The first calculation module 303 is configured to calculate the key performance indicators of each type of service based on the detailed list information of the plurality of users corresponding to each type of service.

[0159] The second calculation module 304 is configured to obtain the score of each type of service based on a preset scoring rule of each type of service and the key performance indicators of each type of service.

[0160] The third calculation module 305 is configured to calculate a comprehensive perception evaluation result according to the preset evaluation factor of each service type and the score of each service type.

[0161] The user evaluation device based on specific services provided in the embodiments of the present application is used to implement the technical method in any of the foregoing method embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0162] Figure 11 The structure diagram of the second embodiment of the user evaluation device based on specific services provided in the embodiments of the present application is shown in FIG. 3B. Figure 11 As shown in FIG. 3B, the user evaluation device 300 further includes:

[0163] The index acquisition module 306 is configured to determine a key performance index according to the base station information and the base station operation history, and according to the time dimension convergence and screening.

[0164] The KPI index includes at least one of the following: access performance, maintenance capability, mobile performance, service quality, and perception performance; and the KQI index includes at least one of the following: webpage delay, instant messaging delay, display delay, video delay, and download rate.

[0165] The evaluation factor of each service type in the user evaluation device based on specific services provided in the embodiments of the present application includes the 4G data evaluation factor, the instant service evaluation factor, and the perception rate evaluation factor.

[0166] Figure 12 The structure diagram of the third embodiment of the user evaluation device based on specific services provided in the embodiments of the present application is shown in FIG. 3C. Figure 12 As shown in FIG. 3C, the second calculation module 304 includes:

[0167] The data service unit 3041 is configured to calculate the score of the 4G data service according to 4G connection rate*(1-4G drop rate)*intra-frequency handover success rate*inter-frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%).

[0168] The instant service unit 3042 is configured to calculate the score of the instant service according to Min[100%-(webpage delay index value / 120-1), 100%]*Min[100%-(instant messaging index value / 120-1), 100%]*Min[100%-(nucleic acid code delay index value / 120-1), 100%]*Min[100%-(video delay index value / 120-1), 100%].

[0169] The sensing rate unit 3043 is used to calculate a score for the sensing rate based on the sensing rate * 2.5 / 100.

[0170] Figure 13 A schematic diagram of the structure of Embodiment 4 of the user evaluation device based on a specific service provided in this application is shown below. Figure 13 As shown, the third calculation module 305 includes:

[0171] The calculation unit 3051 is used to multiply the score of each business type by the evaluation factor of the corresponding business type and then sum them up to obtain the comprehensive perception evaluation result.

[0172] Figure 14 A schematic diagram of the structure of Embodiment 5 of the user evaluation device based on a specific service provided in this application is shown below. Figure 14 As shown, the user evaluation device 300 based on specific business also includes:

[0173] The image generation module 307 is used to generate a long-term trend map of the comprehensive perception evaluation results based on the comprehensive perception evaluation results obtained at each time moment.

[0174] The rating module 308 is used to classify network health levels based on long-term trend change charts and different rating ranges.

[0175] The user evaluation device based on a specific business provided in any of the foregoing embodiments is used to execute the technical methods in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0176] This application also provides an electronic device. Figure 15 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 15 As shown, the electronic device 400 includes: a processor 401, and a memory 402 and a display 403 that are communicatively connected to the processor.

[0177] Memory 401 stores computer-executed instructions.

[0178] Processor 402 executes computer execution instructions stored in memory to implement the method in any of the embodiments.

[0179] Display 403 is used to present a long-term trend graph of the comprehensive perception evaluation results.

[0180] Optionally, the various devices described above in the electronic device 400 can be connected via a system bus.

[0181] The memory 402 can be a separate memory unit or a memory unit integrated into the processor 401. The number of processors 401 can be one or more.

[0182] Optionally, the electronic device 400 can further include an interface for interacting with other devices.

[0183] Optionally, the display in the electronic device can also be used to display the processing result of the processor and the human-computer interaction. In some embodiments, the display can be the front panel of the terminal device; in other embodiments, the display can be a flexible display screen, which is arranged on the curved surface or the folding surface of the terminal device. Even, the display can also be arranged as an irregularly shaped display screen, that is, a special-shaped screen. The display can be made of liquid crystal display (LCD), organic light-emitting diode (OLED) and other materials.

[0184] It should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The steps of the method disclosed in the present application can be directly embodied as the execution of the hardware processor, or the execution of the combination of the hardware and software modules in the processor.

[0185] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0186] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a readable memory. The program, when executed, performs steps including the above-mentioned method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (read-only memory, ROM), RAM, flash memory, hard disk, solid state disk, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0187] The electronic device provided in the embodiment can be used to execute the technical solutions in any of the above method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0188] The embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the method in any of the embodiments.

[0189] The computer readable storage medium described above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0190] Optionally, the readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0191] The embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer readable storage medium. At least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the technical solutions provided by any of the above method embodiments.

[0192] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0193] It is to be understood that the application is not limited to the precise construction herein disclosed and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A method for evaluating a user based on a specific service, characterized by, The method comprises: generating an application detailed record XDR list according to user information and location information reported by a base station, each entry in the XDR list comprising user information and source IP, destination IP, source port, destination port, and protocol type of a service accessed by the user; filtering out detailed information of a plurality of users corresponding to each type of service from the XDR list; based on information of the base station in the detailed information of the plurality of users corresponding to each type of service and a running history log of the base station, aggregating and filtering to calculate a key performance indicator of the type of service in a time dimension, the key performance indicator comprising a key performance KPI indicator and a real-time service KQI indicator; based on a preset scoring rule of each type of service and the key performance indicator of each type of service, obtaining a score of each type of service; based on a preset evaluation factor of each type of service and the score of each type of service, calculating a comprehensive perception evaluation result; the evaluation factor of each type of service comprises a 4G data evaluation factor, a real-time service evaluation factor, and a perception rate evaluation factor, and the evaluation factor is dynamically adjusted according to the type of service and a service proportion.

2. The method of claim 1, wherein, The KPI indicator comprises at least one of access performance, retention capability, mobility performance, service quality, and perception performance; and the KQI indicator comprises at least one of webpage latency, instant messaging latency, display latency, video latency, download rate, and nucleic acid code latency.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: for 4G data service, calculating the score of the 4G data service according to 4G connection rate*(1-4G drop rate)*intra-frequency handover success rate*inter-frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%); for real-time service, calculating the score of the real-time service according to Min[100%-(webpage latency indicator value / 120-1), 100%]*Min[100%-(instant messaging indicator value / 120-1), 100%]*Min[100%-(nucleic acid code latency indicator value / 120-1), 100%]*Min[100%-(video latency indicator value / 120-1), 100%]; for perception rate, calculating the score of the perception rate according to perception rate*2.5 / 100.

4. The method according to claim 1 or 2, characterized in that, The method further comprises: multiplying the score of each type of service by the evaluation factor of the corresponding type of service and then accumulating to obtain the comprehensive perception evaluation result.

5. The method of claim 4, wherein, The method further comprises: generating a long trend change graph of the comprehensive perception evaluation result according to the comprehensive perception evaluation result obtained at each time point; and performing network health level division based on the long trend change graph and a scoring range of different levels.

6. A user evaluation device based on a specific service, characterized by, The method comprises: The data generation module is configured to generate an application detailed record XDR form according to user information and location information reported by the base station, wherein each entry of the XDR form comprises user information, and source IP, destination IP, source port, destination port, and protocol type of a service accessed by the user; The service filtering module is configured to filter out detailed information of a plurality of users corresponding to each type of service from the XDR form; The first calculation module is configured to calculate a key performance indicator of the type of service according to information of the base station in the detailed information of the plurality of users corresponding to each type of service and a base station operation history log, and according to a time dimension, wherein the key performance indicator comprises a key performance KPI indicator and a timely service KQI indicator; The second calculation module is configured to obtain a score of each type of service based on a preset scoring rule of each type of service and the key performance indicator of each type of service; The third calculation module is configured to calculate a comprehensive perception evaluation result according to a preset evaluation factor of each type of service and the score of each type of service. The evaluation factor of each type of service comprises a 4G data evaluation factor, a timely service evaluation factor, and a perception rate evaluation factor, and the evaluation factor is dynamically adjusted according to the type of service and a service proportion.

7. The apparatus of claim 6, The KPI indicator comprises at least one of access performance, maintenance capability, mobility performance, service quality, and perception performance; and the KQI indicator comprises at least one of webpage delay, instant messaging delay, display delay, video delay, download rate, and nucleic acid code delay. The second calculation module comprises:

8. The apparatus of claim 6 or 7, wherein, A data service unit is configured to calculate the score of the 4G data service according to 4G connection rate*(1-4G drop rate)*intra-frequency handover success rate*inter-frequency handover success rate*min(100%*4G average PRB interference value / -110, 100%). A timely service unit is configured to calculate the score of the timely service according to Min[100%-(webpage delay indicator value / 120-1), 100%]*Min[100%-(instant messaging indicator value / 120-1), 100%]*Min[100%-(nucleic acid code delay indicator value / 120-1), 100%]*Min[100%-(video delay indicator value / 120-1), 100%]. A perception rate unit is configured to calculate the score of the perception rate according to perception rate*2.5 / 100. The third calculation module comprises:

9. The apparatus of claim 6 or 7, wherein, A calculation unit is configured to multiply the score of each type of service by the evaluation factor of the corresponding type of service, and then accumulate the products to obtain the comprehensive perception evaluation result. The device further comprises:

10. The apparatus of claim 9, wherein, An image generation module is configured to generate a long trend change graph of the comprehensive perception evaluation result according to the comprehensive perception evaluation result obtained at each time point. A grade division module is configured to divide the network health grade based on the long trend change graph and a scoring range of different grades. ​ 11. An electronic device comprising: a processor, and a memory, a display connected in communication with the processor; the memory stores computer-executed instructions; the processor executes the computer-executed instructions stored in the memory to implement the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executed instructions, and the computer-executed instructions are executed by the processor to implement the method according to any one of claims 1 to 5.

Images