Method and device for tracing complaint problem, medium and program product

Abstract

The embodiment of the invention provides a complaint problem tracing method and device, a medium and a program product, and the method comprises the steps: extracting first information from complaint description information, the first information being used for indicating at least one of a complaint user, the occurrence time of a complaint event, and the occurrence place of the complaint event; determining at least one type of mobile network operation data and / or mobile network service data related to the first information; and obtaining a traceability result based on the at least one mobile network operation data and / or mobile network service data. According to the embodiment of the invention, the problem of low accuracy of a tracing method in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of mobile communication technology, and in particular to a method, device, medium and program product for tracing the source of a complaint problem. Background Art

[0002] Customer complaint handling in mobile communication networks is an important means to improve customer perception, and is also an important issue faced by ensuring the leading edge of network quality in the context of the increasing popularity of 5G services. Complaint tracing support has the important mission of improving customer service complaint handling efficiency and customer satisfaction.

[0003] Existing complaint problem tracing solutions mainly include the following solutions: (1) Pulling signaling data, key performance indicator (KPI) data, etc., based on manually set thresholds to achieve the demarcation and location of complaint problems; (2) Based on a large amount of historical data, enumerate different demarcation and location situations, match them with newly emerged complaint problems, and achieve problem demarcation and location. The above solutions mainly rely on manual experience, and the accuracy of their tracing results is not high. Summary of the invention

[0004] At least one embodiment of the present application provides a method, device, medium and program product for tracing the source of a complaint problem, which are used to solve the problem of low accuracy of the tracing method in the prior art.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for tracing the source of a complaint problem, including:

[0007] Extracting first information from the complaint description information, where the first information is used to indicate at least one of a complaining user, an occurrence time of the complaint event, and an occurrence location of the complaint event;

[0008] Determining at least one mobile network operation data and / or mobile network service data related to the first information;

[0009] Based on the at least one mobile network operation data and / or mobile network service data, a tracing result is obtained.

[0010] Optionally, the above method further includes:

[0011] The tracing result is displayed.

[0012] Optionally, the at least one mobile network operation data includes at least one of wireless network data, external data representation XDR data, and performance data; the wireless network data includes at least one of alarm data, network optimization data, and measurement report MR data; the mobile network service data includes at least one of user service contract status, user speed limit information, shutdown / resumption information, and arrears information.

[0013] Optionally, the determining of at least one type of mobile network operation data and / or mobile network service data related to the first information includes:

[0014] According to the occurrence time of the complaint event, a historical time period related to the complaint event is obtained, wherein the occurrence time of the complaint event is a subset of the historical time period;

[0015] According to the location where the complaint event occurred, a historical cell related to the complaint event is obtained, wherein the location where the complaint event occurred is within the coverage of the historical cell; at least one mobile network operation data related to the historical cell within the historical time period is extracted; and / or, mobile network service data of the complaining user within the historical time period is extracted.

[0016] Optionally, when the at least one mobile network operation data and / or mobile network service data includes the mobile network service data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data includes:

[0017] Determining whether the service status corresponding to the complaint event is in an abnormal state according to the mobile network service data of the complaining user in the historical time period;

[0018] When the business status corresponding to the complaint event is in an abnormal state, the tracing result of the user contract data problem is output and the process ends.

[0019] Optionally, when the at least one mobile network operation data and / or mobile network service data further includes at least one mobile network operation data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data further includes:

[0020] When the service status corresponding to the complaint event is in a normal state, matching the at least one mobile network operation data with the corresponding tracing rule respectively to obtain at least one intermediate tracing result;

[0021] Based on the at least one intermediate tracing result, a final tracing result is obtained.

[0022] Optionally, when the at least one mobile network operation data and / or mobile network service data includes only at least one mobile network operation data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data includes:

[0023] Matching the at least one mobile network operation data with the corresponding tracing rules respectively to obtain at least one intermediate tracing result;

[0024] Based on the at least one intermediate tracing result, a final tracing result is obtained.

[0025] Optionally, when the at least one mobile network operation data includes the wireless network data, the matching of the at least one mobile network operation data with the corresponding tracing rules respectively to obtain at least one intermediate tracing result includes at least one of the following:

[0026] Matching the alarm data related to the historical cell in the historical time period with the rules in the alarm rule base to determine the alarm information in the historical time period;

[0027] Matching the network optimization data related to the historical cell in the historical time period with the judgment rule of abnormal network indicators to determine the abnormal network indicators and / or the number of occurrences of the abnormal network indicators;

[0028] The MR data related to the historical cell in the historical time period is matched with a judgment rule of abnormal network coverage to determine abnormal coverage information.

[0029] Optionally, matching the alarm data related to the historical cell in the historical time period with a rule in an alarm rule base to determine the alarm information in the historical time period includes:

[0030] Matching alarm data related to the first historical cell in a first time period with a first alarm rule base to determine alarm information in the first time period;

[0031] Matching the alarm data related to the second historical cell in the second time period with the second alarm rule base to determine the alarm information in the second time period;

[0032] Among them, the complaining user accessed the first historical cell during the first time period and accessed the second historical cell during the second time period; the first alarm rule base includes matching rules for multiple alarm types, and the second alarm rule base includes matching rules for alarm types of cell exit service.

[0033] Optionally, when the at least one mobile network operation data includes XDR data and / or performance data, the matching of the at least one mobile network operation data with corresponding tracing rules respectively to obtain at least one intermediate tracing result includes at least one of the following:

[0034] Matching the XDR data and / or performance data related to the historical cell in the historical time period with a delimitation rule for determining the location of the complaint event, to obtain a delimitation result of the complaint event;

[0035] The XDR data and / or performance data related to the historical cell in the historical time period are matched with a positioning rule for determining the cause of the complaint event to obtain a positioning result of the complaint event.

[0036] Optionally, the demarcation rule includes at least one of the following: a demarcation rule for core network problems, a demarcation rule for wireless network problems, a demarcation rule for terminal problems, a demarcation rule for service provider problems, and a demarcation rule for terminal service client problems;

[0037] The positioning rule includes at least one of the following: a positioning rule for a TCP connection, a positioning rule for an HTTP connection, a positioning rule for a handover, and a positioning rule for control plane processing.

[0038] Optionally, obtaining a final tracing result based on the at least one intermediate tracing result includes:

[0039] Based on the at least one intermediate tracing result, obtaining first input data;

[0040] The first input data is used as the input of the first model to obtain the final tracing result output by the first model, wherein the first model is a machine learning model pre-trained with the first training data.

[0041] Optionally, obtaining a final tracing result based on the at least one intermediate tracing result includes:

[0042] Based on the at least one intermediate tracing result, obtaining first input data;

[0043] Extracting at least one type of network status data from the at least one type of mobile network operation data and / or mobile network service data, and encoding the at least one type of network status data to obtain second input data;

[0044] The first input data and the second input data are used as inputs of a second model to obtain the final tracing result output by the second model, wherein the second model is a machine learning model pre-trained with second training data.

[0045] Optionally, the above method further includes:

[0046] Performing text preprocessing on the complaint description information to obtain a first text;

[0047] Obtaining summary information of the first text, wherein the summary information includes at least one sentence;

[0048] Classifying the sentences in the summary information to obtain at least one classification;

[0049] Obtain the complaint category corresponding to each classification, where the complaint category includes network-related issues and non-network-related issues;

[0050] In the case where the at least one classified complaint category includes network problems, entering the step of extracting the first information from the complaint description information;

[0051] In the case that none of the complaint categories in the at least one classification includes network-related issues, a prompt message indicating that the current complaint is a non-network-related issue is output and the process ends.

[0052] Optionally, obtaining summary information of the first text includes:

[0053] Obtaining importance indexes of each sentence in the first text, and determining candidate sentences according to the importance indexes; screening the candidate sentences using keywords to obtain key sentences; shielding the key sentences in the first text to obtain a second text;

[0054] The second text is input into a pre-trained language model to obtain summary information of the first text generated by the pre-trained language model.

[0055] Optionally, categorizing the sentences in the summary information includes:

[0056] Performing text preprocessing on the summary information to obtain a document including at least one sentence;

[0057] For each sentence in the document, respectively calculate the weight of each word in the document in the sentence, and form a vector by combining the calculated weights of each word to obtain the vector of the sentence;

[0058] Clustering is performed based on the vectors of each sentence to obtain at least one classification.

[0059] In a second aspect, an embodiment of the present application provides a complaint tracing device, including:

[0060] A first extraction module, used to extract first information from the complaint description information, where the first information is used to indicate at least one of the complaining user, the time when the complaint event occurred, and the location where the complaint event occurred;

[0061] A second determination module, configured to determine at least one type of mobile network operation data and / or mobile network service data related to the first information;

[0062] The first processing module is used to obtain a tracing result based on the at least one mobile network operation data and / or mobile network service data.

[0063] In the third aspect, an embodiment of the present application provides a device for tracing the source of a complaint problem, including: a transceiver, a processor, a memory, and a program or instruction stored on the memory and executable on the processor; when the processor executes the program or instruction, it implements the steps of the method described in any one of the first aspects.

[0064] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0065] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in any one of the first aspects.

[0066] Compared with the prior art, the complaint problem tracing method, device, medium and program product provided in the embodiments of the present application generate at least one tracing result based on at least one mobile network operation data, and then generate the final tracing result based on the at least one tracing result. The above method fully considers the influence of various data and can improve the accuracy of the tracing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0068] Figure 1 A flow chart of a method for tracing the source of a complaint problem according to an embodiment of the present application;

[0069] Figure 2 A schematic diagram of tracing the source of wireless network data in an embodiment of the present application;

[0070] Figure 3Another flow chart of the method for tracing the source of a complaint problem in an embodiment of the present application;

[0071] Figure 4 This is an example diagram showing the traceability results of an embodiment of the present application;

[0072] Figure 5 This is a schematic diagram of the structure of a complaint tracing device according to an embodiment of the present application;

[0073] Figure 6 This is a schematic diagram of the structure of a complaint source tracing device according to another embodiment of the present application. DETAILED DESCRIPTION

[0074] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0075] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0076] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described technology can be used for the above-mentioned systems and radio technologies as well as other systems and radio technologies. The following description describes a new radio (NR) system for example purposes, and NR terms are used in most of the following descriptions, but these technologies can also be applied to systems other than NR systems, such as 6th Generation (6G) communication systems.

[0077] As described in the background technology, the traceability solutions in the prior art have problems such as low accuracy. To solve at least one of the above problems, an embodiment of the present application provides a method for tracing the complaint problem, which can improve the accuracy of the traceability results.

[0078] Please refer to Figure 1 , a method for tracing the source of a complaint problem provided in an embodiment of the present application includes:

[0079] Step 11, extracting first information from the complaint description information, where the first information is used to indicate at least one of the complaining user, the time when the complaint event occurred, and the location where the complaint event occurred.

[0080] Here, the complaint description information may be a complaint ticket or text information generated based on the complaint ticket, which usually includes information such as the complaining user, the complaint event, the time when the complaint event occurred, and the location where the complaint event occurred. The embodiment of the present application extracts the first information from the complaint description information to obtain at least one of the complaining user, the time when the complaint event occurred, and the location where the complaint event occurred.

[0081] Considering that there is usually a certain time interval between the time when the user initiates a complaint and the time when the complaint event occurs, for example, the user usually initiates a complaint after a period of time has passed since the complaint time event occurred. Therefore, in an embodiment of the present application, the time of occurrence of the complaint time is preferentially extracted from the complaint description information, rather than simply using the user complaint time (the time when the user initiates the complaint, such as the time when the user complaint is received) as the time of occurrence of the complaint event. Similarly, the location of the complaint event may not be the same location as the location where the user complained (the location where the user initiated the complaint). Therefore, in an embodiment of the present application, the location of the complaint time extracted from the complaint description information is preferentially used, rather than simply using the location where the user complained as the location where the complaint event occurred. In the above manner, more accurate complaint-related information can be obtained, thereby improving the accuracy of subsequent traceability results.

[0082] Specifically, the embodiments of the present application can use a named entity recognition (NER) algorithm or model to extract the time and location of the complaint event from the complaint description information. If the time of the complaint event cannot be extracted from the complaint description information, the time when the user initiated the complaint can be used as the time of the complaint event; if the location of the complaint event cannot be extracted from the complaint description information, the location where the user was when the complaint was made can be used as the location of the complaint event.

[0083] Step 12: Determine at least one mobile network operation data and / or mobile network service data related to the first information.

[0084] Here, the at least one mobile network operation data includes at least one of wireless network data, (External Data Representation, XDR) data, and performance data. The wireless network data includes at least one of alarm data, network optimization data, and measurement report MR data. The mobile network service data includes at least one of user service contract status, user speed limit information, shutdown / restart information, and arrears information.

[0085] The embodiment of the present application can also obtain a historical time period related to the complaint event according to the time of occurrence of the complaint event, wherein the time of occurrence of the complaint event is a subset of the historical time period. According to the location of occurrence of the complaint event, a historical cell related to the complaint event is obtained, wherein the location of occurrence of the complaint event is within the coverage of the historical cell. Then, extract mobile network operation data related to the historical cell within the historical time period, such as wireless network data, XDR data, performance data, etc.; and / or extract mobile network service data of the complaining user within the historical time period.

[0086] Specifically, the XDR data related to the historical cell may include: HTTP data, S1-MME data, etc. The performance data related to the historical cell may include: performance data of control plane users and cell dimensions, performance data of user plane users and cell dimensions, etc.

[0087] Since there is usually a certain time interval between the time when the user initiates a complaint and the time when the complaint event occurs, as an optional implementation method, when obtaining the historical time period related to the complaint event based on the time when the complaint event occurs: (1) If the time when the complaint event occurs is extracted from the complaint description information and is accurate to the granularity of hours, the historical time related to the complaint event can be set to: the time period between the first moment and the second moment, the second moment is the time when the complaint event occurs, the first moment is located before the second moment, and the interval between the first moment and the second moment is n hours, where n is a preset positive number; (2) If the time when the complaint event occurs is extracted from the complaint description information and is accurate to the granularity of days (natural days), the historical time related to the complaint event can be set to: 0:00-24:00 on the natural day when the complaint event occurs; (3) If the time when the complaint event occurs is the time when the user initiates the complaint, the historical time related to the complaint event can be set to: the time period between the third moment and the fourth moment, the fourth moment is the time when the complaint event occurs, the third moment is located before the second moment, and the interval between the third moment and the second moment is 24 hours.

[0088] Step 13: Obtain a tracing result based on the at least one mobile network operation data and / or mobile network service data.

[0089] Through the above steps, the embodiment of the present application can generate a traceability result based on at least one mobile network operation data and / or mobile network service data. The above method fully considers the influence of various data and can improve the accuracy of the traceability result.

[0090] The traceability process in step 13 is described below.

[0091] If the at least one mobile network operation data and / or mobile network service data includes the mobile network service data, the embodiment of the present application can determine whether the service status corresponding to the complaint event is in an abnormal state based on the mobile network service data of the complaining user within the historical time period. Specifically, the complaint event can be analyzed to determine the service corresponding to the complaint event. For example, when the complaint event is an inability to access the Internet, the service corresponding to the complaint event includes at least data services, etc. By analyzing the mobile network service data of the complaining user within the historical time period, it can be determined whether the service status corresponding to the complaint event is in an abnormal state, for example, whether there are abnormalities in information such as APN contract status, shutdown and resumption information, 4G / 5G services, and speed limits.

[0092] When the business status corresponding to the complaint event is in an abnormal state, the tracing result of the user contract data problem is output and the process ends.

[0093] If the at least one mobile network operation data and / or mobile network service data include both the at least one mobile network operation data and the mobile network service data, the embodiment of the present application can first determine whether the service status corresponding to the complaint event is in an abnormal state in the above manner, and when the service status corresponding to the complaint event is in a normal state, match the at least one mobile network operation data with the corresponding tracing rule to obtain at least one intermediate tracing result. Then, based on the at least one intermediate tracing result, the final tracing result is obtained.

[0094] If the at least one mobile network operation data and / or mobile network service data only includes the at least one mobile network operation data, the embodiment of the present application can match the at least one mobile network operation data with the corresponding tracing rule to obtain at least one intermediate tracing result. Then, based on the at least one intermediate tracing result, a final tracing result is obtained.

[0095] Here, the embodiment of the present application can obtain an intermediate tracing result for each mobile network operation data, and the intermediate tracing result is the root cause of the complaint event obtained based on the mobile network operation data, and can specifically include the delimitation result of the complaint event and / or the location result of the complaint event. Complaint events are usually caused by various problems, and here, the problems can also be referred to as faults, anomalies, etc. The delimitation result of a complaint event refers to the location of the problem that caused the complaint event, and the location result of a complaint event is usually the cause of the problem that caused the complaint event.

[0096] For example, the delimitation results of the complaint event include, but are not limited to: core network problems, wireless network problems, terminal problems, service provider problems, and terminal service client problems. Here, the core network problem means that the complaint event is caused by a problem / abnormality / failure of the core network. Similarly, the wireless network problem, terminal problem, service provider problem, and terminal service client problem respectively mean that the complaint event is caused by a problem / abnormality / failure of the wireless network, a problem / abnormality / failure of the terminal, a problem / abnormality / failure of the service provider, and a problem / abnormality / failure of the terminal service client.

[0097] The positioning results of the complaint event include but are not limited to: low TCP1 / 2 handshake success rate, extended TCP1 / 2 handshake, low HTTP response success rate, extended HTTP response, ping-pong switching, ping-pong tracking area update (Tracking Area Update, TAU), low attachment success rate, low service request success rate, low TAU success rate, low X2 switching success rate, frequent TAU, frequent switching, frequent attachment, low TCP2 / 3 handshake success rate, extended TCP2 / 3 handshake, high TCP retransmission rate, low HTTP large packet download rate, etc.

[0098] It should also be noted that in the embodiment of the present application, a parameter (such as low HTTP response success rate, low attachment success rate, low service request success rate, low TAU success rate, low X2 handover success rate, low TCP2 / 3 handshake success rate, TCP2 / 3 handshake time extension, high TCP retransmission rate, low HTTP large packet download rate, etc.) is low, high or long, which means that the parameter is lower than, higher than or longer than the corresponding threshold. HTTP large packet refers to an HTTP packet whose data packet length is greater than a preset length, for example, a packet greater than 500KB. The ping-pong handover means that the terminal switches from the second cell to the first cell within a preset first time after switching from the first cell to the second cell. The ping-pong TAU means that the tracking area is updated from the second tracking area to the first tracking area within a preset second time after the first tracking area is updated to the second tracking area. The frequent TAU, frequent handover, and frequent attachment refer to the number of TAU, handover, and attachment within a preset time period being greater than the corresponding threshold. The above-mentioned thresholds, thresholds, etc. can be set based on experience, and the embodiment of the present application does not make specific limitations on this.

[0099] The at least one intermediate tracing result is obtained by matching the at least one mobile network operation data with the corresponding tracing rules respectively. For example, a tracing rule corresponding to each mobile network operation data is established in advance, and then each mobile network operation data related to the first information extracted in step 12 is matched with the tracing rule corresponding to the mobile network operation data, so as to determine the tracing result corresponding to the mobile network operation data. The at least one mobile network operation data includes at least one of wireless network data, XDR data, and performance data. Each of them is explained below.

[0100] 1. Wireless network data

[0101] The wireless network data may specifically include at least one of alarm data, network optimization data, and MR data. Figure 2 The method shown determines whether the alarm data, network optimization data, and MR data are empty one by one, and if the current data is not empty, performs traceability processing (delimitation and positioning processing) based on the current data; otherwise, determines whether the next data is empty, until all data processing is completed.

[0102] For example, when the at least one mobile network operation data includes the wireless network data, obtaining the intermediate tracing result based on the wireless network data may include at least one of the following:

[0103] (1) If the wireless network data includes alarm data, the alarm data related to the historical cell in the historical time period is matched with the rules in the alarm rule base to determine the alarm information in the historical time period. These alarm information are intermediate tracing results obtained based on the alarm data. The alarm information includes but is not limited to: GPS alarm, standing wave alarm, RRU closure, RRU decommissioning, base station closure, base station decommissioning, base station decommissioning, cell decommissioning, cell closure.

[0104] For example, it is assumed that the historical time period includes a first time period and a second time period that are continuous in time, and the first time period is earlier than the second time period. In the case where the complaining user accesses the first historical cell during the first time period and accesses the second historical cell during the second time period, at this time, the alarm data related to the first historical cell during the first time period can be matched with the first alarm rule base to determine the alarm information during the first time period. The first alarm rule base includes matching rules for multiple alarm types, and the multiple alarm types include but are not limited to alarm types for cell exit from service, etc.

[0105] And, matching the alarm data related to the second historical cell in the second time period with the second alarm rule base to determine the alarm information in the second time period. The second alarm rule base may only include matching rules for the alarm type of cell exiting service.

[0106] Since in actual network scenarios, the alarm data of a user entering from one cell to another cell is usually related to the cell exiting the service, the embodiment of the present application sets up a second alarm rule base with fewer matching rules through the above method. When the user enters the second historical cell from the first historical cell, the alarm data related to the second historical cell in the second time period is matched with the matching rules of the alarm type of the cell exiting the service, which can reduce the matching calculation amount and improve the efficiency of alarm type matching.

[0107] Specifically, the above-mentioned alarm rule base can be set based on expert experience. Usually, the first alarm rule base includes at least one of the following alarm matching rules: GPS alarm, standing wave alarm, RRU closure, RRU decommissioning, base station closure, base station decommissioning, base station decommissioning, cell decommissioning, cell closure and other alarm rules. Decommissioning refers to exiting the service. By matching the alarm data related to the historical cell in the historical time period with each alarm rule in the first alarm rule base in turn, it is determined whether there is a corresponding alarm. The second alarm rule base only includes alarm rules for cell decommissioning.

[0108] Taking the user complaint time [a, b] as an example, since some serious alarms that affect user perception may occur before the user accesses the cell, therefore, combined with the actual alarm processing requirements, when pulling the alarm data, the alarm data ([a-4h, a]) within the first preset time (4 hours) before the user accesses the cell will be pulled, and it will be matched with the general rule base (first alarm rule base). In particular, considering that the cell decommissioning alarm may be reported only after the user switches out of the cell, therefore, the alarm data ([b, b+30min]) within the second preset time (such as 30 minutes) after the user switches the cell is pulled, and it is matched and screened with the rule base (second alarm rule base) that only contains cell decommissioning alarms.

[0109] (2) If the wireless network data includes network optimization data, the network optimization data related to the historical cell within the historical time period is matched with the judgment rules of abnormal network indicators to determine the abnormal network indicators. The abnormal network indicators may specifically include high load, high interference, low E-RAB establishment success rate, low RRC connection establishment success rate, high maximum number of RRC connections, high paging congestion rate, high wireless disconnection rate and other indicators. These abnormal network indicators are intermediate tracing results obtained based on the network optimization data.

[0110] (3) If the wireless network data includes MR data, the MR data related to the historical cell in the historical time period is matched with the judgment rule of abnormal network coverage to determine abnormal coverage information, such as cross-area coverage, weak coverage, coverage hole, etc. Such abnormal coverage information is the intermediate tracing result obtained based on the MR data.

[0111] 2. XDR data and / or performance data

[0112] In the case where the at least one mobile network operation data includes XDR data and / or performance data, obtaining the intermediate tracing result based on the XDR data and / or performance data may include at least one of the following:

[0113] (1) Match the XDR data and / or performance data related to the historical cell during the historical time period with the demarcation rules used to determine the location of the complaint event to obtain the demarcation results of the complaint event. These demarcation results are intermediate tracing results obtained based on the XDR data and / or performance data. Here, the demarcation rules include at least one of the following: demarcation rules for core network problems, demarcation rules for wireless network problems, demarcation rules for terminal problems, demarcation rules for service provider problems, demarcation rules for terminal service client problems, etc. Correspondingly, the demarcation results of the complaint event include at least one of the following: core network problems, wireless network problems, terminal problems, service provider problems, terminal service client problems, etc.

[0114] (2) Matching the XDR data and / or performance data related to the historical cell in the historical time period with the positioning rules used to determine the cause of the complaint event, and obtaining the positioning result of the complaint event. These positioning results are also intermediate tracing results obtained based on the XDR data and / or performance data. Here, the positioning rules include at least one of the following: positioning rules for TCP connections, positioning rules for HTTP connections, positioning rules for handovers, positioning rules for control plane processing, etc. Correspondingly, the positioning result of the complaint event includes at least one of the following: low TCP1 / 2 handshake success rate, extended TCP1 / 2 handshake, low HTTP response success rate, extended HTTP response, ping-pong handover, ping-pong TAU, low attachment success rate, low service request success rate, low TAU success rate, low X2 handshake success rate, frequent TAU, frequent handover, frequent attachment, low TCP2 / 3 handshake success rate, extended TCP2 / 3 handshake, high TCP retransmission rate, low HTTP large packet download rate, etc.

[0115] It should be noted that the matching rules of alarm types, the judgment rules of abnormal indicators, the judgment rules of network abnormality coverage, the delimitation rules, and the positioning rules of the embodiments of the present application may be rules set in advance based on expert experience, or may be rules learned based on historical data through a machine learning model. The embodiments of the present application do not make specific limitations on this.

[0116] When obtaining the final tracing result based on the at least one intermediate tracing result, the embodiment of the present application can be implemented in multiple ways.

[0117] As one implementation method, an embodiment of the present application can obtain first input data based on the at least one intermediate tracing result, and then use the first input data as the input of a first model to obtain the final tracing result output by the first model, wherein the first model is a machine learning model pre-trained with first training data.

[0118] When the first input data is obtained based on the at least one intermediate tracing result, the embodiment of the present application can form the first input data with the intermediate tracing result obtained in step 13 according to a preset first format. Then, the first input data is input into the first model. The first model extracts a feature vector from the first input data, and generates a final tracing result based on the extracted feature vector.

[0119] For example, assuming that multiple intermediate tracing results are obtained in step 13, a first input data composed of these intermediate tracing results in a preset order is:

[0120] 'TCP1 / 2 handshake success rate is low', 'TCP1 / 2 handshake is extended', 'HTTP response is extended', 'HTTP response success rate is low', 'TCP23 handshake success rate is low', 'TCP23 handshake is extended', 'TCP retransmission rate is high', 'HTTP downlink rate is low', 'Ping-Pong handover', 'Ping-Pong TAU', 'Control plane attachment success rate is low', 'Control plane Service Request success rate is low', 'Control plane TAU success rate is low', 'Control plane X2 handover success rate is low', 'High load', 'High interference', 'E-RAB establishment success rate is low', 'RRC connection establishment success rate is low', 'High maximum number of RRC connections', 'High paging congestion rate', 'High wireless drop rate'.

[0121] The * between '*' represents an intermediate traceability result.

[0122] When the first input data is obtained based on the at least one intermediate tracing result, the embodiment of the present application can also count the number of occurrences of the preset type of intermediate tracing results based on the intermediate tracing result obtained in step 13, for example, the number of occurrences of weak coverage, the number of various alarms, etc. Then, according to the preset second format, the intermediate tracing result obtained in step 13 and the number of occurrences of the preset type of intermediate tracing results are combined into the first input data.

[0123] For example, the number of occurrences of 'weak coverage', the number of occurrences of 'standing wave alarm', the number of occurrences of 'core network problem', the number of occurrences of 'terminal problem', etc. can be counted, and these statistical data and the intermediate tracing results are combined in a preset order to form a first input data:

[0124] 'TCP1 / 2 handshake success rate is low', 'TCP1 / 2 handshake is extended', 'HTTP response is extended', 'HTTP response success rate is low', 'TCP23 handshake success rate is low', 'TCP23 handshake is extended', 'TCP retransmission rate is high', 'HTTP downlink rate is low', 'Ping-Pong switching', 'Ping-Pong TAU', 'Control plane attachment success rate is low', 'Control plane Service Request success rate is low', 'Control plane TAU success rate is low', 'Control plane X2 handover success rate is low', 'High load', 'High interference', 'E-RAB establishment success rate is low', 'RRC connection establishment success rate is low', 'High maximum number of RRC connections', 'High paging congestion rate', 'High wireless drop rate', the number of occurrences of 'weak coverage', the number of occurrences of 'standing wave alarm', the number of occurrences of 'core network problem', and the number of occurrences of 'terminal problem'.

[0125] After obtaining the first input data, the first input data is input into the first model. The first model extracts a feature vector from the first input data, and generates a final traceability result based on the extracted feature vector. The first model can encode the first traceability data in the first input data in a one-hot encoding manner to generate its corresponding features. Of course, it can also extract features from the text content of the first traceability data, which is not specifically limited in the embodiment of the present application.

[0126] It should be noted that the embodiment of the present application pre-uses the first training data to train a first model for generating the final tracing result. The first training data can be obtained based on historical complaint information. For example, the first historical information is extracted from the historical complaint information, and the first historical information is used to indicate at least one of the historical complaint user, the time of occurrence of the historical complaint event, and the location of the historical complaint event, and then at least one historical mobile network operation data related to the first historical information is extracted. Then, based on the at least one historical mobile network operation data, at least one first historical tracing result is obtained, and based on the at least one first historical tracing result, a first historical input data is obtained, and the first historical input data is used as a first training sample. The method for obtaining the historical input data is the same as the method for obtaining the first input data, and will not be repeated here. In addition, the label corresponding to the historical complaint information is obtained, and the label is used to indicate the final tracing result corresponding to the historical complaint information. The label can be labeled according to the real root cause of the historical complaint information. In the above manner, a first training data including the first training sample and the label can be obtained. Here, the historical complaint information is the historical complaint description information collected in advance. The content or acquisition method of the first historical information, historical mobile network operation data, and first historical tracing results can refer to the content or acquisition method of the first information, mobile network operation data, and intermediate tracing results mentioned above, and will not be repeated here.

[0127] A first model can be trained through a first training set consisting of multiple copies of the first training data. Specifically, the first training sample is input into the first model, and the first model generates a first eigenvalue based on the first training sample, and then predicts a traceability result based on the first eigenvalue. The difference between the predicted traceability result and the label is calculated, and the network parameters of the first model are optimized with the goal of minimizing the difference until the preset training end condition is reached, thereby obtaining a trained first model. The first model extracts the first eigenvalue from the first training sample, and then generates a traceability result based on the first eigenvalue.

[0128] As another implementation method, the embodiment of the present application can obtain the first input data based on the at least one intermediate tracing result. The generation method of the first input data can refer to the method above and will not be repeated here. In addition, at least one network status data is extracted from the at least one mobile network operation data and / or mobile network service data, and the at least one network status data is encoded to obtain the second input data. The network status data is the status data of the network accessed by the user, and may specifically include data on data rate, signal coverage, status of the services provided by the network, etc. Then, the first input data and the second input data are used as inputs to the second model to obtain the final tracing result output by the second model, wherein the second model is a machine learning model pre-trained with second training data.

[0129] For example, assume that the XDR data provides the following network status data. It should be noted that these are only some examples of network status data and are not intended to limit the present application: (1) Internet disconnection; (2) All websites, games, videos, apps and other software are disconnected; (3) The area signal is out of coverage; (4) The area has weak coverage; (5) The area has coverage holes; (6) The area has a signal but the download speed is slow; (7) The area has a signal but all websites, games, videos, apps and other software are slow and stuck; (8) The area has a signal but the download speed is slow and stuck. All websites, games, videos, apps and other software cannot be used; (9) There is a signal in the area but the online game is stuck; (10) There is a signal in the area but the online chat is slow; (11) There is a signal in the area but the download is not possible; (12) There is a signal in the area but the online game cannot be played; (13) There is a signal in the area but the online chat cannot be done; (14) There is a signal in the area but the webpage cannot be browsed; (15) There is a signal in the area but the video cannot be watched; (16) There is a signal in the area but the webpage is browsed slowly; (17) There is a signal in the area but the video is watched slowly or stuck; (18) The network speed is slow or the webpage cannot be opened. Here, the above-mentioned signal usually refers to a wireless signal, and the above-mentioned various description information can be obtained based on data statistics such as user feedback information and / or measurement data, which is not specifically limited in the present embodiment of the application. The above-mentioned network status data is the specific data that XDR can provide, and the area can be a historical cell related to the complaint event. The judgment of whether the area has a signal, whether the webpage can be browsed, whether the webpage browsing is slow, etc. can be judged according to the preset judgment conditions. Since the present embodiment of the application does not need to pay attention to how these data are generated, but only uses these data as an input of the second model, the generation of these data will not be described in detail.

[0130] The embodiment of the present application can predetermine at least one predetermined type of network status data, and then extract the predetermined type of network status data from the at least one mobile network operation data, and generate the second input data according to the extraction result. For example, the above-mentioned 1st to 18th types of network status data are extracted from the at least one mobile network operation data, and one-hot encoding is performed according to whether the corresponding network status data is extracted, so as to generate the second input data. Each bit in the second input data corresponds to a type of network status data, and the value of each bit indicates whether the corresponding network status data is extracted. Assuming that the above-mentioned 18 types of network status data are extracted, the generated second input data is: 111111111111111111. Here, 1 is used to indicate that the corresponding type of network status data is extracted, and 0 is used to indicate that the corresponding type of network status data is not extracted. Assuming that the above-mentioned 1st to 2nd types of network status data are not extracted, and the above-mentioned 3rd to 18th types of network status data are extracted, the generated second input data is: 00111111111111111111.

[0131] After obtaining the first input data and the second input data, the first input data and the second input data are input into a second model. The second model extracts feature vectors from the first input data and the second input data, and generates a final traceability result based on the extracted feature vectors.

[0132] Similarly, the embodiment of the present application uses the second training data in advance to train a second model for generating the final traceability result. The second training data can be obtained based on historical complaint information. For example, the first historical information is extracted from the historical complaint information, and the first historical information is used to indicate at least one of the historical complaint user, the time of occurrence of the historical complaint event, and the location of the historical complaint event, and then at least one historical mobile network operation data related to the first historical information is extracted. Then, based on the at least one historical mobile network operation data, at least one first historical traceability result is obtained, and based on the at least one first historical traceability result, a first historical input data is obtained. In addition, the predetermined type of network status data is extracted from the at least one historical mobile network operation data, so that a second training sample can be constructed based on the first historical input data and the network status data extracted from the at least one historical mobile network operation data. In addition, the label corresponding to the historical complaint information is obtained, and the label is used to indicate the final traceability result corresponding to the historical complaint information. The label can be labeled according to the real root cause of the historical complaint information. In the above manner, a second training data including the second training sample and the label can be obtained. Here, the historical complaint information is the historical complaint description information collected in advance. The content or acquisition method of the first historical information, historical mobile network operation data, and first historical tracing results can refer to the content or acquisition method of the first information, mobile network operation data, and intermediate tracing results mentioned above, and will not be repeated here.

[0133] A second model can be trained through a second training set consisting of multiple copies of the second training data. Specifically, the second training sample is input into the second model. The second model generates a first eigenvalue based on the second training sample, and then predicts a traceability result based on the second eigenvalue. By calculating the difference between the predicted traceability result and the label, and minimizing the difference as the goal, the network parameters of the second model are optimized until the preset training end condition is reached, and a trained second model is obtained. The first model extracts the second eigenvalue from the second training sample, and then generates a traceability result based on the second eigenvalue.

[0134] In some scenarios, the user complaint may not be caused by abnormalities / failures of the mobile network. For example, user complaints caused by users owing fees or not subscribing to specific services are not caused by the mobile network itself, so there is no need to perform the above-mentioned tracing process. Figure 1 The traceability process shown.

[0135] Specifically, Figure 3As shown, another embodiment of the present application provides a method for tracing the source of a complaint problem, including:

[0136] Step 31, perform text preprocessing on the complaint description information to obtain a first text.

[0137] Here, the text preprocessing includes but is not limited to: removing irrelevant characters, punctuation marks, HTML tags, etc.; word segmentation; converting the text into a unified format; removing stop words; encoding. The first text obtained after the text preprocessing usually includes multiple sentences.

[0138] Step 32: Obtain summary information of the first text, where the summary information includes at least one sentence.

[0139] Here, the importance index of each sentence in the first text can be obtained. For example, the ROUGE-1 index can be used as the importance index. According to the importance index, the candidate sentences are determined. For example, the sentences whose importance index meets the threshold requirement are selected as candidate sentences. Then, the candidate sentences are screened using preset keywords to obtain key sentences. For example, a plurality of keywords related to network problems are pre-set, and whether the candidate sentence is a key sentence is determined based on whether the number of keywords in the candidate sentence reaches a preset quantity threshold. The key sentences in the first text are shielded to obtain a second text. The second text is then input into a pre-trained language model to obtain a text generated by the pre-trained language model as the summary information of the first text. The summary information usually includes at least one sentence.

[0140] Step 33: classify the sentences in the summary information to obtain at least one classification.

[0141] Here, the summary information may be preprocessed to obtain a document D including at least one sentence. The text preprocessing method may refer to the above. The document D may be represented as {d 1 ,d 2 ,…,d i}. Among them, d i represents the i-th sentence in document D, where it is assumed that document D contains i sentences in total. Then, for each sentence in document D, the weight of each word in the document in the sentence is calculated, and the weight of each word calculated is combined into a vector to obtain the vector of sentence d. Among them, the weight W of word t in sentence d is td It can be calculated by the following formula:

[0142] W td =tft td +logn / df t +tf td·af t / n

[0143] Among them, tf td is the number of times word t appears in document D, n is the total number of documents in the pre-acquired corpus, and the corpus can be updated according to the generated summary information. For example, after generating a document for the summary information, the newly generated document is added to the corpus; df t is the number of documents in the corpus that contain the term t, af t is the total number of occurrences of word t in the corpus.

[0144] Then, clustering is performed based on the vectors of each sentence to obtain at least one classification. Optionally, the embodiment of the present application may perform clustering at least once, and after obtaining at least one classification through each clustering, calculate the silhouette coefficient of the classification result, and determine whether the silhouette coefficient meets the requirements of a preset coefficient threshold. If not, clustering is performed again until a clustering result that meets the requirements of the coefficient threshold is obtained as the final classification.

[0145] Step 34, obtaining the complaint category corresponding to each classification, wherein the complaint category includes network problems and non-network problems.

[0146] Here, the complaint category corresponding to each category can be determined based on expert experience. For example, based on expert experience, keywords corresponding to network problems and non-network problems can be set. If a category contains keywords corresponding to network problems, the complaint category of this category is determined to be network problems; if a category only contains keywords corresponding to non-network problems, the complaint category of this category is determined to be non-network problems.

[0147] Step 35, if the at least one complaint category includes network problems, proceed to step 11, at which point the complaint can be solved by executing Figure 1 Steps 11-13 are performed to obtain the final traceability result.

[0148] Step 36, when the complaint categories of at least one classification do not include network-related issues, output a prompt message that the current complaint is a non-network-related issue and end the process.

[0149] Through the above steps, the embodiment of the present application can give a prompt message that the current complaint is a non-network problem when the user complaint is not a network problem, and no longer perform tracing processing to reduce unnecessary consumption of tracing processing resources.

[0150] After obtaining the traceability results, the embodiment of the present application may further display the traceability results for user viewing.

[0151] Figure 4 An example diagram of a display interface of the traceability results obtained by the traceability method of an embodiment of the present application is shown, which includes two major parts: a complaint work order and a demarcation result. In the complaint work order part, the work order number, basic work order information and basic user information of the complaining user are displayed. Among them, the basic work order information can generally include the work order push time and complaint description information. Basic user information generally includes: IMEI and terminal information (such as the brand and model of the terminal device, etc.). In addition, a "raw data download" function button is also provided. By clicking this button, at least one mobile network operation data related to the complaint description information can be displayed, such as wireless network data (alarm data, network optimization data, MR data), XDR data, performance data, etc., and the above-mentioned mobile network operation data can also be downloaded.

[0152] The demarcation result section shows the tracing results of the complaint ticket, specifically the demarcation results, including user contract data issues, core network issues, wireless network issues, terminal issues, service provider issues, and terminal service client issues. Figure 4 The delimitation result displayed in the figure is: Wireless network problem. In addition, the details of the cells that the user has passed through in the delimitation result section also display all the cells that the user has passed through and the details of the cells in the form of a map. For example, different colors are used to distinguish whether the cells that the user has passed through are abnormal. Red represents an abnormality in the cell, and blue represents no abnormality in the cell. Clicking a cell on the map will display the corresponding cell details. The fault details can further display the following content, such as the faulty cell, abnormality type, abnormality cause, fault level, fault time, and recommended measures. In addition, a "View delimitation basis" function button is provided. By clicking this button, you can jump to the delimitation basis page. Users can view the basis and conclusion of the delimitation, which is convenient for users to identify the accuracy of the system's delimitation results.

[0153] The above describes various methods of the embodiments of the present application. The following further provides devices for implementing the above methods.

[0154] Please refer to Figure 5 The embodiment of the present application also provides a complaint problem tracing device, including:

[0155] A first extraction module 501 is used to extract first information from the complaint description information, where the first information is used to indicate at least one of the complaining user, the time when the complaint event occurred, and the location where the complaint event occurred;

[0156] A first determining module 502, configured to determine at least one type of mobile network operation data and / or mobile network service data related to the first information;

[0157] The first processing module 503 is used to obtain a tracing result based on the at least one mobile network operation data and / or mobile network service data.

[0158] Through the above modules, the embodiments of the present application can improve the accuracy of traceability results.

[0159] Optionally, the above device also includes: a display module, used to display the tracing result.

[0160] Optionally, the at least one mobile network operation data includes at least one of wireless network data, external data representation XDR data, and performance data; the wireless network data includes at least one of alarm data, network optimization data, and measurement report MR data; the mobile network service data includes at least one of user service contract status, user speed limit information, shutdown / resumption information, and arrears information.

[0161] Optionally, the first determining module includes:

[0162] A first obtaining module is used to obtain a historical time period related to the complaint event according to the occurrence time of the complaint event, wherein the occurrence time of the complaint event is a subset of the historical time period;

[0163] The second acquisition module is used to obtain a historical cell related to the complaint event based on the location where the complaint event occurred, wherein the location where the complaint event occurred is within the coverage of the historical cell; extract at least one mobile network operation data related to the historical cell within the historical time period; and / or extract the mobile network service data of the complaining user within the historical time period.

[0164] Optionally, the first processing module includes:

[0165] A second determination module is used to determine whether the service status corresponding to the complaint event is in an abnormal state according to the mobile network service data of the complaining user within the historical time period, when the at least one mobile network operation data and / or mobile network service data includes the mobile network service data;

[0166] The first output module is used to output the tracing results of the user contract data problem and end the process when the business status corresponding to the complaint event is in an abnormal state.

[0167] Optionally, the first processing module further includes:

[0168] A first matching module is used for, when the at least one mobile network operation data and / or the mobile network service data further includes at least one mobile network operation data, and when the service status corresponding to the complaint event is in a normal state, matching the at least one mobile network operation data with the corresponding tracing rule respectively to obtain at least one intermediate tracing result;

[0169] The second processing module is used to obtain a final tracing result based on the at least one intermediate tracing result.

[0170] Optionally, the first processing module includes:

[0171] A second matching module is used to match the at least one mobile network operation data with the corresponding tracing rules respectively to obtain at least one intermediate tracing result when the at least one mobile network operation data and / or the mobile network service data only includes at least one mobile network operation data;

[0172] The third processing module is used to obtain a final tracing result based on the at least one intermediate tracing result.

[0173] Optionally, when the at least one mobile network operation data includes the wireless network data, the first matching module or the second matching module matches the at least one mobile network operation data with the corresponding tracing rule respectively to obtain at least one intermediate tracing result, including at least one of the following:

[0174] Matching the alarm data related to the historical cell in the historical time period with the rules in the alarm rule base to determine the alarm information in the historical time period;

[0175] Matching the network optimization data related to the historical cell in the historical time period with the judgment rule of abnormal network indicators to determine the abnormal network indicators and / or the number of occurrences of the abnormal network indicators;

[0176] The MR data related to the historical cell in the historical time period is matched with a judgment rule of abnormal network coverage to determine abnormal coverage information.

[0177] Optionally, the first matching module or the second matching module is further used to:

[0178] Matching alarm data related to the first historical cell in a first time period with a first alarm rule base to determine alarm information in the first time period;

[0179] Matching the alarm data related to the second historical cell in the second time period with the second alarm rule base to determine the alarm information in the second time period;

[0180] Among them, the complaining user accessed the first historical cell during the first time period and accessed the second historical cell during the second time period; the first alarm rule base includes matching rules for multiple alarm types, and the second alarm rule base includes matching rules for alarm types of cell exit service.

[0181] Optionally, the first matching module or the second matching module is further used to:

[0182] In the case where the at least one mobile network operation data includes XDR data and / or performance data, the at least one mobile network operation data is matched with corresponding tracing rules respectively to obtain at least one intermediate tracing result, including at least one of the following:

[0183] Matching the XDR data and / or performance data related to the historical cell in the historical time period with a delimitation rule for determining the location of the complaint event, to obtain a delimitation result of the complaint event;

[0184] The XDR data and / or performance data related to the historical cell in the historical time period are matched with a positioning rule for determining the cause of the complaint event to obtain a positioning result of the complaint event.

[0185] Optionally, the demarcation rule includes at least one of the following: a demarcation rule for core network problems, a demarcation rule for wireless network problems, a demarcation rule for terminal problems, a demarcation rule for service provider problems, and a demarcation rule for terminal service client problems;

[0186] The positioning rule includes at least one of the following: a positioning rule for a TCP connection, a positioning rule for an HTTP connection, a positioning rule for a handover, and a positioning rule for control plane processing.

[0187] Optionally, the second processing module includes:

[0188] A first obtaining module, configured to obtain first input data based on the at least one intermediate tracing result;

[0189] The first calling module is used to use the first input data as the input of the first model to obtain the final tracing result output by the first model, wherein the first model is a machine learning model pre-trained with the first training data.

[0190] Optionally, the second processing module includes:

[0191] A second obtaining module, configured to obtain first input data based on the at least one intermediate tracing result;

[0192] A third obtaining module is used to extract at least one network status data from the at least one mobile network operation data and / or the mobile network service data, encode the at least one network status data, and obtain second input data;

[0193] The second calling module is used to use the first input data and the second input data as inputs of a second model to obtain the final tracing result output by the second model, wherein the second model is a machine learning model pre-trained with second training data.

[0194] Optionally, the above device further includes:

[0195] A fourth obtaining module is used to perform text preprocessing on the complaint description information to obtain a first text;

[0196] A fifth obtaining module, configured to obtain summary information of the first text, wherein the summary information includes at least one sentence;

[0197] A classification module, used to classify the sentences in the summary information to obtain at least one category;

[0198] A sixth obtaining module is used to obtain the complaint category corresponding to each classification, wherein the complaint category includes network problems and non-network problems;

[0199] A fourth processing module, configured to trigger the first extraction module to execute the step of extracting the first information from the complaint description information when the at least one classified complaint category includes a network problem;

[0200] The fifth processing module is used to output a prompt message that the current complaint is a non-network problem and end the process when the complaint categories of at least one classification do not include network problems.

[0201] Optionally, the fifth obtaining module is further used for:

[0202] Obtaining importance indexes of each sentence in the first text, and determining candidate sentences according to the importance indexes; screening the candidate sentences using keywords to obtain key sentences; shielding the key sentences in the first text to obtain a second text;

[0203] The second text is input into a pre-trained language model to obtain summary information of the first text generated by the pre-trained language model.

[0204] Optionally, the classification module is further used to:

[0205] Performing text preprocessing on the summary information to obtain a document including at least one sentence;

[0206] For each sentence in the document, respectively calculate the weight of each word in the document in the sentence, and form a vector by combining the calculated weights of each word to obtain the vector of the sentence;

[0207] Clustering is performed based on the vectors of each sentence to obtain at least one classification.

[0208] It should be noted that the device in this embodiment is a device corresponding to the above method, and the implementation methods in the above embodiments are all applicable to the embodiments of the device, and can also achieve the same technical effects. The above device provided in the embodiment of the present application can implement all the method steps implemented in the above method embodiment, and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment in this embodiment will not be specifically described here.

[0209] Another embodiment of the present application is a device for tracing the source of a complaint problem, such as Figure 6 As shown, it includes a transceiver 610, a processor 600, a memory 620, and a program or instruction stored in the memory 620 and executable on the processor 600; when the processor 600 executes the program or instruction, each process of the embodiment of the method for tracing the source of the above-mentioned complaint problem is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0210] The transceiver 610 is used to receive and send data under the control of the processor 600 .

[0211] Among them, Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 600 and memory represented by memory 620. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 610 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 may store data used by the processor 600 when performing operations.

[0212] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned complaint problem tracing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0213] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned complaint problem tracing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0214] It should be noted that, in this article, the terms "include", "include" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0215] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), including a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0216] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for tracing the source of a complaint problem, characterized in that: include: Extracting first information from the complaint description information, where the first information is used to indicate at least one of a complaining user, an occurrence time of the complaint event, and an occurrence location of the complaint event; Determining at least one mobile network operation data and / or mobile network service data related to the first information; Based on the at least one mobile network operation data and / or mobile network service data, a tracing result is obtained.

2. The method according to claim 1, characterized in that Also includes: The tracing result is displayed.

3. The method according to claim 1, characterized in that: The at least one mobile network operation data includes at least one of wireless network data, external data representation XDR data, and performance data; the wireless network data includes at least one of alarm data, network optimization data, and measurement report MR data; the mobile network service data includes at least one of user service contract status, user speed limit information, shutdown / resumption information, and arrears information.

4. The method according to claim 3, characterized in that The determining of at least one mobile network operation data and / or mobile network service data related to the first information includes: According to the occurrence time of the complaint event, a historical time period related to the complaint event is obtained, wherein the occurrence time of the complaint event is a subset of the historical time period; According to the location where the complaint event occurred, a historical cell related to the complaint event is obtained, wherein the location where the complaint event occurred is within the coverage of the historical cell; at least one mobile network operation data related to the historical cell within the historical time period is extracted; and / or, mobile network service data of the complaining user within the historical time period is extracted.

5. The method according to claim 4, characterized in that In a case where the at least one mobile network operation data and / or mobile network service data includes the mobile network service data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data includes: Determining whether the service status corresponding to the complaint event is in an abnormal state according to the mobile network service data of the complaining user in the historical time period; When the business status corresponding to the complaint event is in an abnormal state, the tracing result of the user contract data problem is output and the process ends.

6. The method according to claim 5, characterized in that In a case where the at least one mobile network operation data and / or mobile network service data further includes at least one mobile network operation data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data further includes: When the service status corresponding to the complaint event is in a normal state, matching the at least one mobile network operation data with the corresponding tracing rule respectively to obtain at least one intermediate tracing result; Based on the at least one intermediate tracing result, a final tracing result is obtained.

7. The method according to claim 4, characterized in that In a case where the at least one mobile network operation data and / or mobile network service data only includes at least one mobile network operation data, obtaining the tracing result based on the at least one mobile network operation data and / or mobile network service data includes: Matching the at least one mobile network operation data with the corresponding tracing rules respectively to obtain at least one intermediate tracing result; Based on the at least one intermediate tracing result, a final tracing result is obtained.

8. The method according to claim 6 or 7, characterized in that: In the case where the at least one mobile network operation data includes the wireless network data, the matching of the at least one mobile network operation data with the corresponding tracing rules to obtain at least one intermediate tracing result includes at least one of the following: Matching the alarm data related to the historical cell in the historical time period with the rules in the alarm rule base to determine the alarm information in the historical time period; Matching the network optimization data related to the historical cell in the historical time period with the judgment rule of abnormal network indicators to determine the abnormal network indicators and / or the number of occurrences of the abnormal network indicators; The MR data related to the historical cell in the historical time period is matched with a judgment rule of abnormal network coverage to determine abnormal coverage information.

9. The method according to claim 8, characterized in that The step of matching the alarm data related to the historical cell in the historical time period with the rules in the alarm rule base to determine the alarm information in the historical time period includes: Matching alarm data related to the first historical cell in a first time period with a first alarm rule base to determine alarm information in the first time period; Matching the alarm data related to the second historical cell in the second time period with the second alarm rule base to determine the alarm information in the second time period; Among them, the complaining user accessed the first historical cell during the first time period and accessed the second historical cell during the second time period; the first alarm rule base includes matching rules for multiple alarm types, and the second alarm rule base includes matching rules for alarm types of cell exit service.

10. The method according to claim 6 or 7, characterized in that: In the case where the at least one mobile network operation data includes XDR data and / or performance data, the matching of the at least one mobile network operation data with the corresponding tracing rules to obtain at least one intermediate tracing result includes at least one of the following: Matching the XDR data and / or performance data related to the historical cell in the historical time period with a delimitation rule for determining the location of the complaint event, to obtain a delimitation result of the complaint event; The XDR data and / or performance data related to the historical cell in the historical time period are matched with a positioning rule for determining the cause of the complaint event to obtain a positioning result of the complaint event.

11. The method according to claim 10, characterized in that The demarcation rules include at least one of the following: demarcation rules for core network problems, demarcation rules for wireless network problems, demarcation rules for terminal problems, demarcation rules for service provider problems, and demarcation rules for terminal service client problems; The positioning rule includes at least one of the following: a positioning rule for a TCP connection, a positioning rule for an HTTP connection, a positioning rule for a handover, and a positioning rule for control plane processing.

12. The method according to claim 6 or 7, characterized in that: The obtaining a final traceability result based on the at least one intermediate traceability result includes: Based on the at least one intermediate tracing result, obtaining first input data; The first input data is used as the input of the first model to obtain the final tracing result output by the first model, wherein the first model is a machine learning model pre-trained with the first training data.

13. The method according to claim 6 or 7, characterized in that: The obtaining a final traceability result based on the at least one intermediate traceability result includes: Based on the at least one intermediate tracing result, obtaining first input data; Extracting at least one type of network status data from the at least one type of mobile network operation data and / or mobile network service data, and encoding the at least one type of network status data to obtain second input data; The first input data and the second input data are used as inputs of a second model to obtain the final tracing result output by the second model, wherein the second model is a machine learning model pre-trained with second training data.

14. The method according to claim 1, characterized in that Also includes: Performing text preprocessing on the complaint description information to obtain a first text; Obtaining summary information of the first text, wherein the summary information includes at least one sentence; Classifying the sentences in the summary information to obtain at least one classification; Obtain the complaint category corresponding to each classification, where the complaint category includes network-related issues and non-network-related issues; In the case where the at least one classified complaint category includes network problems, entering the step of extracting the first information from the complaint description information; In the case that none of the complaint categories in the at least one classification includes network-related issues, a prompt message indicating that the current complaint is a non-network-related issue is output and the process ends.

15. The method according to claim 14, characterized in that The obtaining summary information of the first text includes: Obtaining importance indexes of each sentence in the first text, and determining candidate sentences according to the importance indexes; screening the candidate sentences using keywords to obtain key sentences; shielding the key sentences in the first text to obtain a second text; The second text is input into a pre-trained language model to obtain summary information of the first text generated by the pre-trained language model.

16. The method according to claim 14, characterized in that Classifying the sentences in the summary information includes: Performing text preprocessing on the summary information to obtain a document including at least one sentence; For each sentence in the document, respectively calculate the weight of each word in the document in the sentence, and form a vector by combining the calculated weights of each word to obtain the vector of the sentence; Clustering is performed based on the vectors of each sentence to obtain at least one classification.

17. A device for tracing the source of a complaint, characterized in that: include: A first extraction module, used to extract first information from the complaint description information, where the first information is used to indicate at least one of the complaining user, the time when the complaint event occurred, and the location where the complaint event occurred; A second determination module, configured to determine at least one type of mobile network operation data and / or mobile network service data related to the first information; The first processing module is used to obtain a tracing result based on the at least one mobile network operation data and / or mobile network service data.

18. A device for tracing the source of complaints, characterized in that: include: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the steps of the method as described in any one of claims 1 to 16 when executing the program or instruction.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 16.

Images