Acquisition method and device of high-frequency feedback problem and computing equipment

Abstract

The invention discloses a high-frequency feedback problem acquisition method and apparatus, and a computing device. The method comprises the following steps: acquiring customer service session data in a first historical time period; for any customer service session data, inputting the customer service session data and the generated question label prompt word into a customer complaint big model, and obtaining a feedback question label of the customer service session data output by the customer complaint big model; and aggregating the feedback problem labels of the customer service session data in the first historical time period, and extracting a high-frequency feedback problem of the first historical time period according to an aggregation result. By adopting the scheme, the automatic generation of the high-frequency feedback problem can be realized, and the acquisition precision and the acquisition efficiency of the high-frequency feedback problem can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device, computing equipment, computer storage medium and computer program product for obtaining high-frequency feedback problems. Background Art

[0002] With the continuous development of science and technology and society, the emergence of various Internet platforms has greatly facilitated people's work and life. In order to improve user experience, some Internet platforms provide customer service functions (referred to as customer service functions), through which users can feedback corresponding problems.

[0003] High-frequency feedback issues are issues that users frequently report through the customer service function. The analysis of high-frequency feedback issues can accurately locate platform failure points or points to be optimized. Therefore, obtaining high-frequency feedback issues is of great significance.

[0004] However, the inventors found during implementation that the prior art has at least the following defects: the prior art obtains high-frequency feedback problems by manual extraction, which results in low efficiency in obtaining high-frequency feedback problems in the prior art. Summary of the invention

[0005] In view of the above problems, the present application is proposed to provide a method, apparatus, computing device, computer storage medium and computer program product for obtaining high-frequency feedback problems that overcome the above problems or at least partially solve the above problems.

[0006] According to a first aspect of the present application, a method for obtaining a high-frequency feedback problem is provided, comprising:

[0007] Obtain customer service session data within the first historical period;

[0008] For any customer service conversation data, the customer service conversation data and the generated question label prompt are input into the customer complaint big model, and the feedback question label of the customer service conversation data output by the customer complaint big model is obtained;

[0009] Feedback question labels of each customer service session data in the first historical period are aggregated, and high-frequency feedback questions in the first historical period are extracted according to the aggregation result.

[0010] In an optional implementation, aggregating the feedback question labels of each customer service session data in the first historical period, and extracting the high-frequency feedback questions in the first historical period according to the aggregation result includes:

[0011] Generate a first tag set according to the feedback question tags of each customer service session data in the first historical period;

[0012] Inputting the first tag set, aggregation conditions, and high-frequency feedback problem extraction conditions into the customer complaint big model;

[0013] Obtain high-frequency feedback issues in the first historical period output by the customer complaint model.

[0014] In an optional implementation, aggregating the feedback question labels of each customer service session data in the first historical period, and extracting the high-frequency feedback questions in the first historical period according to the aggregation result includes:

[0015] Generate a second tag set based on the historical feedback question tags of the second historical period; wherein the second historical period is earlier than the first historical period;

[0016] Input any feedback question label within the first historical period, the second label set, and similar label extraction conditions into the customer complaint big model, and obtain a revised question label of the feedback question label output by the customer complaint big model;

[0017] An ordered set of labels is generated according to each correction problem label in the first historical period, and high-frequency feedback problems in the first historical period are extracted according to the ordered set of labels.

[0018] In an optional implementation, the similar tag extraction condition includes: if there is a historical feedback question tag in the second tag set whose similarity to the feedback question tag is greater than a similarity threshold, outputting the historical feedback question tag with the greatest similarity to the feedback question tag;

[0019] If there is no historical feedback question tag in the second tag set whose similarity with the feedback question tag is greater than the similarity threshold, the feedback question tag is output.

[0020] In an optional embodiment, the method further includes:

[0021] Generate a first set of questions based on the high-frequency feedback questions in the first historical period output by the customer complaint model, and generate a second set of questions based on the high-frequency feedback questions in the first historical period extracted by the ordered set of labels;

[0022] Calculating the set similarity between the first question set and the second question set;

[0023] A final high-frequency feedback question set for the first historical period is generated according to the set similarity, the first question set, and the second question set.

[0024] In an optional implementation, generating a final high-frequency feedback question set for the first historical period according to the set similarity, the first question set, and the second question set includes:

[0025] If the set similarity is greater than a first threshold, taking the first question set as the final high-frequency feedback question set;

[0026] If the set similarity is less than or equal to the first threshold, the second question set is used as the final high-frequency feedback question set.

[0027] In an optional implementation, after acquiring the customer service session data within the first historical period, the method further includes: preprocessing the customer service session data within the first historical period;

[0028] The step of inputting the customer service conversation data and the generated question label prompt words into the customer complaint big model comprises: inputting the pre-processed customer service conversation data and the generated question label prompt words into the customer complaint big model;

[0029] The preprocessing includes at least one of the following processing modes: non-text data elimination processing, non-text data conversion processing, and invalid text elimination processing.

[0030] According to a second aspect of the present application, a device for obtaining a high-frequency feedback problem is provided, comprising:

[0031] A session acquisition module, used to acquire customer service session data within a first historical period;

[0032] A label acquisition module is used to input the customer service session data and the generated question label prompt into the customer complaint model for any customer service session data, and obtain the feedback question label of the customer service session data output by the customer complaint model;

[0033] The aggregation module is used to aggregate the feedback question labels of each customer service session data in the first historical period, and extract the high-frequency feedback questions in the first historical period according to the aggregation result.

[0034] In an optional implementation, the aggregation module is used to: generate a first tag set according to feedback question tags of each customer service session data in the first historical period;

[0035] Inputting the first tag set, aggregation conditions, and high-frequency feedback problem extraction conditions into the customer complaint big model;

[0036] Obtain high-frequency feedback issues in the first historical period output by the customer complaint model.

[0037] In an optional implementation, the aggregation module is used to: generate a second tag set according to the historical feedback question tags of a second historical period; wherein the second historical period is earlier than the first historical period;

[0038] Input any feedback question label within the first historical period, the second label set, and similar label extraction conditions into the customer complaint big model, and obtain a revised question label of the feedback question label output by the customer complaint big model;

[0039] An ordered set of labels is generated according to each correction problem label in the first historical period, and high-frequency feedback problems in the first historical period are extracted according to the ordered set of labels.

[0040] In an optional implementation, the similar tag extraction condition includes: if there is a historical feedback question tag in the second tag set whose similarity to the feedback question tag is greater than a similarity threshold, outputting the historical feedback question tag with the greatest similarity to the feedback question tag;

[0041] If there is no historical feedback question tag in the second tag set whose similarity with the feedback question tag is greater than the similarity threshold, the feedback question tag is output.

[0042] In an optional implementation, the aggregation module is used to: generate a first question set based on the high-frequency feedback questions in the first historical period output by the customer complaint model, and generate a second question set based on the high-frequency feedback questions in the first historical period extracted by the ordered set of labels;

[0043] Calculating the set similarity between the first question set and the second question set;

[0044] A final high-frequency feedback question set for the first historical period is generated according to the set similarity, the first question set, and the second question set.

[0045] In an optional implementation, the aggregation module is used to: if the set similarity is greater than a first threshold, use the first question set as the final high-frequency feedback question set;

[0046] If the set similarity is less than or equal to the first threshold, the second question set is used as the final high-frequency feedback question set.

[0047] In an optional implementation, the device further includes: a preprocessing module, configured to preprocess the customer service session data within the first historical period;

[0048] The label acquisition module is used to: input the pre-processed customer service conversation data and the generated question label prompt words into the customer complaint model;

[0049] The preprocessing includes at least one of the following processing modes: non-text data elimination processing, non-text data conversion processing, and invalid text elimination processing.

[0050] According to a third aspect of the present application, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0051] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned method for obtaining high-frequency feedback problems.

[0052] According to a fourth aspect of the present application, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned method for obtaining the high-frequency feedback problem.

[0053] According to a fifth aspect of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned method for obtaining the high-frequency feedback problem.

[0054] The embodiment of the present application can obtain feedback question labels within the first historical period by processing customer service conversation data through the customer complaint big model, and further obtain high-frequency feedback problems by aggregating the feedback question labels within the first historical period, thereby realizing the automatic generation of high-frequency feedback problems and improving the acquisition accuracy and efficiency of high-frequency feedback problems.

[0055] The embodiment of the present application inputs the first label set, aggregation conditions, and high-frequency feedback problem extraction conditions generated based on the feedback labels of the first historical period into the customer complaint big model, which then aggregates and extracts the final high-frequency feedback problems, thereby improving the processing efficiency and accuracy of high-frequency feedback problems.

[0056] In the embodiment of the present application, the customer complaint model is used to obtain the corrected question labels for each feedback question label in the first historical period, unify the label expression, and improve the extraction accuracy of subsequent high-frequency feedback problems; and by correcting the question labels to obtain an ordered set of labels, it is possible to quickly extract high-frequency feedback problems according to the number of occurrences of the corrected question labels, thereby improving the extraction efficiency of high-frequency feedback problems.

[0057] The similarity label extraction condition in the embodiment of the present application can instruct the large model to return a label with a similarity greater than a similarity threshold and the highest similarity as a corrected problem label, and return the label itself as a corrected problem label when no similarity greater than the similarity threshold can be found, thereby improving the output accuracy of the corrected problem label and ensuring the accuracy of label aggregation.

[0058] The embodiment of the present application integrates the high-frequency feedback problems output by the large customer complaint model and the high-frequency feedback problems extracted from the ordered set of labels to obtain the final high-frequency feedback problems, thereby improving the extraction accuracy of the high-frequency feedback problems.

[0059] The embodiment of the present application generates a first question set based on the high-frequency feedback questions output by the customer complaint model, generates a second question set based on the high-frequency feedback questions extracted from the ordered set of labels, and selects the corresponding question set as the final high-frequency feedback question set under different set similarities, which is beneficial to improving the accuracy of obtaining the final high-frequency feedback questions.

[0060] After obtaining the customer service conversation data, the embodiment of the present application further pre-processes the data, thereby reducing the subsequent data processing volume and reducing interference with subsequent steps, and improving the accuracy of determining high-frequency feedback question labels.

[0061] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0063] Figure 1 A schematic diagram of an operating environment provided for implementing at least one embodiment of the present application is shown;

[0064] Figure 2 A flow chart showing a method for obtaining a high-frequency feedback problem provided in Embodiment 1 of the present application is shown;

[0065] Figure 3 A flow chart showing a method for obtaining a high-frequency feedback problem provided in Embodiment 2 of the present application is shown;

[0066] Figure 4 A flowchart of a method for obtaining a high-frequency feedback problem provided in Embodiment 3 of the present application is shown;

[0067] Figure 5 A flowchart of a method for obtaining a high-frequency feedback problem provided in Embodiment 4 of the present application is shown;

[0068] Figure 6 A structural diagram of a device for obtaining a high-frequency feedback problem provided in Embodiment 5 of the present application is shown;

[0069] Figure 7 A structural diagram of a computing device provided in Example 6 of the present application is shown. DETAILED DESCRIPTION

[0070] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0071] It should be noted that the customer service session data involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0072] Figure 1 The present invention is applicable to an application environment including, but not limited to, a client 2 , a server 4 , and a network 6 .

[0073] in:

[0074] The server 4 may be composed of a single or multiple computing devices. The multiple computing devices may include virtualized computing instances. Virtualized computing instances may include virtual machines, such as simulations of computer systems, operating systems, servers, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., operating system, dedicated application, server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0075] The server 4 may be configured to communicate with the client 2, etc., via a network 6. The network 6 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 6 may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, etc., or wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0076] The server 4 can provide storage, reading, downloading, writing, querying, deleting and other services, such as providing static resource download services to clients through multiple domain names.

[0077] Client 2 can be running Windows, Android TM) or IOS and other operating systems, such as smart phones, tablet devices, laptop computers, virtual reality devices, gaming devices, set-top boxes, car terminals, smart TVs. Based on the above operating systems, various applications, such as browsers, can be run.

[0078] Embodiment 1

[0079] Figure 2 A flowchart of a method for obtaining high-frequency feedback problems provided in Embodiment 1 of the present application is shown. The method provided in this embodiment can be executed on the server side.

[0080] Specifically, Figure 2 As shown, the method comprises the following steps:

[0081] Step S201, obtaining customer service session data within a first historical period.

[0082] The customer service session data specifically refers to the session data between the user and the customer service in the platform. The first historical period is the period before the current time, for example, the first historical period can be the most recent day, the most recent week, and so on.

[0083] In an optional implementation, in order to improve the accuracy and efficiency of obtaining high-frequency feedback issues, this implementation preprocesses the customer service conversation data within the first historical period after obtaining the customer service conversation data within the first historical period, and subsequently inputs the preprocessed customer service conversation data and the generated question label prompt words into the customer complaint model.

[0084] Further optionally, the preprocessing includes at least one of the following processing methods: non-text data elimination processing, non-text data conversion processing, and invalid text elimination processing. Among them, the non-text data elimination processing can eliminate non-text data (such as pictures, videos, emoticons, etc.) in the original customer service conversation data; the non-text data conversion processing can convert the non-text data in the original customer service conversation data into corresponding text data, and insert the converted text data into a matching position, for example, an emoticon package can be converted into an emoticon package name (a type of text data) and inserted into the position in the original emoticon package; the invalid text elimination processing is to eliminate invalid text in the original customer service conversation data, such as eliminating stop words, words without actual semantics, etc.

[0085] Step S202: for any customer service conversation data, the customer service conversation data and the generated question label prompt are input into the customer complaint big model, and the feedback question label of the customer service conversation data output by the customer complaint big model is obtained.

[0086] The customer complaint model is a large neural network model (Large Model, referred to as the large model) built based on a deep learning framework. The customer complaint model can generate results that meet the requirements based on relevant data and prompt words. This application uses the customer complaint model to process customer service conversation data, and the customer complaint model analyzes the user feedback issues involved in the customer service conversation data.

[0087] Specifically, for any customer service conversation data, the customer service conversation data and the prompt word for generating a problem label are input into the customer complaint big model. The prompt word for generating a problem label is a prompt word used to prompt the customer complaint big model to analyze the user feedback problem involved in the customer service conversation data. For example, the prompt word can be "Based on the above conversation, give this conversation a label: similar to [order problem], [logistics problem], or [ticket problem], and the core is to describe where the problem is."

[0088] After the customer service conversation data and the generated question label prompt words are input into the customer complaint big model, the customer complaint big model will output the feedback question label of the customer service conversation data through analysis. The feedback question label is used to describe the core problem of user feedback in the customer service conversation data.

[0089] The feedback question labels of each customer service conversation data in the first historical period can be obtained by processing each customer service conversation data in the first historical period through this step.

[0090] Step S203: Aggregate the feedback question labels of each customer service session data in the first historical period, and extract the high-frequency feedback questions in the first historical period according to the aggregation result.

[0091] Although the feedback problem labels initially output by the customer complaint model in step S202 can reflect the user feedback problems involved in the customer service conversation data, the number of feedback problem labels generated in the first historical period is relatively large, and there are many situations in the feedback problem labels that are essentially the same but expressed differently. In order to improve the acquisition accuracy and efficiency of high-frequency feedback problems, this application aggregates the feedback problem labels of each customer service conversation data in the first historical period, on the one hand to reduce the number of labels after aggregation, and on the other hand to reduce the interference of the problem expression on the essence of the problem. Finally, the high-frequency feedback problems of the first historical period are extracted according to the aggregation results. The high-frequency feedback problems are specifically feedback problems with high frequency in the first historical period after aggregation. For example, for any feedback problem label after aggregation, the corresponding frequency of occurrence of the feedback problem label in the first historical period can be counted, and the aggregated feedback problem labels can be sorted according to the frequency of occurrence, and the top N aggregated feedback problem labels with high frequency of occurrence are selected as the high-frequency feedback problems in the first historical period.

[0092] It can be seen that the method for obtaining high-frequency feedback problems provided in the embodiment of the present application can obtain feedback problem labels within the first historical period by processing customer service conversation data with the customer complaint big model, and further obtain high-frequency feedback problems by aggregating the feedback problem labels within the first historical period. This solution can realize the automatic generation of high-frequency feedback problems, and can improve the accuracy and efficiency of obtaining high-frequency feedback problems.

[0093] Embodiment 2

[0094] Figure 3 A flowchart of a method for obtaining high-frequency feedback problems provided in the second embodiment of the present application is shown. The method provided in this embodiment can be executed on the server side.

[0095] Specifically, Figure 3 As shown, the method comprises the following steps:

[0096] Step S301, obtaining customer service session data within a first historical period.

[0097] Step S302: for any customer service conversation data, the customer service conversation data and the generated question label prompt are input into the customer complaint big model, and the feedback question label of the customer service conversation data output by the customer complaint big model is obtained.

[0098] Step S303: Generate a first tag set according to the feedback question tags of each customer service session data in the first historical period.

[0099] The feedback question tags generated in step S302 in the first historical period are added to the tag set to obtain a first tag set. For example, all feedback question tags in the most recent day can be added to set C1 (first tag set), and two adjacent feedback question tags can be connected by commas or the like.

[0100] Step S304: input the first tag set, aggregation conditions, and high-frequency feedback problem extraction conditions into the customer complaint model.

[0101] The aggregation condition is specifically prompt information indicating under what circumstances the customer complaint model can aggregate multiple feedback tags in the first tag set into one tag. The aggregation condition can specifically be a similarity condition for tag aggregation, that is, the aggregation condition indicates that the feedback tags can be aggregated into one tag when the similarity of multiple feedback tags exceeds a corresponding threshold. For example, the aggregation condition can be "if the similarity of feedback tags exceeds 80%, they can be merged into the same tag."

[0102] The high-frequency feedback problem extraction condition is used to instruct the customer complaint model how to extract prompt information of high-frequency feedback problems based on the aggregated tags after aggregating the tags in the first tag set. The high-frequency feedback problem extraction condition specifically includes the extraction number and sorting rules. For example, the high-frequency feedback problem extraction condition can be "output the 5 questions with the highest frequency".

[0103] The first label set, aggregation conditions, and high-frequency feedback problem extraction conditions are input into the customer complaint big model for processing by the customer complaint big model.

[0104] Step S305, obtaining high-frequency feedback issues in the first historical period output by the customer complaint model.

[0105] The data output by the customer complaint model based on the input content in step S304 is used as the high-frequency feedback issues in the first historical period.

[0106] It can be seen that the method for obtaining high-frequency feedback problems provided in the embodiment of the present application can input the first label set generated according to the feedback labels of the first historical period, the aggregation conditions and the high-frequency feedback problem extraction conditions into the customer complaint big model, and the customer complaint big model will aggregate and extract the final high-frequency feedback problems, thereby improving the processing efficiency and processing accuracy of high-frequency feedback problems.

[0107] Embodiment 3

[0108] Figure 4 A flowchart of a method for obtaining high-frequency feedback problems provided in Embodiment 3 of the present application is shown. The method provided in this embodiment can be executed on the server side.

[0109] Specifically, Figure 4 As shown, the method comprises the following steps:

[0110] Step S401, obtaining customer service session data within a first historical period.

[0111] Step S402: for any customer service conversation data, the customer service conversation data and the generated question label prompt are input into the customer complaint big model, and the feedback question label of the customer service conversation data output by the customer complaint big model is obtained.

[0112] Step S403, generating a second tag set according to the historical feedback question tags of the second historical period; wherein the second historical period is earlier than the first historical period.

[0113] The feedback question tags generated earlier than the first historical period are used as historical feedback question tags, so as to generate a second tag set according to the historical feedback question tags.

[0114] In an optional implementation, if the feedback problem labels output by the customer complaint model in the second historical period are directly added to the second label set, the subsequent second label set capacity will be too large, and in this case the second label set will still contain many similar labels. In view of this, in this implementation, the historical feedback problem labels in the second historical period can be clustered to obtain the historical feedback problem labels after clustering, and then the second label set is generated based on the historical feedback problem labels after clustering.

[0115] The clustering process of historical feedback question labels may be performed in one or more of the following ways:

[0116] Clustering method 1: Obtain all feedback question labels in the second historical period, and use a preset clustering algorithm (such as a density-based clustering algorithm, a distance-based clustering algorithm, etc.) to cluster the feedback question labels to obtain multiple clusters. For each cluster, determine the cluster center of the cluster, use the feedback question label closest to the cluster center as the representative label of the cluster, and then add the representative label of each cluster to the second label set. This clustering method can improve the clustering efficiency of feedback question labels in the second historical period.

[0117] Clustering method two: Select the earliest sub-period from the second historical period, obtain the feedback question labels in the sub-period and add them to the second label set. The feedback question labels after the sub-period in the second historical period are used as labels to be stored in the order of generation. Calculate the similarity between the label to be stored and each label in the current second label set respectively, and determine whether there is a target label in the current second label set whose similarity with the label to be stored exceeds the set threshold; if not, add the label to be stored to the second label set, and use the next feedback question label as the label to be stored; if so, calculate the first average value of the similarity between the label to be stored and all labels in the second label set (that is, the first average value = the sum of the similarities between the label to be stored and each label in the second label set / the total number of labels in the current second label set), and calculate the second average value of the similarity between the target label and other labels in the second label set (that is, the second average value = the sum of the similarities between the target label and the second label set). The sum of the similarities of all tags except the target tag in the second tag set / (the total number of tags in the current second tag set-1)). If the first average value is less than the second average value, indicating that the tag to be stored is more different from the target tag and the tags in the second tag set, then the tag to be stored is added to the second tag set, the target tag is deleted from the second tag set, and the next feedback question tag is used as the tag to be stored; if the first average value is greater than or equal to the second average value, indicating that the tag to be stored is the same or lower than the target tag in the second tag set, then the tag to be stored is discarded (that is, the tag to be stored is not added to the second tag set), and the next feedback question tag is used as the tag to be stored. This method can not only streamline the number of tags in the second tag set and reduce the amount of subsequent data processing, but also ensure the diversity of tags in the second tag set and improve the accuracy of obtaining high-frequency feedback problems.

[0118] Step S404: input any feedback question label, the second label set, and similar label extraction conditions within the first historical period into the customer complaint big model to obtain a revised question label of the feedback question label output by the customer complaint big model.

[0119] This step is aimed at each feedback problem label in the first historical period, and the customer complaint big model is used to obtain the corrected problem label that matches it. Specifically, for each feedback problem label in any historical period, the feedback problem label, the currently generated second label set, and the similar label extraction conditions are input into the customer complaint big model, and the customer complaint big model outputs the corrected problem label of the feedback problem label.

[0120] Among them, the similarity tag extraction condition is a prompt information used to instruct the customer complaint model on how to generate a revised problem tag. Among them, in any feedback problem tag processing process within the first historical time window, the similarity tag extraction condition may include: if there is a historical feedback problem tag in the second tag set whose similarity with the feedback problem tag is greater than the similarity threshold, then the historical feedback problem tag with the greatest similarity to the feedback problem tag is output, so that the historical feedback problem tag with the greatest similarity is used as the revised problem tag of the feedback problem tag; if there is no historical feedback problem tag in the second tag set whose similarity with the feedback problem tag is greater than the similarity threshold, then the feedback problem tag is output, so that the feedback problem tag itself is used as its revised problem tag. For example, the similarity tag extraction condition may be "output a historical tag with a similarity with the current tag that is higher than the similarity threshold and has the highest similarity; if it cannot be found, return itself". For example, for feedback question label A in the first historical period, if there is a historical feedback problem label in the second label set whose similarity with label A is higher than 80%, then the historical feedback problem label B with the highest similarity with label A is returned as the revised question label of label A; if the similarity between each historical feedback problem label in the second label set and feedback problem label A does not exceed 80%, then label A is returned, that is, label A itself is the revised question label of label A.

[0121] In an optional implementation, if the revised question label of the feedback question label output by the customer complaint model is the same as the feedback question label, the feedback question label is added to the second label set, thereby updating and expanding the second label set and improving the generation accuracy of subsequent revised question labels.

[0122] Step S405 , generating an ordered set of labels according to the various correction problem labels in the first historical period, and extracting high-frequency feedback problems in the first historical period according to the ordered set of labels.

[0123] According to the implementation of step S404, the corrected question labels of each feedback question label in the first historical period can be obtained. In order to quickly determine the high-frequency feedback problems, a label ordered set can be generated according to each corrected question label, and the label ordered set is arranged in order according to the number of occurrences of the corrected question label (such as from high to low), so that the first N labels in the label ordered set can be quickly extracted as high-frequency feedback problems.

[0124] For example, the ordered set of labels can be ZSET, which is a set of string members, and it associates a score for each member, so that ZSET can be ordered by score. In this application, each non-repeated corrected problem label is used as a member of ZSET, and the number of occurrences of the corrected problem label in the first historical period is used as the score of the member. ZSET will automatically sort the corrected problem labels according to the frequency of occurrence, and then only the first N members need to be extracted to obtain the TOPN high-frequency feedback problems.

[0125] It can be seen that the method for obtaining high-frequency feedback problems provided in the embodiment of the present application obtains the corrected problem labels of each feedback problem label in the first historical period from the large customer complaint model, unifies the label expression, and improves the extraction accuracy of subsequent high-frequency feedback problems; and by correcting the problem labels to obtain an ordered set of labels, it is possible to quickly extract high-frequency feedback problems according to the number of occurrences of the corrected problem labels, thereby improving the extraction efficiency of high-frequency feedback problems.

[0126] Embodiment 4

[0127] Figure 5 A flowchart of a method for obtaining high-frequency feedback problems provided in Embodiment 4 of the present application is shown. The method provided in this embodiment can be executed on the server side.

[0128] Specifically, Figure 5 As shown, the method comprises the following steps:

[0129] Step S501: Generate a first set of questions based on high-frequency feedback questions in a first historical period output by the customer complaint model.

[0130] Refer to the steps in Example 2 to obtain the high-frequency feedback issues in the first historical period output by the customer complaint model, and add the high-frequency feedback issues in the first historical period output by the customer complaint model to the first issue set.

[0131] Step S502 , generating a second question set based on the high-frequency feedback questions in the first historical period extracted from the ordered set of tags.

[0132] Referring to the steps in the third embodiment, high-frequency feedback questions in the first historical period extracted from the ordered set of tags are obtained, and the high-frequency feedback questions in the first historical period extracted from the ordered set of tags are added to the second question set.

[0133] Step S503: Calculate the set similarity between the first question set and the second question set.

[0134] The present application does not limit the method for calculating set similarity. For example, the overlap of questions in the first question set and the second question set can be used as the set similarity. The question texts in the first question set can also be converted into a first text vector, and the question texts in the second question set can be converted into a second text vector, and the similarity between the first text vector and the second text vector can be used as the set similarity.

[0135] Step S504: Generate a final high-frequency feedback question set for the first historical period according to the set similarity, the first question set, and the second question set.

[0136] In an optional implementation, if the set similarity is greater than a first threshold (such as 50%), the first question set is used as the final high-frequency feedback question set; if the set similarity is less than or equal to the first threshold, the second question set is used as the final high-frequency feedback question set.

[0137] In an optional implementation, overlapping feedback questions in the first question set and the second question set are added to the final high-frequency feedback question set, and the first M feedback questions from the non-overlapping feedback questions in the first question set are selected to be added to the final high-frequency feedback question set, and the first Q feedback questions from the non-overlapping feedback questions in the second question set are selected to be added to the final high-frequency feedback question set, wherein if the set similarity is greater than a second threshold, M is greater than Q; if the set similarity is less than or equal to the second threshold, M is less than or equal to Q.

[0138] It can be seen that the method for obtaining high-frequency feedback problems provided in the embodiment of the present application integrates the high-frequency feedback problems output by the large customer complaint model and the high-frequency feedback problems extracted from the ordered set of labels to obtain the final high-frequency feedback problems, thereby improving the extraction accuracy of high-frequency feedback problems.

[0139] Embodiment 5

[0140] Figure 6 The structure diagram of a device for obtaining high-frequency feedback problems provided in the fifth embodiment of the present application is shown. Figure 6 As shown, the device 600 includes: a session acquisition module 610 , a tag acquisition module 620 , and an aggregation module 630 .

[0141] The session acquisition module 610 is used to acquire customer service session data within a first historical period;

[0142] The label acquisition module 620 is used to input the customer service session data and the generated question label prompt into the customer complaint big model for any customer service session data, and obtain the feedback question label of the customer service session data output by the customer complaint big model;

[0143] The aggregation module 630 is used to aggregate the feedback question labels of each customer service session data in the first historical period, and extract the high-frequency feedback questions in the first historical period according to the aggregation result.

[0144] In an optional implementation, the aggregation module 630 is used to: generate a first tag set according to feedback question tags of each customer service session data in the first historical period;

[0145] Inputting the first tag set, aggregation conditions, and high-frequency feedback problem extraction conditions into the customer complaint big model;

[0146] Obtain high-frequency feedback issues in the first historical period output by the customer complaint model.

[0147] In an optional implementation, the aggregation module 630 is used to: generate a second tag set according to the historical feedback question tags of a second historical period; wherein the second historical period is earlier than the first historical period;

[0148] Input any feedback question label within the first historical period, the second label set, and similar label extraction conditions into the customer complaint big model, and obtain a revised question label of the feedback question label output by the customer complaint big model;

[0149] An ordered set of labels is generated according to each correction problem label in the first historical period, and high-frequency feedback problems in the first historical period are extracted according to the ordered set of labels.

[0150] In an optional implementation, the similar tag extraction condition includes: if there is a historical feedback question tag in the second tag set whose similarity to the feedback question tag is greater than a similarity threshold, outputting the historical feedback question tag with the greatest similarity to the feedback question tag;

[0151] If there is no historical feedback question tag in the second tag set whose similarity with the feedback question tag is greater than the similarity threshold, the feedback question tag is output.

[0152] In an optional implementation, the aggregation module 630 is used to: generate a first question set according to the high-frequency feedback questions in the first historical period output by the customer complaint model, and generate a second question set according to the high-frequency feedback questions in the first historical period extracted by the ordered set of labels;

[0153] Calculating the set similarity between the first question set and the second question set;

[0154] A final high-frequency feedback question set for the first historical period is generated according to the set similarity, the first question set, and the second question set.

[0155] In an optional implementation, the aggregation module 630 is used to: if the set similarity is greater than a first threshold, use the first question set as the final high-frequency feedback question set;

[0156] If the set similarity is less than or equal to the first threshold, the second question set is used as the final high-frequency feedback question set.

[0157] In an optional implementation, the device further includes: a preprocessing module (not shown in the figure), configured to preprocess the customer service session data within the first historical period;

[0158] The label acquisition module 620 is used to: input the pre-processed customer service conversation data and the generated question label prompt words into the customer complaint model;

[0159] The preprocessing includes at least one of the following processing modes: non-text data elimination processing, non-text data conversion processing, and invalid text elimination processing.

[0160] It can be seen that the device for obtaining high-frequency feedback problems provided in the embodiment of the present application can obtain feedback problem labels within the first historical period by processing customer service conversation data with the customer complaint big model, and further obtain high-frequency feedback problems by aggregating the feedback problem labels within the first historical period. This solution can realize the automatic generation of high-frequency feedback problems, and can improve the acquisition accuracy and efficiency of high-frequency feedback problems.

[0161] Embodiment 6

[0162] Figure 7 A structural schematic diagram of a computing device embodiment provided in Example 6 of the present application is shown. The specific embodiments of the present application do not limit the specific implementation of the computing device.

[0163] like Figure 7 As shown, the computing device may include: a processor (processor) 702 , a communications interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

[0164] The processor 702, the communication interface 704, and the memory 706 communicate with each other via the communication bus 708. The communication interface 704 is used to communicate with other devices such as a client or other server network elements. The processor 702 is used to execute the program 710, which can specifically execute the relevant steps in the above-mentioned method for obtaining high-frequency feedback problems of computing devices.

[0165] Specifically, the program 710 may include program codes, which include computer operation instructions.

[0166] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0167] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The program 710 may be specifically used to enable the processor 702 to perform the operations in the above method embodiment.

[0168] Embodiment 7

[0169] Embodiment 7 of the present application provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction or computer program, which enables a processor to perform operations corresponding to the method for obtaining a high-frequency feedback problem in any of the above method embodiments.

[0170] Embodiment 8

[0171] Embodiment 8 of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the method for obtaining the high-frequency feedback problem in any of the above method embodiments.

[0172] In summary, according to the computing device, computer storage medium and computer program product provided by this embodiment, feedback question labels within the first historical period can be obtained by processing customer service conversation data with the customer complaint big model, and high-frequency feedback questions can be obtained by further aggregating the feedback question labels within the first historical period. This solution can realize the automatic generation of high-frequency feedback questions and improve the acquisition accuracy and efficiency of high-frequency feedback questions.

[0173] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0174] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0175] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0176] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0177] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0178] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0179] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A method for obtaining a high-frequency feedback problem, characterized in that: include: Obtain customer service session data within the first historical period; For any customer service conversation data, the customer service conversation data and the generated question label prompt are input into the customer complaint big model, and the feedback question label of the customer service conversation data output by the customer complaint big model is obtained; Feedback question labels of each customer service session data in the first historical period are aggregated, and high-frequency feedback questions in the first historical period are extracted according to the aggregation result.

2. The method according to claim 1, characterized in that Aggregating the feedback question labels of each customer service session data in the first historical period, and extracting the high-frequency feedback questions in the first historical period according to the aggregation result includes: Generate a first tag set according to the feedback question tags of each customer service session data in the first historical period; Inputting the first tag set, aggregation conditions, and high-frequency feedback problem extraction conditions into the customer complaint big model; Obtain high-frequency feedback issues in the first historical period output by the customer complaint model.

3. The method according to claim 2, characterized in that Aggregating the feedback question labels of each customer service session data in the first historical period, and extracting the high-frequency feedback questions in the first historical period according to the aggregation result includes: Generate a second tag set based on the historical feedback question tags of the second historical period; wherein the second historical period is earlier than the first historical period; Input any feedback question label within the first historical period, the second label set, and similar label extraction conditions into the customer complaint big model, and obtain a revised question label of the feedback question label output by the customer complaint big model; An ordered set of labels is generated according to each correction problem label in the first historical period, and high-frequency feedback problems in the first historical period are extracted according to the ordered set of labels.

4. The method according to claim 3, characterized in that The similar tag extraction conditions include: If there is a historical feedback question tag in the second tag set whose similarity to the feedback question tag is greater than the similarity threshold, outputting the historical feedback question tag with the greatest similarity to the feedback question tag; If there is no historical feedback question tag in the second tag set whose similarity with the feedback question tag is greater than the similarity threshold, the feedback question tag is output.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: Generate a first set of questions based on the high-frequency feedback questions in the first historical period output by the customer complaint model, and generate a second set of questions based on the high-frequency feedback questions in the first historical period extracted by the ordered set of labels; Calculating the set similarity between the first question set and the second question set; A final high-frequency feedback question set for the first historical period is generated according to the set similarity, the first question set, and the second question set.

6. The method according to claim 5, characterized in that Generating the final high-frequency feedback question set of the first historical period according to the set similarity, the first question set and the second question set includes: If the set similarity is greater than a first threshold, taking the first question set as the final high-frequency feedback question set; If the set similarity is less than or equal to the first threshold, the second question set is used as the final high-frequency feedback question set.

7. The method according to any one of claims 1 to 6, characterized in that After acquiring the customer service conversation data within the first historical period, the method further includes: preprocessing the customer service conversation data within the first historical period; The step of inputting the customer service conversation data and the generated question label prompt words into the customer complaint big model comprises: inputting the pre-processed customer service conversation data and the generated question label prompt words into the customer complaint big model; The preprocessing includes at least one of the following processing modes: non-text data elimination processing, non-text data conversion processing, and invalid text elimination processing.

8. A device for obtaining high-frequency feedback problems, characterized in that: include: A session acquisition module, used to acquire customer service session data within a first historical period; A label acquisition module is used to input the customer service session data and the generated question label prompt into the customer complaint model for any customer service session data, and obtain the feedback question label of the customer service session data output by the customer complaint model; The aggregation module is used to aggregate the feedback question labels of each customer service session data in the first historical period, and extract the high-frequency feedback questions in the first historical period according to the aggregation result.

9. A computing device, characterized in that include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method for obtaining high-frequency feedback problems according to any one of claims 1-7.

10. A computer storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the method for obtaining a high-frequency feedback problem according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the method for obtaining a high-frequency feedback problem according to any one of claims 1 to 7.

Images