Customer opinion mining method, device and system

Abstract

The invention discloses a customer opinion mining method and device, equipment and a storage medium, and relates to the field of big data, the method comprises the steps of obtaining a customer opinion document and a hotspot label, the customer opinion document being a customer service work order document; performing data cleaning on the customer opinion document; inputting the customer opinion document after data cleaning and the hotspot label into a self-supervision-based abstract extraction model to obtain an abstract extraction result of the customer opinion document; and outputting an abstract extraction result. According to the scheme provided by the invention, the client opinion mining efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a customer opinion mining method, device, equipment and storage medium. Background Art

[0002] Customer opinions play an important role in improving customer service, grasping the pulse of the market, capturing business opportunities, and discovering signs of risk. In the traditional model, customer experience problem analysis is obtained by service experience managers sampling from customer work orders and manually summarizing and refining them. However, this method has problems such as incomplete coverage, insufficient focus on feedback issues, and time-consuming statistical work. How to accurately mine customer opinions and obtain customer opinion summaries is an urgent problem to be solved in improving customer service quality and user experience. Summary of the invention

[0003] The present invention provides a customer opinion mining method, device, equipment and storage medium, which improves the efficiency of mining customer opinions and improves user experience.

[0004] According to one aspect of the present invention, a method for mining customer opinions is provided, comprising:

[0005] Obtain customer opinion documents and hot tags. Customer opinion documents are customer service work order documents.

[0006] Perform data cleaning on customer opinion documents;

[0007] Input the customer opinion document and hot tags into the summary extraction model based on self-supervision to obtain the summary extraction result of the customer opinion document;

[0008] Output summary extraction results.

[0009] In one embodiment, the method further comprises:

[0010] Training a self-supervised summary extraction model based on training data, where the training data includes training data documents and training labels;

[0011] When the evaluation parameter of the summary extraction model based on self-supervision meets the preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

[0012] In one embodiment, when the evaluation parameter of the summary extraction model based on self-supervision meets the preset parameter threshold, determining that the training of the summary extraction model based on self-supervision is completed includes:

[0013] When at least one of the accuracy, precision and recall rate of the summary extraction model based on self-supervision meets the preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

[0014] In one embodiment, obtaining the customer opinion document and hotspot tags includes:

[0015] The customer opinion document is obtained from the customer opinion document storage area, and the hotspot tag selected by the user in the hotspot tag library is obtained.

[0016] In one embodiment, the customer service ticket document includes at least one of a discrete word, an abbreviation, and an abbreviation;

[0017] Data cleaning of customer opinion documents, including:

[0018] Filter stop words and special characters in customer opinion documents, and remove repeated words in customer opinion documents;

[0019] Complete the missing words, abbreviations, and acronyms in customer opinion documents, and combine discrete words into complete sentences.

[0020] Before obtaining the customer opinion document from the customer opinion document storage area, it also includes:

[0021] Store the customer opinion documents manually entered by the customer service staff in the customer opinion document storage area;

[0022] Alternatively, the recorded customer service voice file is subjected to voice recognition to generate a customer opinion document and then stored in the customer opinion document storage area;

[0023] Alternatively, the text information of the interaction between the customer service and the user is integrated into a customer opinion document and stored in the customer opinion document storage area.

[0024] According to another aspect of the present invention, there is provided a customer opinion mining device, comprising:

[0025] The acquisition module is used to obtain customer opinion documents and hot tags. The customer opinion documents are customer service work order documents.

[0026] Data cleaning module, used to clean customer opinion documents;

[0027] An extraction module, used to input the customer opinion document and hot tags into a summary extraction model based on self-supervision to obtain a summary extraction result of the customer opinion document;

[0028] Output module, used to output summary extraction results.

[0029] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0030] Memory stores computer-executable instructions;

[0031] The processor executes the computer-executable instructions stored in the memory to implement the above-mentioned customer opinion mining method.

[0032] According to another aspect of the present invention, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above-mentioned customer opinion mining method.

[0033] According to another aspect of the present invention, a computer program product is provided, including a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and the at least one processor implements the above-mentioned customer opinion mining method when executing the computer program.

[0034] The technical solution of the embodiment of the present invention, after obtaining the customer service work order document and hot tags which are the customer opinion document, first performs data cleaning on the customer opinion document, and then inputs the customer opinion document and hot tags after data cleaning into a summary extraction model based on self-supervision, obtains the summary extraction result of the customer opinion document and outputs the summary extraction result, thereby providing an automated customer opinion mining method, which can realize comprehensive customer opinion mining, improves the mining efficiency of customer opinions, and is conducive to improving user experience.

[0035] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic diagram of a process for mining customer opinions provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the structure of a customer opinion mining device provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention. The acquisition, storage, use, and processing of data in the technical scheme of this application comply with the relevant provisions of national laws and regulations.

[0041] Figure 1 A flow chart of a method for mining customer opinions provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the customer opinion mining method provided in this embodiment includes:

[0042] Step S110, obtaining a customer opinion document and hot tags, where the customer opinion document is a customer service work order document.

[0043] The customer opinion mining method provided in the embodiment of the present application is applied to any computer or server with processing power, which is used to mine customer opinions, and the computer or server can be deployed locally or in the cloud. Customer opinions refer to questions or opinions raised by customers about certain products or services, which are generally recorded by customer service when users communicate with the customer service of the product or service provider via telephone or the Internet. The purpose of recording customer opinions is to enable product developers, service operators, etc. to improve products or services based on customer opinions, so as to improve the quality of service to users.

[0044] However, the user's questions or opinions are raised in natural language, so the customer opinions recorded by the customer service are the natural language of the customer, and it is difficult for the back-end staff to quickly understand the questions or opinions raised by the user based on the natural language. The developers or operators in the back-end need to face a large number of customer opinions, so it is necessary to extract the key information of the customer opinions recorded by the customer service and then provide it to relevant personnel for subsequent research and processing. However, at present, it mainly relies on manual sampling of customer opinions, and manually refines the key information in the customer opinions to form a summary of the customer opinions. However, manual refining is inefficient, and can only sample and refine some customer opinions, which is difficult to fully reflect the content of all customer opinions. At the same time, it also requires a lot of manpower and is costly. Therefore, the embodiment of the present application provides a customer opinion mining method, and provides an automatic customer opinion mining method, which can extract summaries of all customer opinions with high efficiency.

[0045] In this embodiment, customer opinion documents and hot tags are first obtained, where the customer opinion documents are customer service work order documents. A customer service work order document is a document composed of customer opinions and problems recorded by customer service staff when serving users. According to the customer service work requirements of different service providers, customer service work order documents have different formats. Some text content is recorded according to the requirements of different formats. However, no matter which format is used for recording, the content recorded in the customer service work order document reflects the opinions, problems, demands, etc. of users. The hot tags are keyword tags provided by product or service providers according to service requirements, and the hot tags reflect the service providers' understanding requirements for customer opinions. Since customer opinions include the opinions and problems of all customers, some of the opinions and problems may be irrelevant to the product or service itself. Even for the opinions and problems related to the product or service, there are priority distinctions for improving the quality of the product or service. Therefore, service providers can, according to their own service improvement needs, set hot tags to extract only the information related to the hot tags from the customer opinion documents. The hot tags can be changed according to the needs of different periods, and different summary information can be extracted from the customer opinion documents according to different hot tags.

[0046] The customer service work order document includes the problems or opinions raised by users recorded by the customer service when serving users. Specifically, the customer service work order document includes at least one of discrete words, abbreviations, and short names.

[0047] When the customer service work order document is in natural language and even includes at least one of discrete words, abbreviations, and short names, data cleaning is performed on the customer opinion document, including: filtering out stop words and special characters in the customer opinion document, and deleting duplicate words in the customer opinion document; supplementing missing words, abbreviations, and short names in the customer opinion document, and combining discrete words into complete sentences. That is, data cleaning of the customer opinion document includes two aspects. One is to remove redundant content in the customer opinion document, including filtering out stop words and special characters in the customer opinion document. For example, words that appear frequently but have no significant meaning are filtered out, including function words such as modal particles, adverbs, and prepositions, as well as high-frequency words such as "de (的), le (了), jiu (就)", etc., and removing numbers, English, special characters, etc. in the text. And deleting duplicate words in the customer opinion document. The other is to supplement incomplete content in the customer opinion document. For example, changing English abbreviations in the record to Chinese or English full names, supplementing short names to full names, and supplementing incomplete missing content according to the overall meaning of the full text of the customer opinion document, etc.

[0048] Step S120, perform data cleaning on the customer opinion document.

[0049] Since the customer opinion document is a customer service work order document, it is recorded by the customer service based on the communication with the user. At the same time, the user expresses opinions or questions in natural language. Even if the customer service records the user's expression completely, the recorded content also includes a lot of other information besides useful information. In actual situations, most of the time it is difficult for the customer service to fully record all the user's expressions, and the user's expressions can only be recorded through keywords, phrases, abbreviations, etc. In either case, the customer opinion document includes a lot of useless information, and it is impossible to directly extract the summary. Therefore, after obtaining the customer opinion document, it is necessary to clean the data. The purpose of data cleaning is to process the customer opinion document and exclude invalid information. Furthermore, data cleaning can also complete the incomplete valid information in the customer opinion document.

[0050] Step S130 , inputting the customer opinion document and hot tags after data cleaning into a summary extraction model based on self-supervision to obtain a summary extraction result of the customer opinion document.

[0051] The customer opinion mining method provided in the embodiment of the present application uses a summary extraction model based on self-supervision to extract the summary of the customer opinion document. The summary extraction model based on self-supervision is a pre-trained model, which can be any natural language processing model based on a neural network. The summary extraction model based on self-supervision is obtained by training according to the pre-input training data and hot tags. After the customer opinion document and hot tags are input into the trained summary extraction model based on self-supervision, the summary extraction result of the customer opinion document can be obtained. According to different designs of summary extraction models based on self-supervision, the form of the summary extraction result obtained is different. For example, it can be composed of multiple keywords corresponding to hot tags, and one hot tag corresponds to one or more keywords; or it can be composed of one or more complete sentences, and one hot tag corresponds to one or more complete sentences, or multiple hot tags correspond to one complete sentence; or it can be composed of a paragraph of article. Based on the different needs of service providers, different summary extraction models based on self-supervision can be designed, so that summary extraction results that meet the needs can be extracted.

[0052] The training method of the summary extraction model based on self-supervision includes: training the summary extraction model based on self-supervision according to training data, the training data includes training data documents and training labels; when the evaluation parameters of the summary extraction model based on self-supervision meet the preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

[0053] The summary extraction model based on self-supervision is trained using training data documents and training labels. The training data documents and training labels have a matching relationship with the summary extraction result. The more training data documents and training labels there are, the better the training effect of the summary extraction model based on self-supervision. However, since the summary extraction model based on self-supervision is adopted in the embodiment of the present application, and the summary extraction model is self-supervised using a self-supervised objective function, it is possible to fine-tune the model relying on a small amount of training label data, and the generalization is stronger than that of the model that does not use the self-supervised function. Therefore, the summary extraction model based on self-supervision used in the embodiment of the present application improves the precision and accuracy of summary extraction, thereby improving the efficiency of customer opinion mining.

[0054] Among them, the evaluation parameters for evaluating the summary extraction model based on self-supervision include at least one of accuracy, precision and recall. In the process of training the summary extraction model based on self-supervision, when it is determined that at least one of the accuracy, precision and recall rate of the summary extraction model based on self-supervision meets the preset parameter threshold, the training can be determined to be completed. Among them, accuracy = the number of all correctly predicted samples ÷ the total number of samples; precision and recall are for each category, accuracy = the number of correct predictions for a certain category ÷ the total number of model predictions for this category; recall = the number of correct predictions for a certain category ÷ the total number of this category. According to the three parameters of accuracy, precision and recall, if at least one of the three parameters meets expectations, the current model is saved as the final model; if it does not meet expectations, the training data, model parameters, etc. are adjusted and training continues. Which or which of the above three parameters meet the preset parameter threshold to determine that the model training is completed can be set according to customer needs.

[0055] Step S140: outputting the summary extraction result.

[0056] After obtaining the summary extraction results, the summary extraction results can be output according to the preset rules. For example, the summary extraction results can be output to a specific document storage location, and each customer opinion document corresponds to a separate summary extraction result file. Or the summary extraction results can be output to a specific location in a summary extraction summary document. Or the summary extraction results can be output to different locations based on different hot tag matching situations. In short, different summary extraction result output methods are all for the convenience of back-end staff to read and improve products or services based on the read summary extraction results.

[0057] The customer opinion mining method provided in this embodiment, after obtaining the customer service work order document and hot tags which are the customer opinion document, first performs data cleaning on the customer opinion document, and then inputs the customer opinion document and hot tags after data cleaning into a summary extraction model based on self-supervision, obtains the summary extraction result of the customer opinion document and outputs the summary extraction result, thereby providing an automated customer opinion mining method, which can realize comprehensive customer opinion mining, improve the mining efficiency of customer opinions, and is conducive to improving user experience.

[0058] In one embodiment, the customer opinion document is obtained from the customer opinion document storage area, and the hot tags are obtained by the user from the hot tag library. After each customer service serves the customer, the recorded customer opinion documents will be uniformly stored in the customer opinion document storage area, which can be an independent database or a specific storage area of ​​a general database. The hot tag library includes multiple hot tags, and the user can select the required hot tags in the hot tag library for this customer opinion mining. The user can select one or more hot tags from the hot tag library. The hot tags in the hot tag library are all hot tags used when training the summary extraction model based on self-supervision. The user can add new hot tags to the hot tag library according to the needs, but after adding new hot tags, the summary extraction model based on self-supervision needs to be retrained based on the new hot tag library.

[0059] The customer opinion document storage area stores documents recorded by customer service after serving customers. Traditional customer service needs to serve customers over the phone, and the customer service manually records the customer's documents and opinions, so the customer opinion document is manually entered by the customer service. In an embodiment of the present application, in addition to the manual recording by the customer service, the voice of the customer service when serving the customer can also be recorded to generate a customer service voice file, and then the customer service voice file is recognized through voice recognition technology, and a customer opinion document is generated based on the text obtained by the recognition. Or in some scenarios, the customer can also communicate with the customer service through the Internet, then the communication between the customer service and the customer is carried out through text messages, so the text information of the customer service and the user interaction can be integrated into a customer opinion document. Regardless of the method used to obtain the customer opinion document, it is stored in the customer opinion document storage area.

[0060] Figure 2 A schematic diagram of the structure of a customer opinion mining device provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the customer opinion mining device for application software provided in this embodiment includes:

[0061] The acquisition module 21 is used to obtain customer opinion documents and hot tags, where the customer opinion documents are customer service work order documents; the data cleaning module 22 is used to perform data cleaning on the customer opinion documents; the extraction module 23 is used to input the customer opinion documents and hot tags into a summary extraction model based on self-supervision to obtain a summary extraction result of the customer opinion document; and the output module 24 is used to output the summary extraction result.

[0062] The customer opinion mining device for application software provided in this embodiment is used to implement Figure 1 The customer opinion mining method provided in the illustrated embodiment has similar implementation principles and technical effects, which will not be described in detail here.

[0063] In one embodiment, the customer opinion mining device also includes a training module, which is used to train the summary extraction model based on self-supervision according to training data, and the training data includes training data documents and training labels; when the evaluation parameters of the summary extraction model based on self-supervision meet the preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

[0064] In one embodiment, the evaluation parameter includes at least one of accuracy, precision, and recall.

[0065] In one embodiment, the customer service work order document includes questions or comments raised by users recorded when the customer service staff is providing services to the users.

[0066] In one embodiment, the customer service work order document includes at least one of a discrete word, an abbreviation, and an abbreviation.

[0067] In one embodiment, the data cleaning module 22 is specifically used to filter stop words and special characters in the customer opinion document, and delete repeated words in the customer opinion document; complete the missing words, abbreviations, and abbreviations in the customer opinion document, and combine discrete words into complete sentences.

[0068] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device may include: a transceiver 31 , a processor 32 , and a memory 33 .

[0069] The processor 32 executes the computer execution instructions stored in the memory, so that the processor 32 executes the scheme in the above embodiment. The processor 32 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0070] The memory 33 is connected to the processor 32 via a system bus and completes communication between them. The memory 33 is used to store computer program instructions.

[0071] The transceiver 31 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0072] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0073] The electronic device provided in the embodiment of the present application may be a computer or server deployed by the customer opinion mining device in the above-mentioned embodiment.

[0074] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the customer opinion mining method of the above embodiment.

[0075] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the customer opinion mining method in the above embodiment.

[0076] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0077] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A customer opinion mining method, characterized in that: include: Obtaining a customer opinion document and hot tags, wherein the customer opinion document is a customer service work order document; Performing data cleaning on the customer opinion document; Inputting the customer opinion document after data cleaning and the hot tags into a summary extraction model based on self-supervision to obtain a summary extraction result of the customer opinion document; The summary extraction result is outputted.

2. The method according to claim 1, characterized in that Also includes: Training the self-supervised summary extraction model according to training data, wherein the training data includes training data documents and training labels; When the evaluation parameter of the summary extraction model based on self-supervision meets the preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

3. The method according to claim 2, characterized in that When the evaluation parameter of the summary extraction model based on self-supervision meets the preset parameter threshold, determining that the training of the summary extraction model based on self-supervision is completed includes: When at least one of the accuracy, precision and recall rate of the summary extraction model based on self-supervision meets a preset parameter threshold, it is determined that the training of the summary extraction model based on self-supervision is completed.

4. The method according to any one of claims 1 to 3, characterized in that: The obtaining of customer opinion documents and hot tags includes: The customer opinion document is obtained from the customer opinion document storage area, and the hot tags selected by the user in the hot tag library are obtained, where the hot tags are keywords related to the service requirements.

5. The method according to claim 4, characterized in that The customer service work order document includes at least one of a discrete word, an abbreviation, and an abbreviation; The data cleaning of the customer opinion document includes: Filtering stop words and special characters in the customer opinion document, and deleting repeated words in the customer opinion document; Complete the missing words, abbreviations, and acronyms in the customer opinion document, and combine discrete words into complete sentences.

6. The method according to claim 4, characterized in that Before obtaining the customer opinion document from the customer opinion document storage area, the method further includes: Storing the customer opinion document manually input by the customer service staff in the customer opinion document storage area; Or performing voice recognition on the recorded customer service voice file to generate a customer opinion document and then storing it in the customer opinion document storage area; Alternatively, the text information of the interaction between the customer service and the user is integrated into a customer opinion document and then stored in the customer opinion document storage area.

7. A customer opinion mining device, characterized in that: include: An acquisition module is used to acquire a customer opinion document and hot tags, wherein the customer opinion document is a customer service work order document; A data cleaning module, used for cleaning the customer opinion document; An extraction module, used for inputting the customer opinion document and the hot tags into a summary extraction model based on self-supervision to obtain a summary extraction result of the customer opinion document; An output module is used to output the summary extraction result.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the customer opinion mining method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the customer opinion mining method according to any one of claims 1 to 6.

10. A computer program product, characterized in that It includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the customer opinion mining method as described in any one of claims 1 to 6.

Images