Customer objection mining method and device, electronic equipment and storage medium

Abstract

The embodiment of the application provides a customer objection mining method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining a call text, wherein the call text comprises known objection texts and unknown objection texts; inputting the known objection texts into a pre-training model; performing model fine-tuning on the pre-training model based on the call text, calculating a vectorization representation of each call text, and obtaining a vectorization text; clustering the vectorization text, classifying vectorization texts with similar semantics into the same category, and obtaining a clustering result; and screening the clustering result to obtain newly added customer objections. Based on this, the embodiment of the application can automatically mine newly added customer objections, and solve the problems of high labor cost, small coverage and low real-time performance in current manual mining.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a customer objection mining method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Insurance telephone sales will face different types of customer objections (such as: I don't need insurance, I have already bought insurance) in the process of customer management, product sales, and other businesses. The existing electric sales assistants will identify customer objections and recommend corresponding response techniques to the agents to assist the agents in responding. However, with the changes in internal and external environments such as the listing of new products, economic and social development, and improvement of residents' lives, customer objections will also change, and new objections will continue to arise. Currently, for newly added customer objections, a manual mining method is mainly used, and business personnel manually screen out uncovered objections. This solution mainly has three shortcomings:

[0003] (1) High labor cost: the daily call volume of the agents is very large (tens of millions of call text numbers), and a large amount of manpower is required. Moreover, there are difficulties in personnel training and management, such as new employee training, personnel turnover, and the like;

[0004] (2) Small coverage: due to the large call volume of the agents and the limited manpower, only the transaction items (accounting for less than 5%) are covered at present, and a large number of calls of the agents are not covered;

[0005] (3) Low real-time performance: the mining period of new objections is relatively long, generally more than one month, and cannot adapt to the product iteration speed. SUMMARY

[0006] The main purpose of the embodiments of the present application is to propose a customer objection mining method and device, an electronic device, and a storage medium, which can automatically mine newly added customer objections and solve the problems of high labor cost, small coverage, and low real-time performance in the current manual mining.

[0007] To achieve the above purpose, a first aspect of the embodiments of the present application proposes a customer objection mining method, which comprises:

[0008] obtaining call texts, the call texts comprising known objection texts and unknown objection texts;

[0009] inputting the known objection texts into a pre-trained model;

[0010] performing model fine-tuning on the pre-trained model based on the call texts, calculating a vectorized representation of each call text, and obtaining a vectorized text;

[0011] performing clustering on the vectorized text, classifying the vectorized texts with similar semantics into the same category, and obtaining a clustering result;

[0012] Screening the clustering results to obtain new customer objections.

[0013] In some embodiments, the pre-training model comprises a BERT-based text encoder, a linear layer and a softmax layer, and the inputting the known objection text into the pre-training model comprises:

[0014] Inputting the known objection text into the BERT-based text encoder to calculate the vectorized representation of each character in the known objection text;

[0015] Selecting the vectorized representation corresponding to the first character in each character as the sentence vector representation of the known objection text;

[0016] Inputting the sentence vector representation into the linear layer and the softmax layer to obtain the probability that the known objection text belongs to the i-th objection, where i is a positive integer greater than 1.

[0017] In some embodiments, the model fine-tuning of the pre-training model based on the call text to calculate the vectorized representation of each call text to obtain the vectorized text comprises:

[0018] Data augmentation is performed on the call text data to obtain known objection augmented data and unknown objection augmented data;

[0019] Determine the semi-supervised contrastive loss, supervised loss and regularization term according to the known objection augmented data and the unknown objection augmented data;

[0020] Determine the training target of model fine-tuning according to the semi-supervised contrastive loss, the supervised loss and the regularization term;

[0021] Fine-tune the pre-training model based on the training target of model fine-tuning;

[0022] Calculate the vectorized representation of each unknown objection text to obtain the vectorized text.

[0023] In some embodiments, the clustering of the vectorized text comprises:

[0024] Obtain the sentence vector of the unknown objection text;

[0025] Determine the number of categories of the clustering algorithm;

[0026] For each data, calculate the distance between each data and the initial center of multiple categories, and classify the data into the nearest category;

[0027] For each category, the average of the sentence vectors of all data in the category is calculated as a new category center until the category division of all data no longer changes;

[0028] The similar texts are divided into the same category by using the clustering algorithm to obtain a clustering result.

[0029] In some embodiments, the determination of the number of categories of the clustering algorithm comprises:

[0030] A plurality of initial category numbers k' are randomly set to obtain k' initial categories;

[0031] The confidence of each of the initial categories is calculated;

[0032] From the k' initial categories, k categories with a confidence higher than a preset confidence threshold are selected as the number of categories of the clustering algorithm, wherein k is less than or equal to k'.

[0033] In some embodiments, the division of the similar texts into the same category by using the clustering algorithm to obtain a clustering result comprises:

[0034] The similar texts are divided into the same category by using a KMeans clustering algorithm to obtain a classification category;

[0035] The quality of all the classification categories is evaluated, and low-quality categories are filtered out to obtain a clustering result.

[0036] In some embodiments, the quality evaluation of all the classification categories and the filtering out of low-quality categories to obtain a clustering result comprises:

[0037] The silhouette coefficient of data in each category is calculated;

[0038] The data in each category is arranged in descending order of the silhouette coefficient to generate a category description;

[0039] In the case where the silhouette coefficient of the category is lower than a preset silhouette coefficient threshold, the category is determined as a low-quality category;

[0040] The low-quality category is filtered out to obtain a clustering result.

[0041] To achieve the above object, a second aspect of the embodiment of the present application provides a customer objection mining device, which comprises:

[0042] An acquisition module is configured to acquire call texts, wherein the call texts comprise known objection texts and unknown objection texts;

[0043] A pre-training module is configured to input the known objection texts into a pre-training model;

[0044] a fine-tuning module configured to fine-tune the pre-trained model based on the call texts, calculate a vectorized representation of each of the call texts, and obtain vectorized texts;

[0045] a clustering module configured to cluster the vectorized texts, classify the vectorized texts with similar semantics into the same category, and obtain clustering results;

[0046] a screening module configured to screen the clustering results, and obtain new customer objections.

[0047] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0048] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0049] The customer objection mining method and device, electronic device and storage medium provided by the present application obtain call texts, the call texts include known objection texts and unknown objection texts; input the known objection texts into a pre-trained model; fine-tune the pre-trained model based on the call texts, calculate a vectorized representation of each of the call texts, and obtain vectorized texts; cluster the vectorized texts, classify the vectorized texts with similar semantics into the same category, and obtain clustering results; screen the clustering results, and obtain new customer objections. Based on this, the model pre-training of the embodiments of the present application is based on the texts of known objections; the model fine-tuning is based on the texts of known and unknown objections to update the model, and the vectorized representation of each text is calculated; then the vectorized texts are clustered, so that the texts with similar semantics become the same category, i.e. new customer objections; finally, the clustering results are screened to select new customer objections, so as to automatically mine new customer objections, and solve the problems of high labor cost, small coverage and low real-time performance in current manual mining. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the customer objection mining method provided by the embodiments of the present application;

[0051] Figure 2A is a flowchart of step S102 in Figure 1

[0052] Figure 2B is a pre-trained model structure diagram provided by the embodiments of the present application;

[0053] ​Figure 2C is a BERT structure diagram provided by an embodiment of the present application;

[0054] Figure 3 is Figure 1 is a flowchart of step S103 in

[0055] Figure 4 is Figure 3 is a flowchart of step S401 in

[0056] Figure 5 is Figure 4 is a flowchart of step S402 in

[0057] Figure 6 is Figure 4 is a flowchart of step S405 in

[0058] Figure 7 is Figure 6 is a flowchart of step S602 in

[0059] Figure 8 is a customer objection mining principle block diagram provided by an embodiment of the present application;

[0060] Figure 9 is a structural schematic diagram of a customer objection mining device provided by an embodiment of the present application;

[0061] Figure 10 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0063] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0065] First, the terms involved in the present application are analyzed:

[0066] Artificial intelligence (AI): is a new technical science of studying, developing the theory, method, technology and application system for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence attempts to understand the essence of intelligence, and produce a new intelligent machine that can react in a similar way to human intelligence, the research in this field includes robots, language recognition, image recognition, natural language processing and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0067] Insurance: refers to the commercial insurance behavior that the policyholder pays the insurance premium to the insurer according to the contract, and the insurer undertakes the compensation for the insurance responsibility for the property loss caused by the accident as agreed in the contract, or undertakes the payment of the insurance responsibility for the death, disability, disease or reaching the age, period and other conditions agreed in the contract. From the economic point of view, insurance is a kind of financial arrangement for sharing the loss of unexpected accidents; from the legal point of view, insurance is a kind of contract behavior, which is a kind of contract arrangement for one party to compensate for the loss of another party; from the social point of view, insurance is an important part of social and economic security system, and is a "delicate stabilizer" of social production and social life; from the risk management point of view, insurance is a method of risk management.

[0068] Telephone sales: it is to take telephone as the main communication means, telephone sales usually for the mode of active sales by telephone. With the aid of network, fax, short message, mail delivery and other auxiliary means, through the special telephone marketing number, directly contact with the customers in the name of the company, and use the company's automatic information management technology and professional operation platform, complete the main marketing process of the company's product promotion, consultation, quotation and product transaction condition confirmation.

[0069] Customer objection: refers to the disapproval, questioning or refusal of the customer to the salesperson in the sales process.

[0070] Semi-supervised learning (SSL): is a key problem in the field of pattern recognition and machine learning, and is a learning method combining supervised learning and unsupervised learning. Semi-supervised learning uses a large amount of unlabeled data, as well as labeled data, to carry out pattern recognition work. When using semi-supervised learning, as few personnel as possible will be required to work, while high accuracy can be brought, therefore, semi-supervised learning is getting more and more attention.

[0071] Gradient descent: is a kind of iterative method, which can be used to solve the least squares problem (linear and nonlinear). In solving the model parameters of machine learning algorithm, that is, the unconstrained optimization problem, gradient descent is one of the most commonly used methods, and the other commonly used method is least squares method. In solving the minimum value of the loss function, the gradient descent method can be used to solve it step by step, and the minimum loss function and model parameter value are obtained. On the contrary, if we need to solve the maximum value of the loss function, we need to use gradient ascent method to iterate. In machine learning, two kinds of gradient descent methods are developed based on the basic gradient descent method, which are stochastic gradient descent method and batch gradient descent method.

[0072] Confidence: In statistics, the confidence interval of a probability sample is the interval estimate of the population parameter of the sample. The confidence interval shows the degree to which the true value of the parameter falls around the measured result. The confidence interval gives the range of the measured parameter value, that is, the "certain probability" required in the preceding. This probability is called the confidence level.

[0073] Silhouette Coefficient: is a way to evaluate the good and bad of clustering. It was first proposed by Peter J. Rousseeuw in 1986. It combines cohesion and separation. It can be used to evaluate the influence of different algorithms or different running ways of the algorithm on the clustering results on the basis of the same original data.

[0074] In view of the problems of high labor cost, small coverage and low real-time performance in current manual mining, based on this, the embodiment of the present application provides a customer objection mining method and device, electronic equipment and storage medium, obtain the call text, the call text includes known objection text and unknown objection text; input the known objection text into the pre-training model; based on the call text, the pre-training model is fine-tuned, the vectorization representation of each call text is calculated, and the vectorization text is obtained; the vectorization text is clustered, the vectorization texts with similar semantics are classified into the same category, and the clustering result is obtained; the clustering result is screened, and the new customer objection is obtained. Based on this, the model pre-training of the embodiment of the present application is trained based on the text of known objection; the model fine-tuning is based on the text of known and unknown objection to update the model, and the vectorization representation of each text is calculated; then the vectorization text is clustered, so that the texts with similar semantics become the same category, that is, the new customer objection; finally, the clustering result is screened, and the new customer objection is selected, so as to automatically mine the new customer objection, solve the problems of high labor cost, small coverage and low real-time performance in current manual mining.

[0075] The customer objection mining method and device, electronic equipment and storage medium provided by the embodiment of the present application are specifically explained by the following embodiment, first, the customer objection mining method in the embodiment of the present application is described.

[0076] The embodiment of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results.

[0077] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0078] The customer objection mining method provided by the embodiments of the present application relates to the technical field of artificial intelligence. The customer objection mining method provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as a separate physical server, can also be configured as a server cluster or a distributed system formed by multiple physical servers, and can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms; and the software can be an application that implements the customer objection mining method, but is not limited to the above forms.

[0079] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0080] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the user, such as user information, user behavior data, user history data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to normally operate will be obtained.

[0081] Figure 1 is an optional flowchart of the customer objection mining method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to steps S101 to S105.

[0082] Step S101, obtaining call text, the call text including known objection text and unknown objection text;

[0083] Step S102, inputting the known objection text to a pre-trained model;

[0084] Step S103, model fine-tuning the pre-trained model based on the call text, calculating vectorization representation of each call text, and obtaining vectorization text;

[0085] Step S104, clustering the vectorization text, classifying vectorization texts with similar semantics into the same category, and obtaining clustering results;

[0086] Step S105, screening the clustering results, and obtaining new customer objections.

[0087] In step S101 of some embodiments, call text is obtained, and the call text includes known objection text and unknown objection text. The known objection text refers to customer objection text that has been identified, and the unknown objection text refers to customer objection text that has not been identified. For example, in insurance telephone sales, the customer saying "I don't need it" can be used as known objection text.

[0088] In step S102 of some embodiments, the known objection text is input to a pre-trained model. The pre-trained model can include a text encoder based on BERT (Bidirection Ecoder Representations from Transformers), a linear layer, and a softmax layer. In model pre-training, the text encoder based on BERT can significantly improve the classification accuracy of known objections and improve the quality of sentence vectors. It should be noted that the initial parameters of the text encoder (BERT) use open source results, and the remaining parameters are randomly initialized. Through the gradient descent algorithm, the optimal model parameters are obtained.

[0089] In step S103 of some embodiments, the pre-trained model is fine-tuned based on the call text, and the vectorization representation of each call text is calculated to obtain vectorization text. In the model fine-tuning stage, training data can be constructed based on text enhancement, providing samples for semi-supervised learning, and using known objection text and unknown objection text in the call text to improve data usage and improve the classification effect of unknown objections. Regularization terms can also be added to the training target to prevent the model training process from classifying all data into one category.

[0090] In step S104 of some embodiments, the vectorized text is clustered, and vectorized texts with similar semantics are classified into the same category, to obtain a clustering result. After obtaining the sentence vector of the text of each unknown complaint in the model fine-tuning step, the KMeans clustering algorithm can be used to divide similar texts into the same category, and finally obtain k different categories as candidate new customer complaints. By automatically determining the number of categories, the clustering effect is improved.

[0091] In step S105 of some embodiments, the clustering result is screened to obtain a new customer complaint. For example, the clustering result can be submitted to the business end, which will finally check and screen to ensure the relevance and compliance of the new complaint, and screen out the final customer complaint that needs to be added. Based on this, the embodiments of the present application can reduce labor costs, and the identification and aggregation of new customer complaints are automatically implemented by the model. Not only can the coverage be improved, but the real-time performance of the newly mined complaints can also be improved, and the t+1 timeliness can be guaranteed.

[0092] The steps S101 to S105 shown in the embodiments of the present application obtain the call text, which includes known complaint text and unknown complaint text; input the known complaint text into the pre-trained model; fine-tune the pre-trained model based on the call text, calculate the vectorized representation of each call text, and obtain the vectorized text; cluster the vectorized text, classify the vectorized texts with similar semantics into the same category, and obtain the clustering result; screen the clustering result to obtain a new customer complaint. Based on this, the model pre-training of the embodiments of the present application is trained based on the known complaint text; the model fine-tuning is based on the known and unknown complaint text to update the model, and the vectorized representation of each text is calculated; then the vectorized text is clustered, so that the texts with similar semantics become the same category, i.e. new customer complaints; finally, the clustering result is screened to select new customer complaints, so as to automatically mine new customer complaints, and solve the problems of high labor cost, small coverage and low real-time performance in manual mining.

[0093] Referring to FIG. 2, in some embodiments, step S102 can include but is not limited to steps S201 to S203:

[0094] Step S201, input the known complaint text into the BERT-based text encoder to calculate the vectorized representation of each character in the known complaint text;

[0095] Step S202, select the vectorized representation corresponding to the first character in each character as the sentence vector representation of the known complaint text;

[0096] Step S203, input the sentence vector representation into the linear layer and the softmax layer to obtain the probability that the known objection text belongs to the i-th objection, where i is a positive integer greater than 1.

[0097] In some embodiments, the pre-trained model includes a BERT-based text encoder, a linear layer and a softmax layer. The model pre-training stage is trained for known objection texts, and the model input is the call text, as shown in Figure 2B Taking the call text "I don't need" as an example, each character in the text is taken as a character, and then a [CLS] character is added at the beginning to indicate the start position of the text, and then the text is input into the text encoder (BERT), as shown in Figure 2C The vector representation of each character is calculated where is a d-dimensional vector, and the vector corresponding to the [CLS] character is selected as the sentence vector representation of the text, which is input into the linear layer. The linear layer and the softmax layer correspond to the following formula:

[0098]

[0099]

[0100] where n is the number of known objections, W1 is a d x n matrix, b1 is an n-dimensional vector, and p i represents the probability that the text belongs to the i-th objection.

[0101] The training objective of the model pre-training is defined as:

[0102]

[0103] where N is the number of known objection texts, B represents all text data, u t represents the t-th data, y represents whether u t belongs to the i-th objection, and 1 if it belongs to, otherwise 0. p(I i |u t ) represents the probability that u t belongs to the i-th objection, which is calculated according to formula eq-2.

[0104] It should be noted that the initial parameters of the text encoder (BERT) use open source results, and the remaining parameters are randomly initialized. The optimal model parameters are obtained by solving through the gradient descent algorithm.

[0105] In some embodiments, the known objectionable text is input into the text encoder BERT to calculate the vectorized representation of each character in the known objectionable text, the vectorized representation corresponding to the first character in each character is selected as the sentence vector representation of the known objectionable text, and the sentence vector representation is input into the linear layer and the softmax layer to obtain the probability that the known objectionable text belongs to the i-th objection, where i is a positive integer greater than 1. The BERT-based text encoder can significantly improve the classification accuracy of known objections and improve the quality of sentence vectors.

[0106] Please refer to Figure 3 In some embodiments, step S103 can include but is not limited to steps S301 to S305:

[0107] Step S301, data augmentation is performed on the data of the call text to obtain augmented data of known objections and augmented data of unknown objections;

[0108] Step S302, the semi-supervised contrastive loss, the supervised loss and the regularization term are determined according to the augmented data of known objections and the augmented data of unknown objections;

[0109] Step S303, the training target of model fine-tuning is determined according to the semi-supervised contrastive loss, the supervised loss and the regularization term;

[0110] Step S304, the pre-trained model is fine-tuned based on the training target of model fine-tuning;

[0111] Step S305, the vectorized representation of each unknown objectionable text is calculated to obtain the vectorized text.

[0112] In some embodiments, the model fine-tuning stage uses both known objectionable texts and unknown objectionable texts. First, the original data is augmented. For known objectionable data, 2 pieces of data belonging to the same objection are randomly extracted; for unknown objectionable data, 2 pieces of data are generated by randomly replacing synonyms. In this way, any original data u corresponds to 2 augmented data u1 and u2.

[0113] The structure of model fine-tuning is the same as that of the pre-training stage (as shown in Figure 2B ), the parameters of the text encoder (BERT) remain the same, and the parameters of the linear layer are adjusted as follows:

[0114]

[0115] Where m is the number of unknown objections, W2 is a d x (m + n) matrix, and b2 is a (m + n) dimensional vector.

[0116] The training target of model fine-tuning consists of three parts: semi-supervised contrastive loss (L ssc ), supervised loss (Lsrl ), a regularization term (L em ).

[0117]

[0118]

[0119] where B represents all original data, B' represents all enhanced data, N(t) represents all data in B' except u t t; if u t is enhanced data of known objection, P(u t ) represents all data in B' belonging to the same objection as u t ; if u t is enhanced data of unknown objection, P(u t ) represents another piece of enhanced data. p(I i |u t ) represents the probability of u t belonging to the i-th objection, p(I|u t ) = [p(I 1 |u t ), …, p(I m+n |u t )] represents the (m+n)-dimensional probability vector of u t , τ is an adjustable parameter, which can be set to 1.0 in the present application.

[0120] The final training target is:

[0121] L = L ssc + L srl - λ·L em … eq-8

[0122] where λ is an adjustable parameter, which is set to 0.1 in the present application.

[0123] The initial parameters of the text encoder (BERT) are obtained using the model pre-training results, and the remaining parameters are randomly initialized. The optimal parameters of the model are obtained by solving through the gradient descent algorithm.

[0124] After the model fine-tuning is completed, the sentence vector s of each piece of text of unknown objection is calculated.

[0125] In some embodiments, in the model fine-tuning stage, data enhancement is performed on the call text data to obtain enhanced data of known objections and enhanced data of unknown objections; semi-supervised contrast loss, supervised loss and regularization term are determined according to the enhanced data of known objections and the enhanced data of unknown objections; the training target of model fine-tuning is determined according to the semi-supervised contrast loss, the supervised loss and the regularization term; the pre-trained model is fine-tuned based on the training target of model fine-tuning; and the vectorized representation of each unknown objection text is calculated to obtain the vectorized text. In the model fine-tuning stage, the training data is constructed based on text enhancement, the samples of semi-supervised learning are provided, and the known objection texts and the unknown objection texts in the call text are used to improve the data utilization rate and improve the classification effect of unknown objections. The regularization term can also be added to the training target to prevent the model training process from classifying all data into one category.

[0126] Please refer to Figure 4 In some embodiments, step S104 can include but is not limited to steps S401 to S405:

[0127] Step S401, obtaining a sentence vector of unknown objection text;

[0128] Step S402, determining the number of categories of the clustering algorithm;

[0129] Step S403, for each data, calculating the distance between each data and the initial center of multiple categories, and classifying the data into the nearest category;

[0130] Step S404, for each category, calculating the average value of the sentence vectors of all data in the category as a new category center until the category division of all data no longer changes;

[0131] Step S405, using the clustering algorithm to divide similar texts into the same category to obtain the clustering result.

[0132] In some embodiments, in the model fine-tuning step, the sentence vector of each unknown objection text is obtained After that, the clustering algorithm is used, and the specific steps are as follows:

[0133] ① Randomly extract k data from the sentence vector as the initial center of k categories:

[0134] {X1,…,X k},X i ∈R d

[0135] ② For each data, calculate the distance between the data and the k centers, and the distance calculation formula is:

[0136] d(i,j)=||h i -Xj || 2 j = 1, …, k, where h i is the sentence vector of the ith data.

[0137] ③ For each data, it is classified into the nearest class.

[0138] ④ For each class, the average of the sentence vectors of all data in the class is calculated as the new class center.

[0139] ⑤ Repeat steps 2-4 until the class division of all data no longer changes.

[0140] After the clustering algorithm, similar texts are divided into the same class, and finally k different classes are obtained as candidate new customer objections. The embodiment of the application determines the number of classes automatically, improving the clustering effect.

[0141] Please refer to Figure 5 In some embodiments, step S402 can include but is not limited to steps S501-S503:

[0142] Step S501, randomly set a plurality of initial class numbers k', obtaining k' initial classes;

[0143] Step S502, calculate the confidence of each initial class;

[0144] Step S503, select k classes with confidence higher than the pre-set confidence threshold from the k' initial classes as the class number of the clustering algorithm, where k is less than or equal to k'.

[0145] In some embodiments, in the clustering algorithm, the class number k needs to be determined in advance. The general approach is to try different k to run the clustering algorithm respectively, and select k according to the clustering result. The embodiment of the application proposes a new method to determine the class number k, the specific steps are as follows:

[0146] ① Set a larger class number k';

[0147] ② Use the sentence vector of each unknown objection text obtained in the model fine-tuning step as input to run the clustering algorithm;

[0148] ③ Remove the classes with low confidence from the k' classes, and the number of remaining classes that meet the conditions is k, the specific formula is: Where |S i | represents the number of classes belonging to the ith class, and σ(·) is 1 when the condition is met, otherwise it is 0.

[0149] Please refer to Figure 6In some embodiments, step S405 can include, but is not limited to, steps S601 to S602:

[0150] In step S601, similar texts are divided into the same category by using a KMeans clustering algorithm to obtain a classification category.

[0151] In step S602, quality evaluation is performed on all classification categories, and low-quality categories are filtered out to obtain a clustering result.

[0152] In some embodiments, although the clustering algorithm can obtain k categories, the quality of each category is different, the similarity of the data in a high-quality category is high, and the data in the category can be classified as a unified customer objection, but the data in a low-quality category may have differences or even contradictions, and cannot be classified as a unified customer objection. The embodiment of the application divides similar texts into the same category by using a KMeans clustering algorithm to obtain a classification category, performs quality evaluation on all classification categories, filters out low-quality categories, and obtains a clustering result, so as to reduce the workload of the business end.

[0153] Please refer to Figure 7 In some embodiments, step S602 can include, but is not limited to, steps S701 to S704:

[0154] In step S701, the silhouette coefficient of the data in each category is calculated.

[0155] In step S702, the data in each category is arranged in descending order of the silhouette coefficient to generate a category description.

[0156] In step S703, if the silhouette coefficient of the category is lower than a preset silhouette coefficient threshold, the category is determined as a low-quality category.

[0157] In step S704, the low-quality categories are filtered out to obtain a clustering result.

[0158] In some embodiments, although k categories are obtained by the clustering algorithm, it is unknown what content each category specifically represents and what objection it represents, and only by manually checking each piece of data in the category can the meaning of the category be summarized. In the embodiment of the application, the silhouette coefficient s is selected as an index, and the specific calculation formula is as follows:

[0159]

[0160]

[0161]

[0162] wherein S i represents all data of the i th category, and d(i,j) represents the distance between two data.

[0163] The data in each category is arranged in descending order of the silhouette coefficient, and the most representative data in the category can be selected as the textual description of the category. In this way, each category has a corresponding summary, which facilitates screening by the business party.

[0164] The embodiment of the present application proposes a category quality evaluation method, calculates the category silhouette coefficient, and defines it as the average of the silhouette coefficients of all data in the category. If the category silhouette coefficient is lower than the threshold value, the category is directly excluded, low-quality categories are screened out, and the workload of the business end is reduced.

[0165] The customer objection mining method of the present application is further illustrated below in combination with the drawings and specific embodiments.

[0166] As shown in Figure 8 , the present application mainly consists of four modules: model pre-training, model fine-tuning, clustering, and business party screening. The model pre-training is based on known objection texts; the model fine-tuning is based on known and unknown objection texts to update the model and calculate the vectorized representation of each text; then the vectorized texts are clustered so that texts with similar semantics become the same category, i.e. new customer objections; finally, the business party screens and audits the clustering results to select new customer objections.

[0167] 1. Model pre-training

[0168] The model pre-training stage trains on known objection texts, and the model input is the call text. The model pre-training structure is as shown in Figure 2B , taking the call text "I don't need" as an example, each character in the text is taken as a character, then a [CLS] character identifier is added at the beginning to indicate the start of the text, and then the text encoder (BERT) is input, as shown in Figure 2C , the vectorized representation of each character is calculated , where is a d-dimensional vector, and the vector corresponding to the [CLS] character is selected as the sentence vector representation of the text, which is input to the linear layer. The formulas corresponding to the linear layer and the softmax layer are as follows:

[0169]

[0170]

[0171] where n is the number of known objections, W1 is a d x n matrix, b1 is an n-dimensional vector, and p i represents the probability that the text belongs to the i-th objection.

[0172] The training objective of the model pre-training is defined as:

[0173]

[0174] where N is the number of known objections, B represents all text data, u t represents the t-th data, y represents the i-th objection, and p(y t represents whether the i-th objection belongs to the i-th objection, and is 1 if it belongs to the i-th objection, and 0 otherwise. p(y i |u t represents the probability that the i-th objection belongs to the i-th objection, which is calculated according to formula eq-2. t

[0175] It should be noted that the initial parameters of the text encoder (BERT) are obtained from open source results, and the remaining parameters are randomly initialized. Through the gradient descent algorithm, the optimal model parameters are obtained.

[0176] 2. Model fine-tuning

[0177] In the model fine-tuning stage, both known objections and unknown objections are used. First, the original data is augmented. For known objection data, 2 pieces of data belonging to the same objection are randomly selected; for unknown objection data, 2 pieces of data are generated by randomly replacing synonyms. In this way, any original data u corresponds to 2 augmented data u1 and u2.

[0178] The structure of model fine-tuning is the same as that of the pre-training stage (as shown in Figure 2B ), the parameters of the text encoder (BERT) remain the same, and the parameters of the linear layer are adjusted as follows:

[0179]

[0180] where m is the number of unknown objections, W2 is a d x (m + n) matrix, and b2 is a (m + n) dimensional vector.

[0181] The training objective of model fine-tuning consists of three parts: semi-supervised contrastive loss (L ssc ), supervised loss (L srl ), and regularization term (L em ).

[0182]

[0183]

[0184]

[0185] where B represents all original data, B' represents all augmented data, and N(t) represents all data in B' except u t itself. If u t ​P(u) is the known objection-enhanced data, P(u t ) represents all data in B' belonging to the same objection as u t ; if P(u t ) is the unknown objection-enhanced data, P(u t ) represents another piece of objection-enhanced data. p(I i |u t ) represents the probability of u t belonging to the i-th objection, p(I|u t ) = [p(I 1 |u t ), …, p(I m+n |u t )] can be set to 1.0.

[0186] The final training target is the (m+n)-dimensional probability vector of u t , τ is an adjustable parameter, which is set to 0.1 in the present application.

[0187] L = L ssc + L srl - λ·L em … eq-8 where λ is an adjustable parameter, which is set to 0.1 in the present application.

[0188] The initial parameters of the text encoder (BERT) are obtained using the model pre-training result, and the remaining parameters are randomly initialized. The optimal parameters of the model are obtained by solving through the gradient descent algorithm.

[0189] After the model fine-tuning is completed, the sentence vector of each piece of unknown objection text is calculated.

[0190] 3. Clustering

[0191] After obtaining the sentence vector of each piece of unknown objection text in the model fine-tuning step, the clustering algorithm is used, and the specific steps are as follows:

[0192] ① Randomly extract k pieces of data from the sentence vector as the initial centers of k categories:

[0193] {X1, …, X k}, X i ∈R d

[0194] ② For each piece of data, calculate its distance from the k centers, and the distance calculation formula is:

[0195] d(i,j) = ||h i - X j || 2 , j = 1, …, k, where h iSentence vector of the ith data.

[0196] ③ For each data, it is classified into the nearest class.

[0197] ④ For each class, the average of the sentence vectors of all data in the class is calculated as the new class center.

[0198] ⑤ Repeat steps 2-4 until the class division of all data no longer changes.

[0199] After the clustering algorithm, similar texts are divided into the same class, and finally k different classes are obtained as candidate new customer objections.

[0200] In the clustering algorithm, the number of classes k needs to be determined in advance. The general method is to try different k to run the clustering algorithm respectively, and select k according to the clustering result. The embodiment of the present application proposes a new method for determining the number of classes k, and the specific steps are as follows:

[0201] ① Set a larger number of classes k';

[0202] ② Use the sentence vector of each unknown objection text obtained in the model fine-tuning step as input to run the clustering algorithm;

[0203] ③ Remove the classes with low confidence from the k' classes, and the number of remaining classes that meet the conditions is k, and the specific formula is: Where |S i | represents the number of classes belonging to the ith class, and σ(·) is 1 when the condition is met, otherwise it is 0.

[0204] Although k classes are obtained through the clustering algorithm, it is not known what each class specifically represents and what kind of objection it represents. Only by manually checking each data in the class can the meaning of the class be summarized. The embodiment of the present application selects the silhouette coefficient s as an index, and the specific calculation formula is as follows:

[0205]

[0206]

[0207]

[0208] Where S i represents all data in the ith class, and d(i,j) represents the distance between two data.

[0209] The data in each class is arranged in descending order according to the silhouette coefficient, and the most representative data in the class can be selected as the textual description of the class. In this way, each class has a corresponding summary, which facilitates screening by business personnel.

[0210] Although the clustering algorithm can obtain k categories, the quality of each category is different, the data similarity in the high-quality category is high, and it can be classified as a unified customer complaint, but the data in the low-quality category may be different, even contradictory, and cannot be classified as a unified customer complaint. The embodiment of the application proposes a category quality evaluation method, calculates the category silhouette coefficient, which is defined as the average of the silhouette coefficients of all data in the category. If the category silhouette coefficient is lower than the threshold, the category is directly excluded, the low-quality category is screened out, and the workload of the business party is reduced.

[0211] 4. Business party screening

[0212] After submitting the clustering results to the business party, they are responsible for auditing and screening, and manually selecting the final customer complaints that need to be added.

[0213] Based on this, the embodiment of the application obtains call text, the call text includes known complaint text and unknown complaint text; inputs the known complaint text into a pre-trained model; performs model fine-tuning on the pre-trained model based on the call text, calculates the vectorization representation of each call text, and obtains vectorized text; clusters the vectorized text, classifies the vectorized text with similar semantics into the same category, and obtains clustering results; screens the clustering results to obtain new customer complaints. Based on this, the model pre-training of the embodiment of the application is trained based on the text of known complaints; the model fine-tuning is based on the model updating of known and unknown complaints, and the vectorization representation of each text is calculated; then the vectorized text is clustered, so that the texts with similar semantics become the same category, i.e. new customer complaints; finally, the clustering results are screened to select new customer complaints, so as to automatically mine new customer complaints, and solve the problems of high labor cost, small coverage and low real-time performance in current manual mining.

[0214] Please refer to Figure 9 The embodiment of the application also provides a customer complaint mining device, which can realize the above-mentioned customer complaint mining method, and the device comprises:

[0215] The acquisition module 910 is configured to acquire call text, and the call text includes known complaint text and unknown complaint text;

[0216] The pre-training module 920 is configured to input the known complaint text into a pre-trained model;

[0217] The fine-tuning module 930 is configured to perform model fine-tuning on the pre-trained model based on the call text, calculate the vectorization representation of each call text, and obtain vectorized text;

[0218] The clustering module 940 is configured to cluster the vectorized text, classify the vectorized text with similar semantics into the same category, and obtain clustering results.

[0219] The screening module 950 is configured to screen the clustering result to obtain the new customer objection.

[0220] In some embodiments of the present application, the obtaining module 910 obtains the call text, and the call text includes known objection text and unknown objection text; the pre-training module 920 inputs the known objection text into the pre-training model; the fine-tuning module 930 performs model fine-tuning on the pre-training model based on the call text, calculates the vectorization representation of each call text, and obtains the vectorization text; the clustering module 940 clusters the vectorization text, classifies the vectorization texts with similar semantics into the same category, and obtains the clustering result; and the screening module 950 screens the clustering result to obtain the new customer objection.

[0221] In some embodiments of the present application, the obtaining module 910 obtains the call text, and the call text includes known objection text and unknown objection text. The known objection text refers to the customer objection text that has been identified, and the unknown objection text refers to the customer objection text that has not been identified. For example, in insurance telephone sales, the customer saying “I don't need” can be used as the known objection text.

[0222] In some embodiments of the present application, the pre-training module 920 inputs the known objection text into the pre-training model. The pre-training model can include a text encoder based on BERT (Bidirection Ecoder Representations from Transformers), a linear layer, and a softmax layer. In the model pre-training, the text encoder based on BERT can significantly improve the classification accuracy of the known objection and improve the quality of the sentence vector. It should be noted that the initial parameters of the text encoder (BERT) use open source results, and the remaining parameters are randomly initialized. Through the gradient descent algorithm, the optimal model parameters are obtained.

[0223] In some embodiments of the present application, the fine-tuning module 930 performs model fine-tuning on the pre-training model based on the call text, calculates the vectorization representation of each call text, and obtains the vectorization text. In the model fine-tuning stage, the training data can be constructed based on text enhancement, and samples for semi-supervised learning are provided, and the known objection text and the unknown objection text in the call text are used to improve the data usage rate and improve the classification effect of the unknown objection. The regularization term can also be added to the training target to prevent the model training process from classifying all data into one category.

[0224] In some embodiments of the present application, the clustering module 940 clusters the vectorized text, classifies vectorized texts with similar semantics into the same category, and obtains clustering results. After obtaining the sentence vector of the text of each unknown complaint in the model fine-tuning step, the KMeans clustering algorithm can be used to divide similar texts into the same category, and finally obtain k different categories as candidate new customer complaints. By automatically determining the number of categories, the clustering effect is improved.

[0225] In some embodiments of the present application, the screening module 950 screens the clustering results to obtain new customer complaints. For example, the clustering results can be submitted to the business end, and finally screened by the business end to ensure the relevance and compliance of the new complaints through auditing and screening, and the final customer complaints that need to be added are screened out. Based on this, the embodiments of the present application can reduce labor costs, and the identification and aggregation of new customer complaints are automatically realized by the model, which not only can improve the coverage rate and mine the full amount of call data, but also can improve the real-time performance of the newly mined complaints, and can guarantee the t+1 time limit.

[0226] Based on this, the customer complaint mining device of the embodiments of the present application, the acquisition module 910 acquires call text, the call text includes known complaint text and unknown complaint text; the pre-training module 920 inputs the known complaint text into the pre-training model; the fine-tuning module 930 performs model fine-tuning on the pre-training model based on the call text, calculates the vectorized representation of each call text, and obtains vectorized text; the clustering module 940 clusters the vectorized text, classifies vectorized texts with similar semantics into the same category, and obtains clustering results; and the screening module 950 screens the clustering results to obtain new customer complaints. Through the acquisition of call text, the call text includes known complaint text and unknown complaint text; the known complaint text is input into the pre-training model; the model fine-tuning is performed on the pre-training model based on the call text, the vectorized representation of each call text is calculated, and the vectorized text is obtained; the vectorized text is clustered, the vectorized texts with similar semantics are classified into the same category, and the clustering results are obtained; and the clustering results are screened to obtain new customer complaints. Based on this, the model pre-training of the embodiments of the present application is trained based on the known complaint text; the model fine-tuning is performed based on the known and unknown complaint texts to update the model, and the vectorized representation of each text is calculated; then the vectorized text is clustered, so that the texts with similar semantics become the same category, i.e. new customer complaints; finally, the clustering results are screened to select new customer complaints, so as to automatically mine new customer complaints, and solve the problems of high labor cost, small coverage and low real-time performance in the current manual mining.

[0227] The specific embodiments of the customer complaint mining device are basically the same as the specific embodiments of the customer complaint mining method described above, and will not be repeated here.

[0228] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the customer objection mining method described above when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0229] Please refer to Figure 10 , Figure 10 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0230] The processor 1001 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0231] The memory 1002 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1002 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 1002 and called and executed by the processor 1001 to implement the customer objection mining method of the embodiments of the present application, that is, by obtaining call texts, the call texts comprising known objection texts and unknown objection texts; inputting the known objection texts into a pre-trained model; performing model fine-tuning on the pre-trained model based on the call texts, calculating the vectorization representation of each call text, obtaining vectorized texts; clustering the vectorized texts, classifying the vectorized texts with similar semantics into the same category, obtaining clustering results; screening the clustering results to obtain new customer objections. Based on this, the model pre-training of the embodiments of the present application is based on the texts of known objections for training; the model fine-tuning is based on the texts of known and unknown objections for model updating, and the vectorization representation of each text is calculated; then the vectorized texts are clustered, so that the texts with similar semantics become the same category, that is, new customer objections; finally, the clustering results are screened to select new customer objections, so as to automatically mine new customer objections, solving the problems of high labor cost, small coverage and low real-time performance in current manual mining.

[0232] The input / output interface 1003 is used to realize information input and output.

[0233] The communication interface 1004 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable, or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like).

[0234] The bus is configured to transmit information between various components (for example, the processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004) of the device.

[0235] The processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004 are connected to each other through the bus to realize the communication connection between the device.

[0236] The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the computer program is executed by the processor to realize the customer objection mining method.

[0237] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0238] The customer objection mining method, the customer objection mining device, the electronic device, and the storage medium provided in the embodiment of the present application are configured to obtain a call text, the call text including known objection texts and unknown objection texts; input the known objection texts into a pre-training model; perform model fine-tuning on the pre-training model based on the call texts, calculate a vectorization representation of each call text, and obtain a vectorization text; perform clustering on the vectorization text, classify vectorization texts with similar semantics into the same category, and obtain a clustering result; and perform screening on the clustering result, and obtain newly added customer objections. Based on this, the model pre-training in the embodiment of the present application is trained based on the texts of known objections; the model fine-tuning is performed based on the texts of known and unknown objections to update the model, and the vectorization representation of each text is calculated; then the vectorization text is clustered, so that the texts with similar semantics become the same category, that is, new customer objections; finally, the clustering result is screened, and the newly added customer objections are selected, so that the newly added customer objections can be automatically mined, and the problems of high labor cost, small coverage, and low real-time performance in the current manual mining are solved.

[0239] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented by software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable programs, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable programs, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0240] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0241] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine some steps or different steps.

[0242] The device embodiments described above are only schematic, and the units illustrated as separate components can or can not be physically separate, i.e. can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0243] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0244] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a recited step or its integral sub-steps or additional steps whether or not readily ascertainable from the description or the like. Further, the words "a" or "an", as used herein in the disclosure and elsewhere, are used indiscriminately and are to be interpreted in the same way, i.e. as meaning "one or more".

[0245] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or", used to describe the relationship between associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including single or multiple combinations. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0246] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0247] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0248] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0249] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0250] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A customer objection mining method, characterized by, The method comprises: acquiring call text, the call text comprising known objection text and unknown objection text; inputting the known objection text into a pre-training model; performing data enhancement on data of the call text to obtain enhanced data of the known objection text and enhanced data of the unknown objection text; determining a semi-supervised contrastive loss, a supervised loss and a regularization term according to the enhanced data of the known objection text and the enhanced data of the unknown objection text; determining a training target of model fine-tuning according to the semi-supervised contrastive loss, the supervised loss and the regularization term; fine-tuning the pre-training model based on the training target of model fine-tuning, calculating vectorized representations of each of the unknown objection texts to obtain vectorized text; wherein the pre-training model comprises a BERT-based text encoder, a linear layer and a softmax layer, the structure of model fine-tuning is the same as that in the pre-training stage, the parameters of the text encoder remain unchanged, the parameters of the linear layer are adjusted, and the dimension of the matrix of the linear layer is d x (m + n), d is the vector dimension, n is the number of texts of the known objection text, and m is the number of texts of the unknown objection text; performing clustering on the vectorized text, classifying the vectorized text with similar semantics into the same category to obtain clustering results, comprising: clustering similar texts according to sentence vectors of the unknown objection texts, dividing the similar texts into the same category by using a KMeans clustering algorithm to obtain a classification category, calculating the silhouette coefficient of data in each classification category, arranging the data in each classification category in descending order of the silhouette coefficient, generating a category description, and determining that the classification category is a low-quality category if the silhouette coefficient of the classification category is lower than a preset silhouette coefficient threshold, and excluding the low-quality category to obtain the clustering results; screening the clustering results to obtain newly added customer objections.

2. The method of claim 1, wherein, The method comprises: inputting the known objection text into a BERT-based text encoder to calculate the vectorized representation of each character in the known objection text; selecting the vectorized representation corresponding to the first character in each character as the sentence vector representation of the known objection text; inputting the sentence vector representation into a linear layer and a softmax layer to obtain the probability that the known objection text belongs to the i-th objection, wherein i is a positive integer greater than 1.

3. The method of claim 1, wherein, The method comprises: obtaining the sentence vector of the unknown objection text and taking the unknown objection text as data; determining the number of categories of the clustering algorithm; calculating the distance between each data and the initial center of multiple categories, and classifying the data into the nearest category; calculating the average value of the sentence vectors of all data in each category as a new category center until the category division of all data no longer changes to obtain similar texts.

4. The method of claim 3, wherein, The method comprises: randomly setting multiple initial category numbers k' to obtain k' initial categories; calculate a confidence level of each of the initial categories; select k categories with a confidence level higher than a preset confidence threshold from the k' initial categories as the number of categories of the clustering algorithm, where k is less than or equal to k'.

5. A client objection mining apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire call text, wherein the call text comprises known objection text and unknown objection text. A pre-training module is configured to input the known objection text into a pre-training model. A fine-tuning module is configured to perform data augmentation on data of the call text to obtain augmented data of the known objection text and augmented data of the unknown objection text, determine a semi-supervised contrastive loss, a supervised loss, and a regularization term based on the augmented data of the known objection text and the augmented data of the unknown objection text, determine a training target for model fine-tuning based on the semi-supervised contrastive loss, the supervised loss, and the regularization term, fine-tune the pre-training model based on the training target for model fine-tuning, and calculate a vectorized representation of each of the unknown objection texts to obtain vectorized text; wherein the pre-training model comprises a BERT-based text encoder, a linear layer, and a softmax layer, the structure of model fine-tuning is the same as that in the pre-training phase, the parameters of the text encoder remain unchanged, and the dimension of the matrix of the linear layer is d x (m+n), where d is a vector dimension, n is the number of texts of the known objection text, and m is the number of texts of the unknown objection text. A clustering module is configured to cluster the vectorized text, classify the vectorized text with similar semantics into the same category, and obtain a clustering result, including: clustering similar texts based on sentence vectors of the unknown objection texts, dividing the similar texts into the same category by using a KMeans clustering algorithm to obtain a classification category, calculating a silhouette coefficient of data in each of the classification categories, arranging the data in each of the classification categories in descending order of the silhouette coefficient, generating a category description, determining a low-quality category when the silhouette coefficient of the classification category is lower than a preset silhouette coefficient threshold, filtering out the low-quality category, and obtaining the clustering result. A screening module is configured to screen the clustering result to obtain newly added customer objections.

6. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the customer objection mining method in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the customer objection mining method in any one of claims 1 to 4.

Images