Intelligent customer service question classification method and device, electronic equipment and storage medium

Abstract

The application discloses an intelligent customer service question classification method and device, and belongs to the technical field of computers. The method comprises the following steps: inputting a target customer service question into a pre-trained first intention recognition model, performing feature coding and classification mapping on the target customer service question through the first intention recognition model, and obtaining a plurality of question category prediction values matched with the target customer service question and having hierarchical membership relationships; comparing the plurality of question category prediction values with question category true values set for each reply information of an automatic customer service, and obtaining a comparison result; and determining a response category matched with the target customer service question according to the comparison result, so as to complete the classification of the target customer service question. The method adopts multi-level intention prediction of customer service questions, predicts the user intention contained in the customer service question from rough to fine, and the prediction result is more accurate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to intelligent customer service problem classification methods, apparatus, electronic devices, and computer-readable storage media. Background Technology

[0002] Customer service plays a crucial role in the travel industry. Providing timely and accurate responses to customer inquiries not only helps resolve issues but also enhances the customer experience. However, given the large customer base, there is a significant gap between the number of human customer service representatives and their available service hours. Therefore, providing an intelligent customer service solution offers 24 / 7 uninterrupted service by automating processes and autonomously resolving simple, repetitive questions, thus reducing the backlog of customer service inquiries for human agents. Accurately identifying which customer service questions can be answered by automated systems and which require human assistance is a key aspect of improving customer service efficiency. Summary of the Invention

[0003] This application provides an intelligent customer service question classification method, device, electronic device, and storage medium, which can accurately identify customer service questions that can be answered by automated customer service and those that require human customer service. This provides a basis for distributing customer service questions, improves the accuracy of customer service question distribution, avoids distributing customer service questions that do not need to be answered by human customer service to human customer service, thereby reducing the backlog of customer service questions for human customer service and improving customer service efficiency.

[0004] In a first aspect, embodiments of this application provide an intelligent customer service question classification method, including:

[0005] The target customer service question is input into a pre-trained first intent recognition model. The first intent recognition model performs feature encoding and classification mapping on the target customer service question to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question.

[0006] The predicted values ​​of the multiple question categories are compared with the actual values ​​of the question categories set for each reply message preset for the automated customer service, and the comparison results are obtained.

[0007] Based on the comparison results, the response category matching the target customer service question is determined.

[0008] Secondly, embodiments of this application provide an intelligent customer service question classification device, comprising:

[0009] The prediction module is used to input the target customer service question into a pre-trained first intent recognition model, and to perform feature encoding and classification mapping on the target customer service question through the first intent recognition model to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question.

[0010] The prediction result comparison module is used to compare the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each reply message preset for the automatic customer service, and obtain the comparison results;

[0011] The classification result determination module is used to determine the response category that matches the target customer service question based on the comparison results.

[0012] Thirdly, embodiments of this application also disclose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent customer service problem classification method described in embodiments of this application.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, represents the steps of the intelligent customer service problem classification method disclosed in embodiments of this application.

[0014] The intelligent customer service question classification method disclosed in this application involves inputting a target customer service question into a pre-trained first intent recognition model. The first intent recognition model performs feature encoding and classification mapping on the target customer service question to obtain predicted values ​​for multiple question categories with hierarchical relationships. These predicted values ​​are then compared with the actual question category values ​​set for each pre-defined response message in the automated customer service system to obtain a comparison result. Based on the comparison result, the corresponding response category for the target customer service question is determined, thus completing the classification of the target customer service question. This method employs multi-level intent prediction for customer service questions, predicting the user intent contained within the customer service question from coarse to fine, resulting in more accurate predictions compared to directly predicting a single user intent.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] Figure 1 This is a flowchart of the intelligent customer service question classification method disclosed in the embodiments of this application;

[0018] Figure 2 This is a schematic diagram of the first intent recognition model structure disclosed in the embodiments of this application;

[0019] Figure 3 This is a schematic diagram of the second intent recognition model structure disclosed in the embodiments of this application;

[0020] Figure 4 This is a schematic diagram of the intelligent customer service problem classification device disclosed in the embodiments of this application;

[0021] Figure 5 A block diagram schematically illustrates an electronic device for performing the method according to this application; and

[0022] Figure 6 A storage unit for holding or carrying program code implementing the method according to this application is illustrated schematically. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] This application discloses an intelligent customer service question classification method, such as... Figure 1 As shown, the method includes steps 110 to 130.

[0025] Step 110: Input the target customer service question into the pre-trained first intent recognition model, and use the first intent recognition model to perform feature encoding and classification mapping on the target customer service question to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question.

[0026] In the embodiments of this application, a pre-trained neural network model, namely a first intent recognition model, is used to classify and identify customer service questions. This first intent recognition model is a neural network model built on the BERT model, possessing multi-classification capabilities. The BERT model is a pre-trained model, short for Bidirectional Encoder Representations from Transformers. For example, based on a customer service corpus, a pre-trained BERT model is fine-tuned. The final network structure of the model is modified, and the customer service question is used as input. The model predicts the multiple levels within the corpus to which the customer service question belongs.

[0027] The customer service question matching described in this application embodiment has a hierarchical relationship from coarse to fine. The fine question category is a further refinement of the coarse question category. For example, for a geographically related customer service question, the prediction result output by the first intent recognition model can be: region A in the first level, region B in the second level, and region C in the third level, where region B belongs to region A, and region C belongs to region B.

[0028] like Figure 2 As shown, the first intent recognition model may include a feature extraction network 210 and a classification network 220. The feature extraction network 210 may employ a BERT model, and is used to extract textual and semantic features of the input customer service question. The classification network 220 includes multiple network branches, each branch being used to perform feature mapping on the textual features extracted by the feature extraction network 210, obtaining multiple question category prediction values ​​at different levels matching the input customer service question.

[0029] like Figure 2As shown, the feature extraction network 210 further includes: a vectorization layer 2101, a feature encoding layer 2102, and a pooling layer 2103. The vectorization layer 2101 is used to vectorize the input customer service question to obtain a vector representation of the customer service question; the feature encoding layer 2102 is used to extract and encode features from the vector representation output by the vectorization layer 2101 to obtain a hidden layer vector; the pooling layer 2103 performs feature compression on the hidden layer vector output by the feature encoding layer 2102 to obtain a feature vector of the input customer service question. The feature encoding layer 2102 includes a multi-layer encoding structure. Each branch of the classification network 220 further includes: a fully connected layer 2201 and an activation function 2202, used to classify and map the feature vector output by the feature extraction network 210 within the corresponding level's category space, outputting the probability value of the customer service question belonging to different question categories within the corresponding level's category space.

[0030] based on Figure 2 The model structure shown includes the following steps: inputting the target customer service question into a pre-trained first intent recognition model; performing feature encoding and classification mapping on the target customer service question through the first intent recognition model to obtain predicted values ​​of multiple question categories with hierarchical membership matching the target customer service question; inputting the target customer service question into the pre-trained first intent recognition model; extracting and encoding features of the target customer service question through the feature extraction task of the first intent recognition model to obtain a feature vector; and independently classifying and mapping the feature vector in a question category space of corresponding levels from coarse to fine through multiple classification tasks of the first intent recognition model to obtain predicted values ​​of multiple question categories with hierarchical membership matching the target customer service question, wherein each classification task corresponds to a level of a preset question category.

[0031] Optionally, the feature extraction task is performed based on the feature extraction network 210, used to extract and encode features of the target customer service question to obtain a feature vector. Optionally, the feature extraction task can be performed in two stages: In the first stage, the feature extraction task is performed based on the vectorization layer 2101, used to vectorize the input customer service question to obtain a vector representation of the customer service question; In the second stage, the feature extraction task performs feature extraction and encoding on the vector representation output from the first stage based on the feature encoding layer 2102, and then performs feature compression based on the pooling layer 2103 to obtain the feature vector of the input customer service question.

[0032] Since neural network models cannot directly process Chinese text, after the text sequence expressing a customer service question is input into the first intent recognition model, in the first stage, the feature extraction task converts the Chinese text sequence into a vector for feature extraction and encoding processing in the second stage. For example, after the customer service question "How to buy coupons" is input into the first intent recognition model, the feature extraction task first converts the Chinese text sequence "How to buy coupons" into a vector representation: [0.1, 0.221, 0.31, 0.5, ..., 0.823]. Optionally, the vector representation consists of a word embedding tensor, a sentence block tensor, and a position encoding tensor of the text sequence.

[0033] Next, the feature extraction task further performs multi-layer feature extraction and semantic parsing on the transformed vector representation, and finally performs mapping processing to obtain the feature vector of the input customer question.

[0034] The first intent recognition model's feature extraction task extracts and encodes features of the target customer service question to obtain feature vectors. For a specific implementation scheme, refer to the prior art scheme of the BERT model for extracting feature vectors from input text sequences, which will not be repeated here.

[0035] Next, multiple classification tasks of the first intent recognition model are executed independently based on their respective network branches, and the feature vectors are classified and mapped in their respective classification spaces. Taking the first intent recognition model as having K network branches as an example, each classification task is executed based on one network branch, and the K classification tasks independently classify and map the input customer service question at K question levels, obtaining K sets of classification results corresponding to the K question levels. The K question levels have a hierarchical relationship. For example, the first question level is the coarsest question level, corresponding to the largest question category classification granularity; the second question level is the second coarsest question level, corresponding to the second largest question category classification granularity; and the Kth question level is the finest question level, corresponding to the finest question category. The lower the question level, the finer the corresponding question category, and the question category in the kth question level is a further finer classification of the question category in the (k-1)th question level. Here, K is a natural number greater than 1, and k is a natural number greater than or equal to 1 and less than or equal to K.

[0036] For a specific example, taking K=3 as an example, the prediction question hierarchy is assumed as follows: the coarsest question hierarchy (e.g., the first question hierarchy) corresponds to 1 question category, such as a points question; the second coarsest question hierarchy (e.g., the second question hierarchy) corresponds to 2 question categories, such as points redemption questions and points usage questions; the most refined question hierarchy (e.g., the third question hierarchy) corresponds to 5 question categories, including 2 question categories belonging to points redemption questions (e.g., points redemption period and points redemption method) and 3 question categories belonging to points usage questions (e.g., points usage product, points usage method, and points usage period). Accordingly, the first intent recognition model will set up 3 network branches, each corresponding to a classification task, for example, denoted as: the first classification task, the second classification task, and the third classification task, predicting the classification result for each question hierarchy respectively. After inputting the customer service question "Time limit for using points" into the first intent recognition model, the first classification task of the first intent recognition model can output the probability value of the customer service question "Time limit for using points" matching the first-level question category of points questions; the second classification task can output the probability value of the customer service question "Time limit for using points" matching the two second-level question categories of points redemption questions and points usage questions; the third classification task can output the probability value of the customer service question "Time limit for using points" matching the five third-level question categories of points redemption period, points redemption method, products used for points, points usage method, and points usage period.

[0037] Furthermore, based on the probability values ​​of the customer service question matching each level of question category, the question categories matching each level can be obtained. For example, the question category with the highest probability value in each level can be selected as the predicted question category value for each level of customer service question matching. Continuing with the customer service question and question category levels mentioned earlier, based on the probability values ​​of the three levels output by the first intent recognition model for the customer service question "Points Usage Time Limit," the three levels of question categories matching the customer service question "Points Usage Time Limit" can be obtained as follows: Points Question, Points Usage Question, and Points Usage Period.

[0038] Step 120: Compare the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each reply message preset for the automated customer service, and obtain the comparison results.

[0039] Optionally, the automated customer service can be an artificial intelligence customer service, such as a customer service robot.

[0040] In the embodiments of this application, an automated customer service response database can be pre-established, storing automated customer service response information and setting a question category tag for each response. The question category tag includes the actual value of each level of question category to which the customer service question corresponding to the response belongs. For example, for the response "Points are valid for one year from the date of redemption," the first-level question category to which its corresponding customer service question belongs can be set as "Points Question," the second-level question category as "Points Usage Question," and the third-level question category as "Points Usage Period."

[0041] After obtaining the predicted values ​​of the problem categories at each level of the target customer service problem, the predicted values ​​of the multiple problem categories can be compared with the problem category labels of the response information that are set in advance, i.e. the actual values ​​of the problem categories, to obtain the comparison results.

[0042] Optionally, when comparing the predicted values ​​of problem categories with the actual values ​​of problem categories, either a hierarchical comparison method or a group-by-group comparison method can be used. Examples of these two comparison methods are provided below.

[0043] In some embodiments of this application, comparing the predicted values ​​of the plurality of question categories with the actual values ​​of the question categories set for each preset reply message of the automated customer service to obtain a comparison result includes: obtaining a set of actual values ​​of question categories with hierarchical membership corresponding to each preset reply message of the automated customer service; performing a progressively deeper category comparison between the predicted values ​​of the plurality of question categories and the obtained sets of actual values ​​of the question categories in order from coarse to fine hierarchy; obtaining a comparison result indicating successful comparison if the predicted values ​​of the plurality of question categories and any set of actual values ​​of the question categories meet a preset similarity condition; and obtaining a comparison result indicating failed comparison if neither the predicted values ​​of the plurality of question categories nor the actual values ​​of the question categories meet the preset similarity condition.

[0044] For example, for each reply, the true value of the question category corresponding to each reply can be obtained all at once. These true values ​​have a hierarchical relationship; for example, each reply can correspond to K question categories ranging from coarse to fine. Then, the predicted value of the coarsest question category among the predicted values ​​is compared one by one with the true value of the coarsest question category corresponding to each reply, obtaining replies that match. Next, the predicted value of the second coarsest question category is compared one by one with the true value of the second coarsest question category corresponding to each matching reply, obtaining replies that match a second time. This process continues until the comparison of the finest question category is completed.

[0045] If, after comparing the most refined question categories, there are matching responses, then the predicted values ​​of the multiple question categories and at least one set of actual question category values ​​satisfy a preset similarity condition, and a comparison result indicating a successful comparison can be obtained. If, after comparing the most refined question categories, there are no matching responses, then the predicted values ​​of the multiple question categories and all actual question category values ​​satisfy a preset similarity condition, and a comparison result indicating a failed comparison can be obtained.

[0046] Optionally, the preset similarity condition can also be that the coarse question categories are all consistent, and the similarity of the finest question category is greater than a preset similarity threshold. For example, when the predicted question category value for the coarsest question category is "points problem" and the predicted question category value for the second coarser question category is "points usage problem," the range of response information for the customer service question can be determined through the prediction results of the first two levels. Then, the similarity between the response information under the "points usage problem" level and the predicted question category value of the corresponding category level of the target customer service question is calculated. Furthermore, if the calculated similarity is greater than the preset similarity threshold, the predicted value output by the model is considered to have successfully matched the true value of the question category of the response information within the specified range; otherwise, the comparison is considered to have failed.

[0047] Optionally, the matching response information can be used as the automatic response information for the target customer service question.

[0048] In some other embodiments of this application, the step of comparing the predicted values ​​of the plurality of question categories with the actual values ​​of the question categories set for each preset reply message of the automated customer service to obtain a comparison result includes: obtaining a set of actual values ​​of question categories with hierarchical membership corresponding to each preset reply message of the automated customer service; generating a first category sequence corresponding to each reply message according to the membership level of the question categories and the actual values ​​of the question categories corresponding to each reply message; generating a second category sequence according to the membership level of the question categories and the predicted values ​​of the plurality of question categories; performing consistency comparison between the second category sequence and the first category sequence in sequence; obtaining a comparison result indicating successful comparison in response to the second category sequence being consistent with any one of the first category sequences; and obtaining a comparison result indicating failed comparison in response to the second category sequence and the first category sequence being inconsistent with each other.

[0049] For example, for each reply, the true values ​​of the question categories corresponding to each reply can be obtained all at once. These true values ​​have a hierarchical relationship; for example, each reply can correspond to N question categories ranging from coarse to fine. Then, for each reply, the true values ​​of the question categories corresponding to that reply are arranged from front to back according to the hierarchical relationship of coarse to fine or fine to coarse, resulting in a first category sequence for that reply. The predicted values ​​of the multiple question categories are arranged from front to back in the same manner to obtain a second category sequence. Next, the second category sequence is sequentially compared with the generated first category sequence for consistency until it matches a first category sequence, or until it matches all first category sequences.

[0050] After class sequence alignment, if the second class sequence matches a certain first class sequence, an alignment result indicating successful alignment can be obtained; if the second class sequence does not match any of the first class sequences, an alignment result indicating failed alignment can be obtained.

[0051] Optionally, the response information corresponding to the first category sequence that matches the comparison can be used as the automatic response information for the target customer service question.

[0052] Step 130: Based on the comparison results, determine the response category that matches the target customer service question.

[0053] Optionally, based on the comparison result, the response category matched by the target customer service question is determined, including: in response to the comparison result indicating a successful comparison, determining that the target customer service question is an automated customer service response question; in response to the comparison result indicating a failed comparison, determining that the target customer service question is a human customer service response question.

[0054] As mentioned above, if the comparison result indicates a successful comparison, it means that there is a corresponding reply in the automatic customer service's automatic reply information database. In this case, the target customer service question can be considered an automatic customer service reply question, and the target customer service question can be dispatched to the automatic customer service for processing to reduce the pressure on human customer service.

[0055] If the comparison result indicates a failure, it means that there is no corresponding reply information in the automatic customer service's automatic reply information database. Therefore, the target customer service question is considered to be a question requiring a human customer service reply and needs to be dispatched to a human customer service representative for processing.

[0056] In some other embodiments of this application, when determining the response category matching the target customer service question based on the comparison result, the method further includes: in response to the comparison result indicating a successful comparison, determining the target customer service question as an automated customer service response question; in response to the comparison result indicating a failed comparison, determining the response category matching the target customer service question based on the task saturation of human customer service and / or the comparison result obtained by comparing the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each reply message preset for automated customer service.

[0057] For example, when the task saturation of human customer service is high, the preset similarity condition when comparing the predicted value of the question category with the actual value of the question category set for each reply message preset for the automatic customer service can be reduced. The predicted values ​​of the multiple question categories can be compared with the actual values ​​of the question categories set for each reply message preset for the automatic customer service to obtain a second comparison result. Based on the second comparison result, the response category matched with the target customer service question can be determined.

[0058] This completes the categorization of the target customer service issues.

[0059] To make the solution of this application clearer, the training scheme of the first intent recognition model will be described below.

[0060] In some embodiments of this application, the first intent recognition model is trained using the following method: a second intent recognition model including multiple classification tasks is constructed based on a pre-trained BERT model; the second intent recognition model is fine-tuned based on a preset training sample set, wherein each classification task is used to output the predicted question category value of each training sample, and the predicted question category value of each training sample has a hierarchical membership relationship; the loss value of the second intent recognition model is calculated by weighting and summing the predicted loss value of each classification task for the training sample based on the weights corresponding to the classification tasks, wherein the weights are related to the hierarchical membership relationship of the predicted question category value output by the classification tasks; some hidden layers of the fine-tuned second intent recognition model are transferred to obtain the first intent recognition model; the fine-tuned second intent recognition model is used as the teacher network, and the first intent recognition model is used as the student network, and the first intent recognition model is trained using a knowledge distillation architecture.

[0061] In the embodiments of this application, a second intent recognition model with a large number of parameters is first trained based on training data. Then, some network layer weights of the encoding part of the second intent recognition model are extracted, and the network is reconstructed to form a first intent recognition model. Therefore, the network structure of the second intent recognition model is similar to that of the first intent recognition model, both being built based on the BERT model. On the basis of the original BERT model, the final network structure of the model is modified to form multiple network branches.

[0062] like Figure 3 As shown, the second intent recognition model may include a feature extraction network 310 and a classification network 320. The feature extraction network 310 may employ a BERT model and is used to extract textual features of the input customer service question. The classification network 320 includes multiple network branches, each branch being used to perform feature mapping on the textual features extracted by the feature extraction network 310, obtaining multiple question category prediction values ​​at different levels matching the input customer service question.

[0063] The feature extraction network 310 further includes: a vectorization layer 3101, a feature encoding layer 3102, and a pooling layer 3103. The difference from the first intent recognition model is that the feature encoding layer 3102 includes far more encoding structures than the feature encoding layer 2102. For example, the feature encoding layer 3102 includes twice the encoding structures of the feature encoding layer 2102.

[0064] The second intent recognition model includes the same functions of the feature extraction network 310 and the classification network 320 as the first intent recognition model, and will not be described again here.

[0065] In the second intent recognition model, the feature extraction network 310 is built based on the pre-trained BERT model. Therefore, the second intent recognition model only needs to be fine-tuned based on the preset training sample set.

[0066] The pre-set training sample set is training data constructed based on user inquiries (i.e., customer service questions) and the real values ​​of the corresponding question categories. The user inquiries serve as sample data, and the real values ​​of each pre-set question category level matched to the question serve as sample labels. A supervised training method is used to train the second intent recognition model. For example, by collecting real user inquiries and corresponding response information from the knowledge base, a training sample set is created. Each inquiry question constructs one training sample, and each training sample corresponds to multiple sample labels. Each sample label represents a different question category level to which the response information for that inquiry belongs. For instance, for a user inquiry about redeeming coupons with points, the corresponding question category level in the knowledge base is: Points Question -> Points Redemption -> How to Redeem Coupons.

[0067] Optionally, different problem category levels can be defined based on business needs and problem classification.

[0068] During the training of the second intent recognition model, the vectorization layer 3101 can convert the text sequence corresponding to the question into a vector representation according to the following formula:

[0069] TE = w T X;

[0070]

[0071]

[0072] output = TE + SE + PE;

[0073] Where X represents the current word, pos represents the position of the current word, and d model Indicates the input / output dimensions, where i represents the current word's index, and w... T TE represents the word embedding tensor, SE represents the sentence block tensor, PE represents the position encoding tensor, and output represents the vector representation of the current word.

[0074] Other methods in the prior art can also be used to implement the vectorization layer 3101, which will not be listed here.

[0075] During the training of the second intent recognition model, the feature encoding layer 3102 includes multiple encoding layers based on the attention mechanism. The vector representation of the output sequence is processed by the feature encoding layer 3102 for feature extraction and semantic parsing to obtain the hidden layer vector, which is then compressed into a feature vector x of a specified dimension by the pooling layer 3103.

[0076] Next, the classification task corresponding to each network branch of the second intent recognition model performs classification mapping on the feature vector x and outputs the predicted value y of the question category corresponding to the current training sample data at different levels of question category.

[0077] Then, the model's loss is calculated based on the sample label (i.e., the true value of the question category) of each training sample and the predicted value of the question category output by all network branches.

[0078] Optionally, the model calculates the predicted loss value of each training sample in each question category space based on the predicted value of the question category in the training sample and the true value of the question category set in the sample label. The predicted loss values ​​of all question category spaces are then weighted according to their corresponding weights to obtain the loss value corresponding to that training sample.

[0079] Optionally, the weights of the predicted loss values ​​for different problem category spaces are negatively correlated with the hierarchical affiliation of the corresponding problem category spaces. For example, the coarser the problem category, the smaller the weight; the more refined the problem category, the larger the weight. Specifically, coarse problem categories have a larger classification granularity, while refined problem categories have a smaller classification granularity. Refined problem categories are the subcategories included within the coarse problem categories.

[0080] Optionally, taking a second intent recognition model that includes K network branches as an example, the loss value of the second intent recognition model can be calculated using the following formula:

[0081]

[0082] Where N represents the number of training samples, x i Let W represent the i-th training sample. k f represents the weight of the classification task corresponding to the k-th network branch. k () represents the classification task corresponding to the k-th network branch, f k (x i y represents the predicted problem category value obtained by classifying the i-th training sample for the classification task corresponding to the k-th network branch. ki This represents the true value of the problem category in the problem category space corresponding to the problem category level (e.g., the kth problem level) of the sample label of the i-th training sample at the k-th network branch.

[0083] Optionally, the weight of the classification task corresponding to the k-th network branch is related to the hierarchical membership of the predicted problem category output by the classification task, i.e., it is related to the value of k. Taking the example that a smaller k value results in a coarser prediction of the problem category by the classification task corresponding to the k-th network branch, the weight W corresponding to the classification task of the k-th network branch... k It is positively correlated with k. That is, the coarser the problem category, the smaller the weight of the predicted loss value in that problem category space; the more refined the problem category, the larger the weight of the predicted loss value in that problem category space.

[0084] During training, the network parameters of the second intent recognition model are optimized with the goal of optimizing the loss value until the loss value converges, thus completing the fine-tuning of the second intent recognition model.

[0085] Next, some hidden layers of the second intent recognition model after fine-tuning training are transferred to obtain the first intent recognition model.

[0086] Optionally, the step of transferring some hidden layers of the fine-tuned second intent recognition model to obtain the first intent recognition model includes: constructing a vectorized layer of the first intent recognition model based on the vectorized layer of the fine-tuned second intent recognition model; constructing an encoding layer of the first intent recognition model based on some encoding layers of the fine-tuned second intent recognition model; and constructing a pooling layer and a fully connected layer of the first intent recognition model based on the pooling layer and the fully connected layer of the fine-tuned second intent recognition model, respectively, to obtain the first intent recognition model.

[0087] For example, the vectorization layer 3101, partial feature encoding layer 3102, full pooling layer 3103, and classification network 320 of the feature extraction network 310 of the second intent recognition model can be extracted to form the first intent recognition model. Further, the feature encoding layer 3102 of the second intent recognition model can be extracted layer by layer, or the first few layers of the feature encoding layer 3102 can be obtained as the feature encoding layer 2102 of the first intent recognition model. Taking the feature encoding layer 3102 of the second intent recognition model as including 12 encoding layers as an example, after extracting every two layers, four encoding layers can be obtained, and the feature encoding layer 2102 of the first intent recognition model can be constructed based on these four encoding layers; after extracting every one layer, six encoding layers can be obtained, and the feature encoding layer 2102 of the first intent recognition model can be constructed based on these six encoding layers; or, the first four encoding layers of the feature encoding layer 3102 can be selected as the feature encoding layer 2102 of the first intent recognition model.

[0088] Because the first intent recognition model prunes the feature encoding layer 3102 of the second intent recognition model, the parameters of the feature encoding layer 2102 are greatly reduced, making the prediction speed of the first intent recognition model faster.

[0089] The process involves transferring some hidden layers of the second intent recognition model after fine-tuning training to obtain the first intent recognition model. It also includes initializing the corresponding network parameters of the second intent recognition model according to the network parameter weights of the corresponding network layers of the second intent recognition model.

[0090] Next, the first intent recognition model is trained using a knowledge distillation architecture.

[0091] Optionally, the step of using the fine-tuned second intent recognition model as the teacher network and the first intent recognition model as the student network, and training the first intent recognition model using a knowledge distillation architecture, includes: inputting preset training samples into the teacher network and the student network respectively; performing feature encoding and classification mapping on the preset training samples through the teacher network and the student network to obtain the feature layer vector corresponding to the teacher network and multiple question category prediction values ​​with hierarchical membership, and the feature layer vector corresponding to the student network and multiple question category prediction values ​​with hierarchical membership; and based on the feature layer vector corresponding to the teacher network... The first prediction loss value is calculated based on the vector and the feature layer vector corresponding to the student network; the second prediction loss value is calculated based on the predicted values ​​of the multiple question categories corresponding to the teacher network and the multiple question categories corresponding to the student network; the third prediction loss value is calculated based on the true values ​​of the question categories set in the sample labels of the preset training samples and the predicted values ​​of the multiple question categories corresponding to the student network; the model loss value of the student network is calculated based on the first prediction loss value, the second prediction loss value and the third prediction loss value; the student network is iteratively trained with the goal of optimizing the model loss value to obtain the first intent recognition model.

[0092] During the training of the first intent recognition model, for each training sample, it needs to be simultaneously input into both the second intent recognition model (i.e., the teacher network) and the first intent recognition model (i.e., the student network). The first part of the predicted loss is calculated based on the features of the corresponding network layer in the second intent recognition model (i.e., the teacher network) as labels. The second part of the predicted loss is calculated based on the pre-set sample labels (i.e., the true values ​​of the question categories) corresponding to the training sample and the multiple predicted values ​​of question categories output by the first intent recognition model (i.e., the student network). Then, based on the first and second part of the predicted loss, the loss value of the student network is calculated to update the network parameters and optimize the student network. After multiple iterations of optimization, the optimal student network is obtained and used as the first intent recognition model.

[0093] During the training of the first intent recognition model, the first intent recognition model (i.e., the student network) learns knowledge from the second intent recognition model (i.e., the teacher network) and transfers it. The first intent recognition model tries to retain the knowledge of the second intent recognition model. Therefore, it is hoped that the features of each layer of the first intent recognition model (i.e., the student network) are as close as possible to those of the second intent recognition model (i.e., the teacher network). New loss values ​​are added during the model training process.

[0094] Optionally, the loss function of the first intent recognition model can be expressed by the following formula:

[0095]

[0096]

[0097]

[0098] Loss s =(1-α)C s +αL DS +βL PT ;

[0099] Among them, L PT This represents the first prediction loss value. In the formula for calculating the first prediction loss value: N represents the number of samples, and M represents the number of feature layers. This represents the hidden layer vector values ​​of the feature layer in the student network (i.e., the first intent recognition model). This represents the hidden layer vector values ​​of the feature layer of the teacher network (i.e., the second intent recognition model), where ||*|2 represents the L2 norm. Vector normalization is performed using the L2 norm, L PT This represents the first prediction loss value. From the above formula, we can deduce that the first prediction loss value represents the difference between the features extracted by the student network and the teacher network when extracting features from the training samples.

[0100] L DS This represents the second prediction loss value, which is the loss of each output layer. In the formula for calculating the second prediction loss value: This represents the model parameters learned by the sample in the teacher network (i.e., the second intent recognition model). When reasoning, the probability value P predicted by the teacher network. s (y i =c|x i ;θ s ) represents the student network (i.e., the first intent recognition model) in parameter θ sWhen inferring from the training samples, the predicted probability value is denoted by c, which represents the sample label corresponding to the input sample. As shown in the formula for calculating the second prediction loss, the second prediction loss is obtained by using the question category prediction value output by the teacher network (i.e., the second intent recognition model) as the new label for the training sample, and calculating the difference between this prediction value and the question category prediction value output by the student network (i.e., the first intent recognition model).

[0101] C s This represents the third prediction loss value. The third prediction loss value is calculated based on the difference between the student network's (i.e., the first intent recognition model's) predicted value for the question category of the training samples and the pre-set sample labels (i.e., the true value of the question category). The calculation method for the third prediction loss value is the same as that for the teacher network's (i.e., the second intent recognition model's) loss value calculation method, and will not be repeated here.

[0102] Finally, the model loss value Loss of the first intent recognition model is obtained by weighted summation of the first, second, and third predicted loss values. s .

[0103] Then, optimize the model loss value Loss s To achieve this, the parameters of the student network are updated and iteratively trained to obtain the first intent recognition model.

[0104] Through model training, knowledge distillation, and knowledge transfer, the final first intent recognition model is obtained. Not only does the obtained first intent recognition model run fast, but the inference accuracy of the model can also be guaranteed, thereby achieving fast and accurate customer service question classification.

[0105] The intelligent customer service question classification method disclosed in this application involves inputting a target customer service question into a pre-trained first intent recognition model. The first intent recognition model performs feature encoding and classification mapping on the target customer service question to obtain predicted values ​​for multiple question categories with hierarchical relationships. These predicted values ​​are then compared with the actual question category values ​​set for each pre-defined response message in the automated customer service system to obtain a comparison result. Based on the comparison result, the corresponding response category for the target customer service question is determined, thus completing the classification of the target customer service question. This method employs multi-level intent prediction for customer service questions, predicting the user intent contained within the customer service question from coarse to fine, resulting in more accurate predictions compared to directly predicting a single user intent.

[0106] Furthermore, based on the simultaneous prediction of multiple question categories with hierarchical relationships, different methods can be used to compare the predicted values ​​output by the model with the actual values ​​of the predicted question categories preset for customer service responses, depending on the required prediction accuracy. This allows the prediction results to be more flexibly applied to customer service applications.

[0107] Furthermore, the first intent recognition model is trained through knowledge transfer and knowledge distillation. While ensuring prediction accuracy, the model is compressed to effectively improve the inference speed of the first intent recognition model and reduce device dependence.

[0108] This application also discloses an intelligent customer service question classification device, such as... Figure 4 As shown, the device includes:

[0109] The prediction module 410 is used to input the target customer service question into a pre-trained first intent recognition model, and to perform feature encoding and classification mapping on the target customer service question through the first intent recognition model to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question.

[0110] The prediction result comparison module 420 is used to compare the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each reply message preset for the automatic customer service, and obtain the comparison results;

[0111] The classification result determination module 430 is used to determine the response category matched by the target customer service question based on the comparison result.

[0112] Optionally, based on the comparison results, the response category matching the target customer service question is determined, including:

[0113] In response to the comparison result indicating a successful comparison, the target customer service question is determined to be an automated customer service response question;

[0114] In response to the comparison result indicating a failure, the target customer service question is determined to be a question requiring a human customer service representative.

[0115] Optionally, the prediction module 410 is further configured to:

[0116] The target customer service question is input into a pre-trained first intent recognition model. The first intent recognition model's feature extraction task is used to extract and encode features from the target customer service question to obtain a feature vector.

[0117] The first intent recognition model independently performs classification mapping on the feature vector in the corresponding question category space from coarse to fine levels through multiple classification tasks, thereby obtaining multiple question category prediction values ​​with hierarchical membership for matching the target customer service question, wherein each classification task corresponds to a level of a preset question category.

[0118] Optionally, the prediction result comparison module 420 is further configured to:

[0119] Obtain the actual values ​​of a set of question categories with hierarchical relationships corresponding to each preset reply message for automated customer service;

[0120] According to the hierarchical order of the problem categories from coarse to fine, the predicted values ​​of the multiple problem categories are compared with the actual values ​​of each group of the problem categories in a progressively higher level.

[0121] In response to the fact that the predicted values ​​of the multiple problem categories and any set of the true values ​​of the problem categories meet a preset similarity condition, a comparison result indicating successful comparison is obtained;

[0122] In response to the fact that the predicted values ​​of the multiple problem categories and the actual values ​​of the problem categories do not meet the preset similarity conditions, a comparison result indicating a failure is obtained.

[0123] Optionally, the prediction result comparison module 420 is further configured to:

[0124] Obtain the actual values ​​of a set of question categories with hierarchical relationships corresponding to each preset reply message for automated customer service;

[0125] Based on the hierarchical classification of the question categories, and according to the actual value of the question category corresponding to each reply, a first category sequence corresponding to that reply is generated.

[0126] A second category sequence is generated based on the predicted values ​​of the multiple problem categories, according to the hierarchical classification of the problem categories.

[0127] The second category sequence is sequentially compared with the first category sequence for consistency.

[0128] In response to the second category sequence matching any one of the first category sequences, an alignment result indicating a successful alignment is obtained;

[0129] In response to the inconsistency between the second category sequence and the first category sequence, an alignment result indicating an alignment failure is obtained.

[0130] Optionally, the first intent recognition model is trained using the following method:

[0131] Based on the pre-trained BERT model, a second intent recognition model including multiple classification tasks is constructed.

[0132] Based on a preset training sample set, the second intent recognition model is fine-tuned and trained. Each classification task is used to output the predicted question category value of each training sample. The predicted question category value of each training sample has a hierarchical membership relationship. The loss value of the second intent recognition model is calculated by weighting and summing the predicted loss values ​​of each classification task for the training sample based on the weights corresponding to the classification tasks. The weights are related to the hierarchical membership relationship of the predicted question category value output by the classification tasks.

[0133] The first intent recognition model is obtained by transferring some hidden layers of the second intent recognition model after fine-tuning training.

[0134] The second intent recognition model, after fine-tuning and training, is used as the teacher network, and the first intent recognition model is used as the student network. The first intent recognition model is trained using a knowledge distillation architecture.

[0135] Optionally, the step of using the finely tuned second intent recognition model as the teacher network and the first intent recognition model as the student network, and training the first intent recognition model using a knowledge distillation architecture, includes:

[0136] Preset training samples are input into the teacher network and the student network respectively. The teacher network and the student network perform feature encoding and classification mapping on the preset training samples to obtain the feature layer vector and multiple question category prediction values ​​with hierarchical membership for the teacher network, as well as the feature layer vector and multiple question category prediction values ​​with hierarchical membership for the student network.

[0137] Calculate the first prediction loss value based on the feature layer vector corresponding to the teacher network and the feature layer vector corresponding to the student network;

[0138] Calculate the second prediction loss value based on the predicted values ​​of the multiple question categories corresponding to the teacher network and the multiple question categories corresponding to the student network;

[0139] The third prediction loss value is calculated based on the true value of the question category set in the sample label of the preset training sample and the predicted value of the multiple question categories corresponding to the student network.

[0140] The model loss value of the student network is calculated based on the first predicted loss value, the second predicted loss value, and the third predicted loss value.

[0141] With the goal of optimizing the model loss value, the student network is iteratively trained to obtain the first intent recognition model.

[0142] The intelligent customer service problem classification device disclosed in this application is used to implement the intelligent customer service problem classification method described in this application. The specific implementation methods of each module of the device will not be repeated here, but can be found in the specific implementation methods of the corresponding steps in the method embodiments.

[0143] The intelligent customer service question classification device disclosed in this application involves inputting a target customer service question into a pre-trained first intent recognition model. The first intent recognition model performs feature encoding and classification mapping on the target customer service question to obtain multiple predicted question categories with hierarchical relationships. These predicted question category values ​​are then compared with the actual question category values ​​set for each pre-defined response message in the automated customer service system to obtain a comparison result. Based on the comparison result, the corresponding response category for the target customer service question is determined, thus completing the classification of the target customer service question. This method employs multi-level intent prediction for customer service questions, predicting the user intent contained within the customer service question from coarse to fine, resulting in more accurate predictions compared to directly predicting a single user intent.

[0144] Furthermore, based on the simultaneous prediction of multiple question categories with hierarchical relationships, different methods can be used to compare the predicted values ​​output by the model with the actual values ​​of the predicted question categories preset for customer service responses, depending on the required prediction accuracy. This allows the prediction results to be more flexibly applied to customer service applications.

[0145] Furthermore, the first intent recognition model is trained through knowledge transfer and knowledge distillation. While ensuring prediction accuracy, the model is compressed to effectively improve the inference speed of the first intent recognition model and reduce device dependence.

[0146] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they are fundamentally similar to the method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0147] The above provides a detailed description of the intelligent customer service question classification method and apparatus provided by this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method of this application and its core idea. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0148] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0149] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0150] For example, Figure 5An electronic device is shown that can implement the methods according to this application. The electronic device may be a PC, mobile terminal, personal digital assistant, tablet computer, etc. The electronic device conventionally includes a processor 510 and a memory 520, and program code 530 stored in the memory 520 and executable on the processor 510, which, when executing the program code 530, implements the methods described in the above embodiments. The memory 520 may be a computer program product or a computer-readable medium. The memory 520 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 520 has a storage space 5201 for the program code 530 of a computer program for performing any of the method steps described above. For example, the storage space 5201 for the program code 530 may include various computer programs for implementing the various steps in the above methods. The program code 530 is computer-readable code. These computer programs can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The computer program includes computer-readable code that, when executed on an electronic device, causes the electronic device to perform the method according to the above embodiments.

[0151] This application also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the intelligent customer service problem classification method as described in this application.

[0152] Such a computer program product can be a computer-readable storage medium, which can have the same characteristics as... Figure 5 The memory 520 in the illustrated electronic device is similarly arranged as storage segments, storage spaces, etc. Program code can be stored, for example, in a compressed form on the computer-readable storage medium. The computer-readable storage medium is typically as shown in the reference... Figure 6 The portable or fixed storage unit is described above. Typically, the storage unit includes computer-readable code 530', which is code read by a processor and, when executed by the processor, implements the various steps in the method described above.

[0153] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0154] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0155] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for classifying intelligent customer service questions, characterized in that, The method includes: The target customer service question is input into a pre-trained first intent recognition model. The first intent recognition model performs feature encoding and classification mapping on the target customer service question to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question. The predicted values ​​of the multiple question categories are compared with the actual values ​​of the question categories set for each reply message preset for the automated customer service, and the comparison results are obtained. Based on the comparison results, determine the response category that matches the target customer service question; The first intent recognition model was trained using the following method: Based on the pre-trained BERT model, a second intent recognition model including multiple classification tasks is constructed. The second intent recognition model is fine-tuned based on a preset training sample set. The first intent recognition model is obtained by transferring some hidden layers of the second intent recognition model after fine-tuning training. The second intent recognition model, after fine-tuning and training, is used as the teacher network, and the first intent recognition model is used as the student network. The first intent recognition model is trained using a knowledge distillation architecture.

2. The method according to claim 1, characterized in that, The process involves inputting the target customer service question into a pre-trained first intent recognition model, and then using the first intent recognition model to perform feature encoding and classification mapping on the target customer service question to obtain predicted values ​​for multiple question categories with hierarchical relationships matching the target customer service question. This includes: The target customer service question is input into a pre-trained first intent recognition model. The first intent recognition model's feature extraction task is used to extract and encode features from the target customer service question to obtain a feature vector. The first intent recognition model independently performs classification mapping on the feature vector in the corresponding question category space from coarse to fine levels through multiple classification tasks, thereby obtaining multiple question category prediction values ​​with hierarchical membership for matching the target customer service question, wherein each classification task corresponds to a level of a preset question category.

3. The method according to claim 1, characterized in that, The step of comparing the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each pre-set reply message of the automated customer service, and obtaining the comparison results, includes: Obtain the actual values ​​of a set of question categories with hierarchical relationships corresponding to each preset reply message for automated customer service; According to the hierarchical order of the problem categories from coarse to fine, the predicted values ​​of the multiple problem categories are compared with the actual values ​​of each group of the problem categories in a progressively higher level. In response to the fact that the predicted values ​​of the multiple problem categories and any set of the true values ​​of the problem categories meet a preset similarity condition, a comparison result indicating successful comparison is obtained; In response to the fact that the predicted values ​​of the multiple problem categories and the actual values ​​of the problem categories do not meet the preset similarity conditions, a comparison result indicating a failure is obtained.

4. The method according to claim 1, characterized in that, The step of comparing the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each pre-set reply message of the automated customer service, and obtaining the comparison results, includes: Obtain the actual values ​​of a set of question categories with hierarchical relationships corresponding to each preset reply message for automated customer service; Based on the hierarchical classification of the question categories, and according to the actual value of the question category corresponding to each reply, a first category sequence corresponding to that reply is generated. A second category sequence is generated based on the predicted values ​​of the multiple problem categories, according to the hierarchical classification of the problem categories. The second category sequence is sequentially compared with the first category sequence for consistency. In response to the second category sequence matching any one of the first category sequences, an alignment result indicating a successful alignment is obtained; In response to the inconsistency between the second category sequence and the first category sequence, an alignment result indicating an alignment failure is obtained.

5. The method according to claim 1, characterized in that, The step of fine-tuning the second intent recognition model based on a preset training sample set includes: Each of the classification tasks is used to output the predicted question category value for each training sample. The predicted question category value of each training sample has a hierarchical membership relationship. The loss value of the second intent recognition model is calculated by weighting and summing the predicted loss value of each classification task for the training sample based on the weights corresponding to the classification tasks. The weights are related to the hierarchical membership relationship of the predicted question category value output by the classification tasks.

6. The method according to claim 1, characterized in that, The step of using the finely tuned second intent recognition model as the teacher network and the first intent recognition model as the student network, and training the first intent recognition model using a knowledge distillation architecture, includes: Preset training samples are input into the teacher network and the student network respectively. The teacher network and the student network perform feature encoding and classification mapping on the preset training samples to obtain the feature layer vector and multiple question category prediction values ​​with hierarchical membership for the teacher network, as well as the feature layer vector and multiple question category prediction values ​​with hierarchical membership for the student network. Calculate the first prediction loss value based on the feature layer vector corresponding to the teacher network and the feature layer vector corresponding to the student network; Calculate the second prediction loss value based on the predicted values ​​of the multiple question categories corresponding to the teacher network and the multiple question categories corresponding to the student network; The third prediction loss value is calculated based on the true value of the question category set in the sample label of the preset training sample and the predicted value of the multiple question categories corresponding to the student network. The model loss value of the student network is calculated based on the first predicted loss value, the second predicted loss value, and the third predicted loss value. With the goal of optimizing the model loss value, the student network is iteratively trained to obtain the first intent recognition model.

7. An intelligent customer service question classification device, characterized in that, The device includes: The prediction module is used to input the target customer service question into a pre-trained first intent recognition model, and to perform feature encoding and classification mapping on the target customer service question through the first intent recognition model to obtain multiple question category prediction values ​​with hierarchical membership matching the target customer service question; The prediction result comparison module is used to compare the predicted values ​​of the multiple question categories with the actual values ​​of the question categories set for each reply message preset for the automatic customer service, and obtain the comparison results; The classification result determination module is used to determine the response category matched to the target customer service question based on the comparison results; The first intent recognition model was trained using the following method: Based on the pre-trained BERT model, a second intent recognition model including multiple classification tasks is constructed. The second intent recognition model is fine-tuned based on a preset training sample set. The first intent recognition model is obtained by transferring some hidden layers of the second intent recognition model after fine-tuning training. The second intent recognition model, after fine-tuning and training, is used as the teacher network, and the first intent recognition model is used as the student network. The first intent recognition model is trained using a knowledge distillation architecture.

8. The apparatus according to claim 7, characterized in that, The prediction module is further used for: The target customer service question is input into a pre-trained first intent recognition model. The first intent recognition model's feature extraction task is used to extract and encode features from the target customer service question to obtain a feature vector. The first intent recognition model independently performs classification mapping on the feature vector in the corresponding question category space from coarse to fine levels through multiple classification tasks, thereby obtaining multiple question category prediction values ​​with hierarchical membership for matching the target customer service question, wherein each classification task corresponds to a level of a preset question category.

9. An electronic device, comprising a memory, a processor, and program code stored in the memory and executable on the processor, characterized in that, When the processor executes the program code, it implements the intelligent customer service question classification method according to any one of claims 1 to 6.

10. A computer-readable storage medium having program code stored thereon, characterized in that, When the program code is executed by the processor, it implements the steps of the intelligent customer service problem classification method according to any one of claims 1 to 6.

Images