Comprehensive coordination work order service perception intelligent quality inspection method and system based on multi-round dialogue

Abstract

The present invention discloses a comprehensive coordination work order service perception intelligent quality inspection method and system based on multi-round dialogue, which belongs to the field of natural language processing technology, including: S1: data processing of work order information; S2: semantic representation of work order text based on RoBERTa; S3: construction of Fasttext classification model; S4: model training. The present invention uses a context-dependent RoBERTa pre-training model to perform text semantic representation based on the text information of the customer service and user conversation record in the work order, and combines service perception quality inspection requirement labels such as whether the communication method is compliant, whether taboo terms appear, whether the service is thoughtful, whether it is enthusiastic, etc., to input the RoBERTa text semantic representation into the fast classification Fasttext model, so as to enhance the semantic knowledge of the work order text, thereby training the work order information perception evaluation model, and at the same time classify and judge whether the supporting service is standardized and whether the business capability meets the standard, so as to realize the automatic detection of the perceived quality of the comprehensive coordination service, solve the problem of low coverage of manual sampling, and improve the perceived quality of customer service.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for intelligent quality inspection of integrated debugging work order services based on multi-round dialogues. Background Art

[0002] In the operator field, there are many kinds of knowledge in the front-line support field, such as intelligent classification of work orders, intelligent quality inspection of work orders, intelligent interception of problems, intelligent automatic typesetting, etc. In the field of intelligent quality inspection support for work orders, the quality inspection rules are complex, the workload of manual quality inspection is large, the efficiency is low, and the accumulation and transfer of quality inspection experience are relatively difficult. To this end, a comprehensive adjustment work order service perception intelligent quality inspection method and system based on multi-round dialogue is proposed. Summary of the invention

[0003] The technical problem to be solved by the present invention is: how to solve the problems existing in the prior art, such as complex quality inspection rules, large workload of manual quality inspection content, low efficiency, and relatively difficult accumulation and transmission of quality inspection experience, and provide a comprehensive adjustment work order service perception intelligent quality inspection method based on multi-round dialogue. According to the text information of the customer service and user conversation record in the work order, the text semantic representation is performed through the context-dependent RoBERTa pre-training model, and the service perception quality inspection requirement labels such as whether the communication method is compliant, whether taboo terms appear, whether the service is thoughtful, and whether it is enthusiastic are combined. The RoBERTa text semantic representation is input into the fast classification Fasttext model to achieve work order text semantic knowledge enhancement, thereby training the work order information perception evaluation model, and at the same time classify and judge whether the supporting service (quality inspection) is standardized and whether the business capabilities meet the standards. Automatic detection of the perceived quality of the comprehensive adjustment service is achieved, solving the problem of low coverage of manual sampling, and improving the perceived quality of customer service.

[0004] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0005] S1: Data processing of work order information

[0006] Extract the conversation content and work order number from the work order information table, as well as the support service standardization label, business capability compliance label and work order number from the comprehensive adjustment work order information content table, and associate and match the information in the two tables according to the work order number to build a work order information data set; perform data cleaning on the work order information data set; then remove stop words, perform word segmentation, text balancing and deduplication operations on the work order information conversation text;

[0007] S2: Semantic representation of work order text based on RoBERTa

[0008] Based on the RoBERTa pre-trained model, the context-related semantic representation of the words in the work order information dialogue text content processed by step S1 is performed to obtain the semantic representation vector of each word in the work order information dialogue text;

[0009] S3: Fasttext classification model construction

[0010] Use the Fasttext algorithm to build the model and obtain the Fasttext classification model;

[0011] S4: Model training

[0012] During training, the work order information data is trained based on the labels of whether the supporting services are standardized and whether the business capabilities meet the standards. The work order information is semantically represented by using the RoBERTa pre-trained model and then connected to the Fasttext classification model for model training. The accuracy of whether the supporting services are standardized and whether the business capabilities meet the standards is calculated respectively, to achieve automatic detection of the perceived quality of the comprehensive coordination service.

[0013] Furthermore, in the step S1, the work order number and work order information dialogue text of each work order information in the work order information data set, as well as the support service standardization label, business capability compliance label and work order number in the comprehensive adjustment work order information content table are associated through the work order number.

[0014] Furthermore, in the step S1, the work order information data set is cleaned using regular expressions, and the work order information data set is divided into a test set and a training set in a ratio of 3:7.

[0015] Furthermore, in the step S1, a work order quality inspection stop word list is constructed according to the work order information dialogue text, the jieba word segmentation tool is used to perform word segmentation and remove stop words, and then data balancing and deduplication operations are performed.

[0016] Furthermore, in step S3, the Fasttext classification model includes a hidden layer, an output layer and a softmax layer connected in sequence; wherein each word in the work order information dialogue text is input into the hidden layer based on the semantic representation vector of RoBERTa, and a nonlinear transformation is performed on the hidden layer to obtain the label of the output layer, and the softmax layer is used to calculate the probability of the work order information belonging to the quality inspection yes / no specification and the business capability yes / no compliance category using the softmax function.

[0017] The present invention also provides an intelligent quality inspection system for integrated debugging work order service perception based on multi-round dialogue, which uses the above-mentioned quality inspection method to detect the perceived quality of integrated debugging work order service, including:

[0018] The data processing module is used to extract the conversation content and work order number in the work order information table, as well as the support service standardization label, business capability compliance label and work order number in the comprehensive adjustment work order information content table, and associate and match the information in the two tables according to the work order number to build a work order information data set; and perform data cleaning on the work order information data set; and then remove stop words, segment words, balance text and remove duplicates from the work order information conversation text;

[0019] A semantic representation module is used to perform semantic representation of the words in the work order information dialogue text content processed by step S1 based on the RoBERTa pre-trained model, and obtain a semantic representation vector of each word in the work order information dialogue text;

[0020] A model building module is used to build a model using the Fasttext algorithm to obtain a Fasttext classification model;

[0021] The model training module is used to semantically represent the work order information using the RoBERTa pre-training model and then connect it to the Fasttext classification model for model training. During training, the work order information data is trained based on the labels of whether the supporting service is standardized and whether the business capability meets the standards, and the probability of whether the supporting service is standardized and whether the business capability meets the standards is calculated to achieve automatic detection of the perceived quality of the comprehensive coordination service;

[0022] The control processing module is used to issue instructions to other modules to complete related actions;

[0023] The data processing module, semantic representation module, model training module and model construction module are all communicatively connected with the control processing module.

[0024] Compared with the prior art, the present invention has the following advantages: the intelligent quality inspection method for service perception of comprehensive coordination work orders based on multi-round dialogues, for front-line support events in the operator field, performs text semantic representation based on the text information of the customer service and user conversation records in the work order through the context-dependent RoBERTa pre-training model, and inputs the RoBERTa text semantic representation into the fast classification Fasttext model in combination with service perception quality inspection requirement labels such as whether the communication method is compliant, whether taboo terms appear, whether the service is thoughtful, whether the service is enthusiastic, etc., to achieve work order text semantic knowledge enhancement, thereby training the work order information perception evaluation model, and at the same time classifying and judging whether the supporting service is standardized and whether the business capability meets the standards, to achieve automatic detection of the perceived quality of comprehensive coordination services, solve the problem of low coverage of manual sampling, and improve the perceived quality of customer service, which is worthy of promotion and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1It is a schematic flowchart of the intelligent quality inspection method for integrated dispatching work order service perception based on multi-round dialogue in the embodiments of the present invention;

[0026] Figure 2 It is a schematic diagram of the Fasttext classification model structure in the embodiments of the present invention. Specific embodiments

[0027] The following is a detailed description of the embodiments of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.

[0028] Embodiment 1

[0029] As Figure 1 shown, this embodiment provides a technical solution: an intelligent quality inspection method for integrated dispatching work order service perception based on multi-round dialogue, including main steps such as data processing of work order information, semantic representation of work order text based on RoBERTa, and construction of an enhanced knowledge Fasttext classification model, as follows:

[0030] 1.1 Data processing of work order information

[0031] First, manually export the conversation content and work order numbers in the work order information table from the system background data, as well as the labels of whether the support service is standardized, whether the business ability meets the standard, and the work order numbers in the integrated dispatching work order information content table. The labels of whether the support service is planned and whether the business ability meets the standard are marked through manual quality inspection. Then, the information in the two tables is associated and matched according to the work order number to construct a work order information dataset, including work order information conversation text, labels of whether the support service is standardized, and labels of whether the business ability meets the standard. Then, the present invention uses regular expressions to clean the associated and matched work order information, and divides the work order information dataset into a test set and a training set according to 3:7. The obtained data form is as follows: "Installation and maintenance personnel: #Installation and maintenance personnel: Your problem has been processed. Can it be completed? If so, please confirm. If not, please choose to continue asking questions. #Installation and maintenance personnel: Your problem work order integrated dispatching has been completed. If you are satisfied, please give a good review. Thank you! #Customer: The problem has been solved. Please complete it.", which contains many stop words such as "of", "right", etc. According to the work order information conversation text, a work order quality inspection stop word table is constructed, the jieba word segmentation tool is used for word segmentation, and the stop word table is loaded to remove stop words. The cleaned data form is as follows: "Installation and maintenance personnel problem has been processed whether can be completed if can please confirm if not please choose continue asking questions installation and maintenance personnel problem work order integrated dispatching has been completed if satisfied please give good review thank you customer problem has been solved please complete", and then data balancing and duplicate removal operations are performed on the cleaned dataset.

[0032] 1.2 Semantic representation of work order text based on RoBERTa

[0033] The text content in the processed work order information is semantically represented based on the RoBERTa pre-training model. The RoBERTa pre-training model is an improved version based on the Bert pre-training model. It mainly adjusts the learning rate and uses larger data for model training, increasing from 16GB to 160GB. A larger batch is used in the model for experiments, and a dynamic mask is used to replace the static mask, which speeds up the training of the model and improves the performance of the model. Specifically, the present invention is based on the fine-tuning of the RoBERTa pre-training model, and uses the RoBERTa pre-training model to semantically represent the vocabulary in the text content of the work order information dialogue in the context of the context.

[0034] 1.3 Fasttext classification model construction with enhanced knowledge

[0035] The model architecture of Fasttext is relatively simple, which is a hidden layer and an output layer of word embedding. Its network structure is as follows Figure 2 shown.

[0036] Among them, W(1) to W(n) are the semantic representation vectors of each word in the work order information based on RoBERTa. Compared with the traditional Fasttext model that directly uses randomly initialized words to embed semantics, the present invention uses the semantic representation of the work order information dialogue text based on RoBERTa as the input of the model, which can enhance the text semantic knowledge. Therefore, the content represented by the work order information in the present invention is the mean representation of the accumulated semantic representation vectors of all words based on RoBERTa, that is:

[0037]

[0038] Then, a nonlinear transformation is performed on the hidden layer to obtain the label z of the output layer:

[0039] z=sigmoid(W o h) (2)

[0040] Among them, W o represents the weight from the hidden layer to the output layer.

[0041] Since the work order quality inspection task includes whether the quality inspection is standardized and whether the business capability is up to standard, in the final model training stage, the work order information data is trained based on the support service standard label and the business capability standard label. After the Fasttext classification model with enhanced knowledge, the present invention adds a layer of softmax function to calculate the probability y of which category the work order information belongs to:

[0042] y=softmax(z) (3)

[0043] Finally, the work order information is semantically represented using the RoBERTa pre-trained model and then connected to the Fasttext classification model for model training. The probability of whether the supporting service is standardized and whether the business capability meets the standard is calculated to achieve automatic detection of the perceived quality of the comprehensive coordination service.

[0044] In this embodiment, by comparing with the existing Fasttext model, the RoBERTa pre-trained model is used to fuse the full-text semantic information with the work order information to achieve the semantic representation of the work order text with enhanced knowledge, so as to compare the text classification performance on the operator field events, and use the classification accuracy as the evaluation indicator:

[0045]

[0046] Among them, TP represents the number of work order information whose true label and predicted label are both correct; FN represents the number of work order information whose true label is correct and predicted label is incorrect; TN represents the number of work order information whose true label and predicted label are incorrect; FP represents the number of work order information whose true label is incorrect and predicted label is correct.

[0047] In this embodiment, the text representation of this method has integrated full-text semantic information, and then accesses the Fasttext model with fast training speed and good accuracy. The accuracy rates of the two quality inspections of whether the supporting services are standardized and whether the business capabilities meet the standards are 0.9363 and 0.9215 respectively. Compared with the traditional Fasttext text classification algorithm, the accuracy of the present invention in these two quality inspection tasks is improved by 1.87% and 1.58% respectively.

[0048] To sum up, the above-mentioned embodiment of the intelligent quality inspection method for comprehensive coordination work order service perception based on multi-round dialogue, for front-line support events in the operator field, performs text semantic representation through the context-dependent RoBERTa pre-training model based on the text information of the customer service and user conversation records in the work order, and combines service perception quality inspection requirement labels such as whether the communication method is compliant, whether taboo terms appear, whether the service is thoughtful, whether it is enthusiastic, etc., and inputs the RoBERTa text semantic representation into the fast classification Fasttext model to enhance the semantic knowledge of the work order text, thereby training the work order information perception evaluation model, and at the same time classifying and judging whether the support service (quality inspection) is standardized and whether the business capability meets the standards, to realize automatic detection of the perceived quality of comprehensive coordination services, solve the problem of low coverage of manual sampling, and improve the perceived quality of customer service, which is worthy of promotion and use.

[0049] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. The process of the intelligent quality inspection method of comprehensive debugging work order service perception based on multi-round dialogue is as follows: include: S1: Data processing of work order information Extract the conversation content and work order number from the work order information table, as well as the support service standardization label, business capability compliance label and work order number from the comprehensive adjustment work order information content table, and associate and match the information in the two tables according to the work order number to build a work order information data set; perform data cleaning on the work order information data set; then remove stop words, perform word segmentation, text balancing and deduplication operations on the work order information conversation text; S2: Semantic representation of work order text based on RoBERTa Based on the RoBERTa pre-trained model, the context-related semantic representation of the words in the work order information dialogue text content processed by step S1 is performed to obtain the semantic representation vector of each word in the work order information dialogue text; S3: Fasttext classification model construction Use the Fasttext algorithm to build the model and obtain the Fasttext classification model; S4: Model training Based on the work order information data with labels of whether the supporting service is standardized and whether the business capability meets the standards, the work order information is semantically represented by using the RoBERTa pre-trained model and then connected to the Fasttext classification model for model training. The accuracy of whether the supporting service is standardized and whether the business capability meets the standards is calculated respectively, realizing automatic detection of the perceived quality of the comprehensive coordination service; In step S1, the work order number and the work order information dialogue text of each work order information in the work order information data set, and the work order number and the support service standard label and the business capability compliance label in the work order information label data set are matched and associated through the work order number; In the step S1, a work order quality inspection stop word list is constructed according to the work order information dialogue text, the jieba word segmentation tool is used to perform word segmentation and remove stop words, and then data balancing and deduplication operations are performed; In step S3, the Fasttext classification model includes a hidden layer, an output layer and a softmax layer connected in sequence; wherein each word in the work order information dialogue text is input into the hidden layer based on the semantic representation vector of RoBERTa, and a nonlinear transformation is performed once in the hidden layer to obtain the label of the output layer, and the softmax layer is used to calculate the probability of the work order information belonging to the quality inspection yes / no specification and the business capability yes / no compliance category using the softmax function; The intelligent quality inspection method detects the perceived quality of the integrated debugging work order service based on the following integrated debugging work order service perception intelligent quality inspection system based on multi-round dialogue, and the quality inspection system includes: The data processing module is used to extract the conversation content and work order number in the work order information table, as well as the support service standardization label, business capability compliance label and work order number in the comprehensive adjustment work order information content table, and associate and match the information in the two tables according to the work order number to build a work order information data set; and perform data cleaning on the work order information data set; and then remove stop words, segment words, balance text and remove duplicates from the work order information conversation text; A semantic representation module is used to perform semantic representation of the words in the work order information dialogue text content processed by step S1 based on the RoBERTa pre-trained model, and obtain a semantic representation vector of each word in the work order information dialogue text; A model building module is used to build a model using the Fasttext algorithm to obtain a Fasttext classification model; The model training module is used to semantically represent the work order information using the RoBERTa pre-training model and then connect it to the Fasttext classification model for model training. During training, the work order information data is trained based on the labels of whether the supporting service is standardized and whether the business capability meets the standards, and the probability of whether the supporting service is standardized and whether the business capability meets the standards is calculated to achieve automatic detection of the perceived quality of the comprehensive coordination service; The control processing module is used to issue instructions to other modules to complete related actions; The data processing module, semantic representation module, model training module and model construction module are all communicatively connected with the control processing module.

2. The method for intelligent quality inspection of integrated debugging work order service perception based on multi-round dialogue according to claim 1 is characterized by: In the step S1, a regular expression is used to clean the work order information data set, and the work order information data set is divided into a test set and a training set in a ratio of 3:7.

Images