Quality inspection method, device, equipment and storage medium for customer service recordings

Abstract

The present invention discloses a method, device, equipment and storage medium for quality inspection of customer service recordings. The method comprises: obtaining a customer service recording text, mining segments of the customer service recording text; calculating the scores of the segments, and screening out candidate segments based on the scores; inputting the candidate segments and their scores into a quality inspection model, and determining the quality inspection results of the customer service recordings according to the prediction results output by the quality inspection model. Thus, segments are mined from the customer service recordings, and only the screened candidate segments are quality inspected, which reduces the workload of quality inspection of customer service recordings and improves quality inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a quality inspection method, device, equipment and storage medium for customer service recordings. Background Art

[0002] With the development of computer technology, more and more technologies (big data, distributed, blockchain, artificial intelligence, etc.) are being applied in the financial field. The traditional financial industry is gradually transforming to financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] At present, for quality inspection recordings including customer service and customers, if the input method is single-channel, it is impossible to physically distinguish the speaking roles, and then all recordings need to be quality inspected, or the roles of the quality inspection recording texts need to be labeled first, and then selective quality inspection is performed based on the role labeling results. Therefore, the workload of quality inspection of customer service recordings is large, time-consuming and labor-intensive. Summary of the invention

[0004] The present invention provides a customer service recording quality inspection method, device, equipment and storage medium, aiming to reduce the workload of customer service recording quality inspection and improve quality inspection efficiency.

[0005] To achieve the above object, the present invention provides a method for quality inspection of customer service recordings, the method comprising:

[0006] Obtaining customer service recording text, and mining fragments of the customer service recording text;

[0007] Calculating scores of the fragments, and screening out candidate fragments based on the scores;

[0008] The candidate segments and their scores are input into a quality inspection model, and the quality inspection result of the customer service recording is determined according to the prediction result output by the quality inspection model.

[0009] Preferably, the step of calculating the scores of the fragments and screening out candidate fragments based on the scores comprises:

[0010] Obtaining the similarity between the segment and the quality inspection item, and calculating the score of the segment belonging to each quality inspection item based on the similarity;

[0011] If the score is greater than or equal to the score threshold, the corresponding segment is screened as a candidate segment.

[0012] Preferably, before the step of inputting the candidate segments and their scores into a quality inspection model and determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model, the step further includes:

[0013] The quality inspection model is obtained by training based on the quality inspection example sentences and their corresponding quality inspection labels;

[0014] Determining whether the quality inspection model converges according to the loss function;

[0015] If the initial model is in a convergence state, the training is stopped to obtain the quality inspection model.

[0016] Preferably, the step of mining the fragments of the customer service recording text includes:

[0017] Obtain keywords of the customer service recording text, and obtain a keyword set based on the shutdown words;

[0018] The segment is determined according to a word graph and the keyword set, wherein the word graph is constructed according to the standard example sentence of the quality inspection item.

[0019] Preferably, the step of obtaining keywords of the customer service recording text and obtaining a keyword set based on the keywords includes:

[0020] Perform word segmentation and part-of-speech tagging on the customer service recording text, and filter based on the part-of-speech tagging results to obtain candidate keywords;

[0021] Selecting one candidate keyword from the candidate keywords in turn, and constructing a keyword graph of each candidate keyword, wherein the keyword graph includes four edges respectively formed by the selected candidate keyword and four candidate keywords following the word;

[0022] According to a preset formula, iteratively propagate the weights of each node of the keyword graph until convergence;

[0023] The candidate keywords are sorted according to the weights in the iteration result, the candidate keywords are selected according to the sorting result, and the candidate keywords and their corresponding weights are saved as the keyword set.

[0024] Preferably, the step of determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model includes:

[0025] Screening the prediction results to obtain target prediction labels whose probabilities are greater than or equal to a probability threshold;

[0026] Compare the target prediction label with the quality inspection item to determine whether the customer service recording meets the requirements;

[0027] If the target prediction tag includes all of the quality inspection items, it is determined that the customer service recording meets the requirements and the quality inspection result is determined to be qualified;

[0028] If the target prediction label does not include all of the quality inspection items, it is determined that the customer service recording text does not meet the requirements and the quality inspection result is determined to be unqualified.

[0029] Preferably, before the step of obtaining the customer service recording text and mining the fragments of the customer service recording text, the step further includes:

[0030] One or more quality inspection items are set, and several quality inspection item standard example sentences are set for the quality inspection items.

[0031] To achieve the above object, the present invention further provides a quality inspection device for customer service recordings, the quality inspection device for customer service recordings comprising:

[0032] A mining module, used to obtain customer service recording text and mine segments of the customer service recording text;

[0033] A screening module, used to calculate the scores of the fragments and screen out candidate fragments based on the scores;

[0034] The quality inspection module is used to input the candidate segments and their scores into the quality inspection model, and determine the quality inspection results of the customer service recording according to the prediction results output by the quality inspection model.

[0035] To achieve the above-mentioned purpose, the present invention also provides a quality inspection device for customer service recordings, wherein the quality inspection device for customer service recordings includes a processor, a memory, and a quality inspection program for customer service recordings stored in the memory. When the quality inspection program for customer service recordings is executed by the processor, the steps of the quality inspection method for customer service recordings as described above are implemented.

[0036] To achieve the above objectives, the present invention also provides a computer storage medium, on which a quality inspection program for customer service recordings is stored. When the quality inspection program for customer service recordings is executed by a processor, the steps of the quality inspection method for customer service recordings as described above are implemented.

[0037] Compared with the prior art, the present invention provides a method, device, equipment and storage medium for quality inspection of customer service recordings, which obtains the text of customer service recordings, mines segments of the customer service recording text; calculates the scores of the segments, and screens out candidate segments based on the scores; inputs the candidate segments and their scores into a quality inspection model, and determines the quality inspection results of the customer service recordings based on the prediction results output by the quality inspection model. Thus, segments are mined from the customer service recordings, and only the screened candidate segments are quality inspected, which reduces the workload of quality inspection of customer service recordings and improves quality inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the hardware structure of the quality inspection equipment for customer service recording involved in each embodiment of the present invention;

[0039] Figure 2 It is a flow chart of a first embodiment of a quality inspection method for customer service recordings of the present invention;

[0040] Figure 3 It is a flow chart of a second embodiment of the quality inspection method for customer service recordings of the present invention;

[0041] Figure 4 It is a flow chart of a second embodiment of the quality inspection method for customer service recordings of the present invention;

[0042] Figure 5 It is a functional module diagram of the first embodiment of the quality inspection device for customer service recording of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0045] The quality inspection device for customer service recordings mainly involved in the embodiments of the present invention refers to a network connection device that can achieve network connection. The quality inspection device for customer service recordings can be a server, a cloud platform, etc.

[0046] Reference Figure 1 , Figure 1 It is a schematic diagram of the hardware structure of the quality inspection equipment for customer service recordings involved in various embodiments of the present invention. In an embodiment of the present invention, the quality inspection equipment for customer service recordings may include a processor 1001 (such as a central processing unit, CPU), a communication bus 1002, an input port 1003, an output port 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the input port 1003 is used for data input; the output port 1004 is used for data output, and the memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0047] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a readable storage medium may include an operating system, a network communication module, an application module, and a quality inspection program for customer service recordings. Figure 1In the embodiment, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the quality inspection program of the customer service recording stored in the memory 1005, and execute the quality inspection method of the customer service recording provided in the embodiment of the present invention.

[0048] An embodiment of the present invention provides a method for quality inspection of customer service recordings.

[0049] Reference Figure 2 , Figure 2 It is a flow chart of the first embodiment of the quality inspection method for customer service recordings of the present invention.

[0050] In this embodiment, the customer service recording quality inspection method is applied to the customer service recording quality inspection equipment, and the method includes:

[0051] Step S101, obtaining a customer service recording text, and mining a segment of the customer service recording text;

[0052] In this embodiment, the customer service recording text to be inspected can be obtained through a preset text input interface. The customer service recording text can be without role tags, and the customer service recording text can also be without sentence divisions. The format of the customer service recording text can be txt, doc, xls, pdf, etc.

[0053] In this embodiment, the textrank algorithm is used to mine keywords. The textrank algorithm is an algorithm for mining text keywords, which can be implemented based on a keyword graph. Generally speaking, if a word appears after many words, it means that the word is more important; if a word follows a word with a high textrank value, the textrank value of this word will also be relatively high.

[0054] The step of mining the fragments of the customer service recording text comprises:

[0055] Step S101-1, obtaining keywords of the customer service recording text, and obtaining a keyword set based on the keywords;

[0056] The customer service recording text is scanned, and natural language technology is used to mine keywords of the customer service recording text to obtain one or more keywords.

[0057] Specifically, the step of obtaining keywords of the customer service recording text and obtaining a keyword set based on the keywords includes:

[0058] Step a, segmenting and tagging the customer service recording text, and filtering based on the part-of-speech tagging results to obtain candidate keywords;

[0059] In this embodiment, word segmentation mainly refers to Chinese word segmentation, which refers to the process of cutting a Chinese character sequence into individual words, and recombining a continuous character sequence into a word sequence according to certain specifications. Generally, there are word segmentation methods based on string matching, word segmentation methods based on understanding, and word segmentation methods based on statistics. Among them, the word segmentation method based on string matching includes forward maximum matching, reverse maximum matching, bidirectional maximum matching, N-gram (Chinese language model) bidirectional maximum matching, etc. Among them, the forward maximum match refers to matching the maximum number of consecutive characters in the customer service recording text with the vocabulary from left to right. If a match is achieved, a word is segmented out; the reverse maximum match refers to matching the maximum number of consecutive characters in the customer service recording text with the vocabulary from right to left. If a match is achieved, a word is segmented out; the bidirectional maximum match includes a forward maximum match algorithm and a reverse maximum match algorithm. If the two algorithms obtain the same word segmentation results, the segmentation is determined to be successful, otherwise, it is determined that an ambiguous phenomenon or an unregistered word problem occurs; the N-gram bidirectional maximum match is based on the forward maximum match algorithm and the reverse maximum match algorithm in the word segmentation method of the string, and then the different parts of the sequence results obtained by matching in the two directions are calculated using Bi-gram to obtain the parts with a higher probability, and finally spliced ​​to obtain the best word sequence.

[0060] After the customer service recording text is segmented, part-of-speech tagging can be performed based on a dictionary search algorithm based on string matching. Specifically, the part-of-speech of each word is searched from the dictionary, and corresponding tagging is performed according to the found part-of-speech.

[0061] After the parts of speech are marked, the customer service recording text is filtered based on the part of speech tagging results, for example, words with specified parts of speech are filtered out; words appearing in a stop word list, which is preset; words with a length less than a preset value can also be filtered out based on the length of the word, for example, the preset value can be 3. The remaining words after filtering are marked as candidate keywords.

[0062] Step b, selecting a candidate keyword from the candidate keywords in turn, and constructing a keyword graph of each candidate keyword, wherein the keyword graph includes four edges respectively formed by the selected candidate keyword and four candidate keywords following the word;

[0063] Taking each of the candidate keywords as the center, a keyword graph of each of the candidate keywords is constructed.

[0064] Specifically, firstly, a candidate keyword is selected, the candidate keyword is represented by A, and the candidate keyword A is marked as a target candidate keyword;

[0065] Taking the target candidate keyword as the center, obtain four candidate keywords following the target candidate keyword, assuming that the four candidate keywords following the candidate keyword A are B, C, D, and E respectively;

[0066] The candidate keyword and four candidate keywords following the candidate keyword respectively form four edges of the keyword graph, and the keyword graph includes four edges composed of keywords and their corresponding weights.

[0067] The keyword graph is composed of four edges consisting of the selected candidate keyword and the four candidate keywords following it, which are (A, B), (A, C), (A, D), and (A, E). The initial weight of each edge is 1. When this edge appears again or multiple times, it is added by 1 each time it appears based on the initial weight, and the sum is finally used as the weight.

[0068] Each time a candidate keyword is selected, a keyword graph for the candidate keyword is constructed. Candidate keywords are selected in sequence until corresponding keyword graphs are constructed for all candidate keywords.

[0069] For example, "your", "ID number", "last four digits", "is", and "how much". In this sentence, if "ID number" is selected as the candidate keyword, the corresponding four edges are (ID number, last digit), (ID number, four digits), (ID number, is), and (ID number, how much).

[0070] Step c, iteratively propagating the weights of each node of the keyword graph according to a preset formula until convergence;

[0071] In this embodiment, the preset formula is:

[0072]

[0073] S represents the weight, d represents the damping coefficient, ε represents the candidate keyword set, i represents the target candidate keyword, j represents the candidate keywords before i, w represents the importance between i and j, out(vj) is the number of candidate keywords. The damping coefficient d can be 0.85.

[0074] Therefore, the weight of i depends on the weight of the edge (j,i) formed by i and each point j in front of i, as well as the sum of the weights of j to other edges.

[0075] Step d: sorting the candidate keywords according to the weights in the iteration result, selecting candidate keywords according to the sorting result, and saving the candidate keywords and their corresponding weights as the keyword set.

[0076] In this embodiment, the keywords can be sorted in reverse order according to the weights, and a number of candidate keywords with the highest rankings can be selected according to the sorting results. The number of candidate keywords can be selected according to actual conditions. Finally, the candidate keywords and their corresponding weights are saved as the keyword set, thereby obtaining the keyword set.

[0077] Step S101-2, determining the segment according to the word graph and the keyword set, wherein the word graph is constructed according to the standard example sentence of the quality inspection item. And the word graph defines a transfer matrix between words. For example, for "you", "your", "ID number", "back", "four digits", "is", "how much", "ID number" can be transferred to the front of "of".

[0078] After obtaining the keywords, the segments are determined based on the word graph. In this embodiment, the word graph is constructed based on the standard example sentences of the quality inspection items.

[0079] Therefore, only the customer service recording text needs to be processed to obtain fragments, and there is no requirement for punctuation of the customer service recording text, so there will be no misjudgment due to the speaker's pause, which can improve the accuracy of quality inspection. For example, in the auto finance loan scenario, "Customer service: Where is the address of your work unit; Customer: In Nanshan District, Shenzhen City, Guangdong". Because the role of the speaker is unknown, the quality inspection model needs to perform quality inspection on both sentences (what the customer says should not be included in the quality inspection scope), which affects the efficiency of quality inspection. Also, due to the speaker's pause, the customer service recording: "Customer service: Where is the address of your work unit" may be recognized by the speech recognition system as "your work unit" and "where is the address". At this time, the quality inspection model is likely to confuse "your work unit" with the quality inspection item "customer work unit verification", resulting in errors.

[0080] Step S102, calculating the scores of the segments, and screening out candidate segments based on the scores;

[0081] The score is a criterion for screening candidate fragments. It can be understood that the score can be a percentage system, a ten-point system, etc.

[0082] Specifically, the step of calculating the score of the segment according to a preset function and screening out candidate segments based on the score includes:

[0083] Step S102a, obtaining the similarity between the segment and the quality inspection item, and calculating the score of the segment belonging to each quality inspection item based on the similarity;

[0084] In this embodiment, the function is defined as Score(Span, Si, Li, Di, Zi), where Score represents the score, Span represents the segment, Si represents the keyword set, Li represents the word graph, Di represents the customer service recording text, and Zi represents the quality inspection item.

[0085] The score of the segment belonging to each quality inspection item is calculated according to the similarity between the segment Span and the quality inspection item Zi, and the higher the similarity, the higher the score. The interval of the score can be set to 0-100.

[0086] Step S102b: if the score is greater than or equal to the score threshold, the corresponding segment is screened as a candidate segment.

[0087] The score threshold may be 80, 70, 60, etc. If the score is greater than or equal to the score threshold, the corresponding segment is screened as a candidate segment.

[0088] On the contrary, if the score is less than the score threshold, the segment corresponding to the score is ignored.

[0089] Step S103: input the candidate segments and their scores into a quality inspection model, and determine the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model.

[0090] Specifically, the step of determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model includes:

[0091] Step S103a: Screening the prediction results to obtain target prediction labels whose probabilities are greater than or equal to a probability threshold;

[0092] Obtain the prediction label output by the quality inspection model at any time, and the prediction label includes the prediction result and its probability. The prediction result is screened based on probability, and the prediction result is divided into two parts: one is the first prediction result with a probability greater than or equal to the probability threshold; the other is the second prediction result with a probability less than the probability threshold. In this embodiment, in order to obtain accurate quality inspection results, the first prediction result with a probability greater than or equal to the probability threshold is marked as the target prediction result, and the target prediction label corresponding to the target prediction result is obtained.

[0093] Step S103b: Compare the target prediction label with the quality inspection item to determine whether the customer service recording meets the requirements;

[0094] In this embodiment, the target prediction labels correspond to the quality inspection items. Compare whether the number of the target prediction labels is consistent with the number of the quality inspection items;

[0095] If the number of the target prediction labels is consistent with the number of the quality inspection items, then further compare whether the contents of each target prediction label are exactly the same as those of the quality inspection items; if the contents of each prediction label are exactly the same as those of the quality inspection items, then determine that the target prediction label includes all of the quality inspection items.

[0096] If the number of the target prediction labels is less than the number of the quality inspection items, it is directly determined that the target prediction label does not include all of the quality inspection items.

[0097] It can be understood that if there are identical prediction labels, all identical target prediction labels are pre-merged.

[0098] Step S103c: If the target prediction tag includes all the quality inspection items, it is determined that the customer service recording meets the requirements and the quality inspection result is determined to be qualified;

[0099] If the target prediction label does not include all of the quality inspection items, it is determined that the customer service recording text does not meet the requirements and the quality inspection result is determined to be unqualified.

[0100] For example, in loan risk assessment, ten quality inspection items are set. If the target prediction label includes these ten quality inspection items, the customer service recording is judged to meet the requirements and the quality inspection result is determined to be qualified; if the target prediction label does not completely include these ten quality inspection items, the customer service recording is judged to be unqualified and the quality inspection result is determined to be unqualified.

[0101] This embodiment uses the above scheme to obtain customer service recording text, mine segments of the customer service recording text; calculate the scores of the segments, and screen candidate segments based on the scores; input the candidate segments and their scores into the quality inspection model, and determine the quality inspection results of the customer service recording according to the prediction results output by the quality inspection model. In this way, segments are mined from the customer service recording, and only the screened candidate segments are quality inspected, which reduces the workload of customer service recording quality inspection and improves quality inspection efficiency.

[0102] Based on the above Figure 2 The first embodiment described above proposes a second embodiment of the present invention, such as Figure 3 As shown, Figure 3 This is a flow chart of the second embodiment of the quality inspection method for customer service recordings of the present invention.

[0103] Furthermore, before the step S101: obtaining the customer service recording text and mining the fragments of the customer service recording text, the following steps are further included:

[0104] Step S100: setting one or more quality inspection items, and setting a number of quality inspection item standard examples for the quality inspection items.

[0105] The quality inspection items are set according to the quality inspection requirements. For example, for loan risk assessment, the quality inspection items may be "whether the person is the person, whether the person has the ability to repay, whether the person is willing to repay", etc.

[0106] Furthermore, several corresponding standard sentences for quality inspection items are set according to the quality inspection items. The standard sentences for quality inspection items are one or more relevant standard sentences for the quality inspection items. For example, for whether it is the person himself, the standard sentences for quality inspection items can be "what is your name", "what is your contact number", "what is your work unit", "what is your ID number". For another example, for the quality inspection item "verification of the customer's work unit address", the corresponding standard sentences can be set to "where is your work unit address", "where do you work now", etc.

[0107] This embodiment uses the above scheme to set one or more quality inspection items, and sets several standard examples for the quality inspection items; obtains the customer service recording text, mines the segments of the customer service recording text; calculates the scores of the segments, and screens out candidate segments based on the scores; inputs the candidate segments and their scores into the quality inspection model, and determines the quality inspection results of the customer service recording based on the prediction results output by the quality inspection model. In this way, segments are mined from the customer service recording, and only the selected candidate segments are quality inspected. In addition, setting the quality inspection items can conduct targeted quality inspections on customer service recordings, which reduces the workload of quality inspection of customer service recordings and improves quality inspection efficiency.

[0108] Based on the above Figure 1 , Figure 2 The first and second embodiments described above propose a third embodiment of the present invention, such as Figure 4 As shown, Figure 4 This is a flow chart of the third embodiment of the quality inspection method for customer service recordings of the present invention.

[0109] Before the step of inputting the candidate segments and their scores into a quality inspection model and determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model, the step further includes:

[0110] Step S1031, training is performed according to quality inspection example sentences and their corresponding quality inspection labels to obtain a quality inspection model;

[0111] The quality inspection model includes a representation of a word embedding layer and a representation of a multi-layer neural network.

[0112] This embodiment builds the quality inspection model based on a multilayer perceptron (MLP), and the layers of the multilayer perceptron are fully connected. The bottom layer of the multilayer perceptron is the input layer, the middle layer is the hidden layer, and the last layer is the output layer. Word embedding is a class of methods that use dense vector representations to represent words and documents. This is an improvement on the traditional bag-of-word model encoding scheme, in which large sparse vectors are used to represent each word or each word in the vector is digitally assigned to represent the entire vocabulary. These representations are sparse because the vocabulary is extensive, so that a given word or document will be geometrically represented by a vector consisting mainly of zero values. In this embodiment, word embedding can be performed by a Word2Vec neural network or a GloVe neural network.

[0113] For the quality inspection model, the initial parameters of the word embedding layer and the multi-layer neural network are random, or the initial parameters are determined based on experience. Therefore, the quality inspection model needs to be trained, and the specific training process is as follows:

[0114] A large number of quality inspection examples are collected and sorted, and the quality inspection labels of the quality inspection examples are marked, and the quality inspection labels are marked as quality inspection example X1, quality inspection example X2...quality inspection example Xn, and the quality inspection labels corresponding to the quality inspection examples are respectively expressed as quality inspection label Y1, quality inspection label Y2...quality inspection label Yn. The quality inspection example is input into the quality inspection model constructed based on the word embedding layer and the multi-layer neural network. The word embedding layer and the multi-layer neural network in the quality inspection model process the quality inspection example according to the initial parameters and output the predicted label Z. After obtaining the predicted label Z, the cross entropy loss function is calculated based on the predicted label Z and the quality inspection label. In this embodiment, the cross entropy loss function is calculated based on mini-batch. The gradients corresponding to the various parameters in the initial model are calculated according to the cross entropy loss function, and the various parameters are updated according to the gradients of the various parameters, that is, the various parameters of the word embedding layer and the multi-layer neural network in the quality inspection model are adjusted. Here, the process of updating the number according to the cross entropy loss function is similar to the existing model parameter update process, and will not be described in detail here.

[0115] Step S1032, judging whether the quality inspection model has converged according to the loss function;

[0116] Determine whether the cross entropy loss function converges. If the cross entropy loss function converges, determine that the corresponding quality inspection model converges.

[0117] Step S1033: if the quality inspection model is in a convergence state, stop training, save model parameters, and obtain the quality inspection model.

[0118] If the initial model is in a convergence state, the training is stopped, the parameters of the last training are saved as the final model parameters, and the quality inspection model is obtained based on the final model parameters.

[0119] On the contrary, if the quality inspection model has not reached a convergence state, the training continues: the iterative update is continuously performed until convergence, and the quality inspection model is finally obtained.

[0120] In this embodiment, the quality inspection model is a classifier, and the output prediction result is a prediction label and its probability. The prediction label corresponds to the quality inspection item. It can be understood that in the actual operation process, there will be a situation where it does not meet any quality inspection item, so the prediction label also includes "others".

[0121] This embodiment uses the above scheme to train a quality inspection model based on quality inspection examples and their corresponding quality inspection labels; judge whether the quality inspection model converges based on the loss function; if the quality inspection model is in a converged state, stop training, save model parameters, and obtain the quality inspection model. Thus, the quality inspection model is trained based on quality inspection examples, which improves the pertinence of the quality inspection model and can improve the effect and accuracy of customer service recording quality inspection.

[0122] In addition, this embodiment also provides a quality inspection device for customer service recordings. Figure 5 , Figure 5 This is a schematic diagram of the functional modules of the first embodiment of the quality inspection device for customer service recording of the present invention.

[0123] In this embodiment, the quality inspection device of the customer service recording is a virtual device, which is stored in Figure 1 The memory 1005 of the quality inspection device for the customer service recording shown is used to implement all functions of the quality inspection program for the customer service recording: used to obtain the customer service recording text and mine fragments of the customer service recording text; used to calculate the scores of the fragments and screen out candidate fragments based on the scores; used to input the candidate fragments and their scores into the quality inspection model, and determine the quality inspection results of the customer service recording according to the prediction results output by the quality inspection model.

[0124] Specifically, the quality inspection device for the customer service recording includes:

[0125] A mining module, used to obtain customer service recording text and mine segments of the customer service recording text;

[0126] A screening module, used to calculate the scores of the fragments and screen out candidate fragments based on the scores;

[0127] The quality inspection module is used to input the candidate segments and their scores into the quality inspection model, and determine the quality inspection results of the customer service recording according to the prediction results output by the quality inspection model.

[0128] Furthermore, the screening module includes:

[0129] An acquisition unit, configured to acquire the similarity between the segment and the quality inspection item, and calculate the score of the segment belonging to each quality inspection item based on the similarity;

[0130] The first screening unit is configured to screen the corresponding segment as a candidate segment if the score is greater than or equal to a score threshold.

[0131] Furthermore, the quality inspection module also includes:

[0132] A training unit, used to obtain a quality inspection model by training based on quality inspection example sentences and their corresponding quality inspection labels;

[0133] A judgment unit, used to judge whether the quality inspection model converges according to the loss function;

[0134] The obtaining unit is used to stop training, save model parameters, and obtain the quality inspection model if the quality inspection model is in a convergence state.

[0135] Furthermore, the mining module also includes:

[0136] A mining unit, used to obtain keywords of the customer service recording text, and obtain a keyword set based on the shutdown words;

[0137] A determination unit is used to determine the segment according to a word graph and the keyword set, wherein the word graph is constructed according to the standard example sentence of the quality inspection item.

[0138] Furthermore, the excavation unit further comprises:

[0139] The tagging subunit is used to perform word segmentation and part-of-speech tagging on the customer service recording text, and filter based on the part-of-speech tagging results to obtain candidate keywords;

[0140] A first selection subunit is used to select a candidate keyword from the candidate keywords in turn, and construct a keyword graph of each candidate keyword, wherein the keyword graph includes four edges respectively formed by the selected candidate keyword and four candidate keywords following the word;

[0141] An iterative subunit, used for iteratively propagating the weights of each node of the keyword graph according to a preset formula until convergence;

[0142] The second selection subunit is used to sort the candidate keywords according to the weights in the iteration result, select the candidate keywords according to the sorting result, and save the candidate keywords and their corresponding weights as the keyword set.

[0143] Furthermore, the quality inspection module also includes:

[0144] A second screening unit, used to screen the prediction results to obtain a target prediction label with a probability greater than or equal to a probability threshold;

[0145] A comparison unit, used to compare the target prediction label with the quality inspection item to determine whether the customer service recording meets the requirements;

[0146] A first determination unit is configured to determine that the customer service recording meets the requirements and determine the quality inspection result as qualified if the target prediction tag includes all the quality inspection items;

[0147] The second determination unit is configured to determine that the customer service recording text does not meet the requirements and determine the quality inspection result as unqualified if the target prediction label does not include all of the quality inspection items.

[0148] Furthermore, the mining module also includes:

[0149] The setting unit is used to set one or more quality inspection items and set a number of quality inspection item standard examples for the quality inspection items.

[0150] In addition, an embodiment of the present invention further provides a computer storage medium, on which a quality inspection program for customer service recordings is stored. When the quality inspection program for customer service recordings is executed by a processor, the steps of the quality inspection method for customer service recordings described above are implemented, which will not be repeated here.

[0151] Compared with the prior art, the present invention proposes a method, device, equipment and storage medium for quality inspection of customer service recordings, which includes: obtaining the text of the customer service recordings, mining segments of the customer service recordings; calculating the scores of the segments, and screening out candidate segments based on the scores; inputting the candidate segments and their scores into the quality inspection model, and determining the quality inspection results of the customer service recordings according to the prediction results output by the quality inspection model. In this way, segments are mined from the customer service recordings, and only the screened candidate segments are quality inspected, which reduces the workload of quality inspection of customer service recordings and improves the efficiency of quality inspection.

[0152] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0153] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0155] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for quality inspection of customer service recordings, characterized in that: The method comprises: Obtaining customer service recording text, and mining fragments of the customer service recording text; Calculating scores of the fragments, and screening out candidate fragments based on the scores; Input the candidate segments and their scores into a quality inspection model, and determine the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model, wherein the quality inspection model includes a representation of a word embedding layer and a representation of a multi-layer neural network; The step of mining the fragments of the customer service recording text includes: Obtain keywords of the customer service recording text, and obtain a keyword set based on the keywords; The step of obtaining keywords of the customer service recording text and obtaining a keyword set based on the keywords includes: The customer service recording text is segmented and tagged with parts of speech using the TextRank algorithm, and filtered based on the part of speech tagging results to obtain candidate keywords; Selecting one candidate keyword from the candidate keywords in turn, and constructing a keyword graph of each candidate keyword, wherein the keyword graph includes four edges respectively formed by the selected candidate keyword and four candidate keywords following the word; According to a preset formula, iteratively propagate the weights of each node of the keyword graph until convergence; Sorting the candidate keywords according to the weights in the iteration result, selecting the candidate keywords according to the sorting result, and saving the candidate keywords and their corresponding weights as the keyword set; The segment is determined according to a word graph and the keyword set, wherein the word graph is constructed according to a standard example sentence of a quality inspection item.

2. The method according to claim 1, characterized in that The step of calculating the scores of the fragments and selecting candidate fragments based on the scores comprises: Obtaining the similarity between the segment and the quality inspection item, and calculating the score of the segment belonging to each quality inspection item based on the similarity; If the score is greater than or equal to the score threshold, the corresponding segment is screened as a candidate segment.

3. The method according to claim 1, characterized in that Before the step of inputting the candidate segments and their scores into a quality inspection model and determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model, the step further includes: The quality inspection model is obtained by training based on the quality inspection example sentences and their corresponding quality inspection labels; Determining whether the quality inspection model converges according to the loss function; If the quality inspection model is in a convergence state, the training is stopped, the model parameters are saved, and the quality inspection model is obtained.

4. The method according to claim 1, characterized in that: The step of determining the quality inspection result of the customer service recording according to the prediction result output by the quality inspection model includes: Screening the prediction results to obtain target prediction labels whose probabilities are greater than or equal to a probability threshold; Compare the target prediction label with the quality inspection item to determine whether the customer service recording meets the requirements; If the target prediction tag includes all of the quality inspection items, it is determined that the customer service recording meets the requirements and the quality inspection result is determined to be qualified; If the target prediction label does not include all of the quality inspection items, it is determined that the customer service recording text does not meet the requirements and the quality inspection result is determined to be unqualified.

5. The method according to claim 1, characterized in that Before the step of obtaining the customer service recording text and mining the fragments of the customer service recording text, the step further includes: One or more quality inspection items are set, and several quality inspection item standard example sentences are set for the quality inspection items.

6. A quality inspection device for customer service recordings, characterized in that: The quality inspection device for the customer service recording includes: A mining module is used to obtain customer service recording text and mine segments of the customer service recording text. The mining module is specifically used to obtain keywords of the customer service recording text and obtain a keyword set based on the keywords; the segment is determined according to a word graph and the keyword set, and the word graph is constructed according to standard examples of quality inspection items; wherein the mining module is specifically used to segment and tag the customer service recording text through the TextRank algorithm, and filter based on the results of the part-of-speech tagging to obtain candidate keywords; one candidate keyword is selected from the candidate keywords in turn, and a keyword graph of each candidate keyword is constructed, the keyword graph includes four edges respectively composed of the selected candidate keyword and the four candidate keywords following the word; according to a preset formula, the weights of each node of the keyword graph are iteratively propagated until convergence; the candidate keywords are sorted according to the weights in the iterative results, candidate keywords are selected according to the sorting results, and the candidate keywords and their corresponding weights are saved as the keyword set; A screening module, used to calculate the scores of the fragments and screen out candidate fragments based on the scores; A quality inspection module is used to input the candidate segments and their scores into a quality inspection model, and determine the quality inspection results of the customer service recording according to the prediction results output by the quality inspection model, wherein the quality inspection model includes a representation of a word embedding layer and a representation of a multi-layer neural network.

7. A quality inspection device for customer service recordings, characterized in that: The quality inspection device for the customer service recording includes a processor, a memory, and a quality inspection program for the customer service recording stored in the memory. When the quality inspection program for the customer service recording is executed by the processor, the steps of the quality inspection method for the customer service recording as described in any one of claims 1 to 5 are implemented.

8. A computer storage medium, characterized in that: The computer storage medium stores a quality inspection program for customer service recordings, and when the quality inspection program for customer service recordings is executed by the processor, the steps of the quality inspection method for customer service recordings as described in any one of claims 1 to 5 are implemented.

Images