Voice quality inspection method, device, equipment and storage medium for customer service personnel

Abstract

The present invention relates to the field of artificial intelligence, and discloses a method, device, equipment and storage medium for voice quality inspection of customer service personnel, which are used to improve the accuracy of scoring the voice of customer service personnel, thereby improving the comprehensiveness and accuracy of voice quality inspection of customer service personnel. The voice quality inspection method of customer service personnel includes: obtaining a voice file of the customer service personnel, the voice file is a set of complete voice dialogues; performing sound recognition on the voice file to obtain a sound vector and a context vector; performing recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data; performing score calculation at the semantic level and the key information level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

Description

Technical Field

[0001] The present invention relates to the field of voice processing, and in particular to a voice quality inspection method, device, equipment and storage medium for customer service personnel. Background Art

[0002] With the improvement of living standards, many people's requirements for customer service quality have also increased accordingly. Therefore, many companies have certain quality inspection requirements for the voice conversations of customer service personnel, that is, the assessment of service quality. The more common quality inspection method is to collect the voice conversations of customer service personnel and score them. The score obtained is the quality inspection result corresponding to the voice of the customer service personnel.

[0003] In the prior art, the quality inspection method is usually to convert speech into text data, and then score it in combination with the text data. Since the scoring is not combined with the context, the accuracy of the scoring is low, resulting in incomplete and inaccurate quality inspection of the customer service staff's speech. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for voice quality inspection of customer service personnel, which are used to improve the accuracy of scoring the voice of customer service personnel, thereby improving the comprehensiveness and accuracy of voice quality inspection of customer service personnel.

[0005] The first aspect of the present invention provides a method for voice quality inspection of customer service personnel, comprising: obtaining a voice file of the customer service personnel, wherein the voice file is a complete set of voice conversations; performing sound recognition on the voice file to obtain a sound vector and a context vector; performing recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data; performing score calculation at the semantic level and the key information level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, wherein the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, performing sound recognition on the voice file to obtain a sound vector and a context vector includes: performing principal component analysis on the voice file to generate a principal component voice file; inputting the voice file into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, the recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data includes: inputting the sound vector into a pre-trained language model, performing text recognition in combination with a preset acoustic model, and generating text data; inputting the sound vector and the context vector into a pre-trained voiceprint recognition model to perform voiceprint recognition and generate at least two voiceprint features.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, score calculation is performed at the semantic level and the key information pair level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel, including: score calculation is performed based on the context vector, the at least two voiceprint features and the text data to generate a semantic score; score calculation is performed based on the context vector, the at least two voiceprint features and the text data to generate a key information pair score; calculation is performed based on the semantic score and the key information pair score to generate a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0009] Optionally, in a fourth implementation manner of the first aspect of the present invention, the score calculation based on the context vector, the at least two voiceprint features and the text data to generate a semantic score includes: filtering the context vector, the text data and the at least two voiceprint features to generate a filtered context vector, the filtered text data and the at least two filtered voiceprint features; inputting the filtered context vector, the filtered text data and the at least two filtered voiceprint features into a pre-trained semantic model for calculation to generate a semantic score.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, the score calculation based on the context vector, the at least two voiceprint features and the text data to generate a key information pair score includes: performing domain matching based on the text data to obtain a target question and answer domain; based on the target question and answer domain, searching in a preset model library to obtain a target intent recognition model; inputting the filtered context vector, the filtered text data and the at least two filtered voiceprint features into the target intent recognition model for calculation to generate a key information pair score.

[0011] The second aspect of the present invention provides a voice quality inspection device for customer service personnel, including: an acquisition module, used to obtain the voice file of the customer service personnel, the voice file is a complete set of voice conversations; a first recognition module, used to perform sound recognition on the voice file to obtain a sound vector and a context vector; a second recognition module, used to perform recognition based on the sound vector and the context vector, and generate at least two voiceprint features and text data; a calculation module, used to perform score calculation at the semantic level and the key information level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0012] Optionally, in a first implementation method of the second aspect of the present invention, the first recognition module can also be specifically used to: perform principal component analysis on the voice file to generate a principal component voice file; input the voice file into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

[0013] Optionally, in a second implementation method of the second aspect of the present invention, the second recognition module can also be specifically used to: input the sound vector into a pre-trained language model, perform text recognition in combination with a preset acoustic model, and generate text data; input the sound vector and the context vector into a pre-trained voiceprint recognition model for voiceprint recognition, and generate at least two voiceprint features.

[0014] Optionally, in a third implementation method of the second aspect of the present invention, the calculation module includes: a semantic score calculation unit, used to perform score calculation according to the context vector, the at least two voiceprint features and the text data to generate a semantic score; an information pair score calculation unit, used to perform score calculation according to the context vector, the at least two voiceprint features and the text data to generate a key information pair score; a comprehensive score calculation unit, used to calculate the score based on the semantic score and the key information pair to generate a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0015] Optionally, in a fourth implementation of the second aspect of the present invention, the semantic score calculation unit can also be used to: filter the context vector, the text data and the at least two voiceprint features to generate a filtered context vector, a filtered text data and the at least two filtered voiceprint features; input the filtered context vector, the filtered text data and the at least two filtered voiceprint features into a pre-trained semantic model for calculation to generate a semantic score.

[0016] Optionally, in the fifth implementation method of the second aspect of the present invention, the information pair score calculation unit can also be specifically used to: perform domain matching based on the text data to obtain a target question and answer domain; based on the target question and answer domain, search in a preset model library to obtain a target intention recognition model; input the filtered context vector, the filtered text data and the at least two filtered voiceprint features into the target intention recognition model for calculation to generate a key information pair score.

[0017] The third aspect of the present invention provides a voice quality inspection device for customer service personnel, comprising: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory so that the voice quality inspection device for customer service personnel executes the above-mentioned voice quality inspection method for customer service personnel.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned voice quality inspection method for customer service personnel.

[0019] In the technical solution provided by the present invention, a voice file of a customer service staff is obtained, and the voice file is a set of complete voice dialogues; sound recognition is performed on the voice file to obtain a sound vector and a context vector; recognition is performed based on the sound vector and the context vector to generate at least two voiceprint features and text data; scores are calculated at the semantic level and the key information pair level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service staff. In an embodiment of the present invention, a voice file is recognized as at least two voiceprint features and text data, and then the voice data of the customer service staff is scored at the semantic level and the key information pair level based on the at least two voiceprint features and the text data to complete the quality inspection of the customer service staff's voice, thereby improving the accuracy of the score, thereby improving the comprehensiveness and accuracy of the voice quality inspection of the customer service staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an embodiment of a method for voice quality inspection of customer service personnel in an embodiment of the present invention;

[0021] Figure 2 It is a schematic diagram of another embodiment of the method for voice quality inspection of customer service personnel in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of an embodiment of a voice quality inspection device for customer service personnel in an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of another embodiment of the voice quality inspection device for customer service personnel in an embodiment of the present invention;

[0024] Figure 5 The figure is a schematic diagram of an embodiment of a voice quality inspection device for customer service personnel in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The embodiments of the present invention provide a method, device, equipment and storage medium for voice quality inspection of customer service personnel, which are used to improve the accuracy of scoring the voice of customer service personnel, thereby improving the comprehensiveness and accuracy of voice quality inspection of customer service personnel.

[0026] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the voice quality inspection method of customer service personnel in an embodiment of the present invention includes:

[0028] 101. Obtain the voice file of the customer service staff. The voice file is a complete set of voice dialogues.

[0029] The server obtains the voice file of the customer service staff. It should be noted that the voice file is a complete set of voice dialogues, that is, a complete set of dialogues between the customer service staff and the user.

[0030] When the customer service staff's voice file needs to be quality-checked, the server randomly obtains a certain voice file of the customer service staff, or specifies to obtain a certain voice file of the customer service staff, but the voice file is a complete voice conversation. Voice files are files saved to the storage device through terminal recording. A complete voice conversation refers to the conversation between the time the customer service staff answers the call and the time the customer service staff hangs up the call. For example, the customer service staff answers the call at 10:02 and hangs up at 10:04. The conversation between 10:02 and 10:04 is a complete voice conversation, that is, a voice file of the customer service staff.

[0031] It is understandable that the execution subject of the present invention may be a voice quality inspection device of a customer service staff, or may be a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0032] 102. Perform sound recognition on the voice file to obtain a sound vector and a context vector;

[0033] After acquiring the voice file, the server performs sound recognition on the voice file to obtain a sound vector and a context vector.

[0034] After the server obtains the voice file of the customer service staff, it also needs to perform sound recognition on the voice file. The process of sound recognition in this embodiment can be understood as the process of preprocessing the voice file, that is, performing data dimension reduction on the voice file, thereby removing noise and unimportant features, and retaining only the most important features in the voice file, thereby obtaining a sound vector and a context vector. After implementing this step, the sound vector and the context vector are obtained, and subsequent calculations based on the sound vector and the context vector can effectively improve the speed of data processing and reduce the time calculation cost of the algorithm.

[0035] It should be noted that, in this embodiment, the voice file needs to be vectorized first. At this time, the vectorized voice file obtained may be a 768-dimensional vector. After obtaining the vectorized voice file, the 768-dimensional vector is compressed to a 128-dimensional vector, and then the compressed vector is recognized to generate a sound vector and a context vector.

[0036] 103. Perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data;

[0037] The server performs voiceprint recognition and text recognition based on the sound vector and the context vector, thereby generating at least two voiceprint features and text data.

[0038] In this embodiment, two recognition models are involved. When performing voiceprint recognition, the voiceprint recognition model is used. By inputting the sound vector and the context vector into the voiceprint recognition model, at least two voiceprint features can be generated. When performing text recognition, the text recognition model is used. By inputting the sound vector and the context vector into the text recognition model, text data can be generated.

[0039] Since a conversation involves the voices of two or more people, the server needs to recognize the voices of different people. The method used is voiceprint recognition. The voice of speaker A is recognized as voice 1, and the voice of speaker 2 is recognized as voice 2. At the same time, the sound vector is combined with the context vector to recognize it as text data.

[0040] 104. Based on the context vector, at least two voiceprint features and text data, scores are calculated at the semantic level and the key information pair level to obtain a target comprehensive score, which is used to represent the voice quality inspection result of the customer service staff.

[0041] The server calculates scores at the semantic level and the key information pair level based on the context vector, at least two voiceprint features and text data, thereby obtaining a target comprehensive score for identifying the customer service voice quality inspection results.

[0042] It should be noted that in this embodiment, the customer service voice is quality inspected at two levels, one is quality inspected at the semantic level, and the other is quality inspected at the key information level. The quality inspection results at the two levels are integrated to generate a voice quality inspection result for identifying the customer service personnel. When quality inspecting the customer service voice at the semantic level, a semantic scoring model is required for quality inspection, and when quality inspecting the customer service voice at the key information level, a key information pair scoring model is required for quality inspection.

[0043] In an embodiment of the present invention, a voice file is identified as at least two voiceprint features and text data, and then the voice data of the customer service personnel is scored from a semantic level and a key information level based on the at least two voiceprint features and the text data to complete the quality inspection of the customer service personnel's voice, thereby improving the accuracy of the scoring and thus improving the comprehensiveness and accuracy of the quality inspection of the customer service personnel's voice.

[0044] See also Figure 2 Another embodiment of the voice quality inspection method for customer service personnel in the embodiment of the present invention includes:

[0045] 201. Obtain the voice file of the customer service staff, which is a complete voice conversation.

[0046] The server obtains the voice file of the customer service staff. It should be noted that the voice file is a complete set of voice dialogues, that is, a complete set of dialogues between the customer service staff and the user.

[0047] When the customer service staff's voice file needs to be quality-checked, the server randomly obtains a certain voice file of the customer service staff, or specifies to obtain a certain voice file of the customer service staff, but the voice file is a complete voice conversation. Voice files are files saved to the storage device through terminal recording. A complete voice conversation refers to the conversation between the time the customer service staff answers the call and the time the customer service staff hangs up the call. For example, the customer service staff answers the call at 10:02 and hangs up at 10:04. The conversation between 10:02 and 10:04 is a complete voice conversation, that is, a voice file of the customer service staff.

[0048] 202. Perform sound recognition on the voice file to obtain a sound vector and a context vector;

[0049] After acquiring the voice file, the server performs sound recognition on the voice file to obtain a sound vector and a context vector.

[0050] After the server obtains the voice file of the customer service staff, it also needs to perform sound recognition on the voice file. The process of sound recognition in this embodiment can be understood as the process of preprocessing the voice file, that is, performing data dimension reduction on the voice file, thereby removing noise and unimportant features, and retaining only the most important features in the voice file, thereby obtaining a sound vector and a context vector. After implementing this step, the sound vector and the context vector are obtained, and subsequent calculations based on the sound vector and the context vector can effectively improve the speed of data processing and reduce the time calculation cost of the algorithm.

[0051] It should be noted that, in this embodiment, the voice file needs to be vectorized first. At this time, the vectorized voice file obtained may be a 768-dimensional vector. After obtaining the vectorized voice file, the 768-dimensional vector is compressed to a 128-dimensional vector, and then the compressed vector is recognized to generate a sound vector and a context vector.

[0052] Specifically, the server performs principal component analysis on the voice file to generate a principal component voice file; then the server inputs the voice file into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

[0053] In this embodiment, the server needs to perform principal component analysis on the voice file, and adopts the principal component analysis method (PCA) for analysis. Before the principal component analysis, the voice file is first vectorized to obtain the vectorized voice file, and then the vectorized voice file is subjected to identification principal component analysis to generate the principal component voice file, which at this time exists in the form of vectors. In this embodiment, the singular value decomposition algorithm is mainly used to implement the principal component analysis, and the specific process is as follows: the vectorized voice file is subjected to average value removal processing, that is, each eigenvalue is subtracted from the respective average value to generate the voice file after the average value is removed; then the covariance matrix is ​​calculated based on the voice file after the average value is removed, and then the singular value decomposition algorithm is called to calculate the eigenvalues ​​and eigenvectors of the covariance matrix; the eigenvalues ​​are sorted in order from small to large, and multiple eigenvalues ​​with larger values ​​are extracted as multiple target eigenvalues; the eigenvectors corresponding to the multiple target eigenvalues ​​are respectively used as column vectors to form an eigenvector matrix; the vectorized voice file is converted into a space constructed by the eigenvector matrix, so as to generate the principal component voice file.

[0054] The server inputs the main component voice file into the unsupervised deep learning network trained by cosine for recognition, and generates a sound vector and a context vector. The recognition process is as follows: input the main component voice file into the first neural network layer of the learning network to generate a first sound feature vector and a first context feature vector; input the first sound feature vector and the first context feature vector into the second neural network layer to generate a second sound feature vector and a second context feature vector. In this way, the output of each neural network layer is used as the input of the next neural network layer for convolution, and after the calculation of the last neural network layer, a sound vector and a context vector are generated.

[0055] 203. Perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data;

[0056] The server performs voiceprint recognition and text recognition based on the sound vector and the context vector, thereby generating at least two voiceprint features and text data.

[0057] In this embodiment, two recognition models are involved. When performing voiceprint recognition, the voiceprint recognition model is used. By inputting the sound vector and the context vector into the voiceprint recognition model, at least two voiceprint features can be generated. When performing text recognition, the text recognition model is used. By inputting the sound vector and the context vector into the text recognition model, text data can be generated.

[0058] Since a conversation involves the voices of two or more people, the server needs to recognize the voices of different people. The method used is voiceprint recognition. The voice of speaker A is recognized as voice 1, and the voice of speaker 2 is recognized as voice 2. At the same time, the sound vector is combined with the context vector to recognize it as text data.

[0059] Specifically, the server inputs the sound vector into a pre-trained language model, combines it with a preset acoustic model for text recognition, and generates text data; the server inputs the sound vector and context vector into a pre-trained voiceprint recognition model for voiceprint recognition, and generates at least two voiceprint features.

[0060] The server inputs the sound vector into a pre-trained language model, and performs text recognition in combination with a preset acoustic model to generate text data. It should be noted that since the text recognition process is a prior art, no further explanation is given. At the same time, the server inputs the sound vector and the context vector into a pre-trained voiceprint recognition model for voiceprint recognition. The specific process is as follows: extract each frame of Mel spectrum based on the sound vector and the context vector to obtain multiple frames of Mel spectrum; then input the multiple frames of Mel spectrum into the pre-trained voiceprint recognition model to obtain the probability parameters corresponding to each frame of Mel spectrum, obtain multiple probability parameters, and then regularize the multiple probability parameters to generate multiple regularized probability parameters, add the sub-sound vectors and sub-context vectors corresponding to the probability parameters belonging to the same label, and generate at least two voiceprint features.

[0061] It should be noted that the sound vector includes multiple sub-sound vectors, and the context vector also includes multiple sub-context vectors. The server extracts the Mel spectrum corresponding to each frame according to different sub-sound vectors and corresponding sub-context vectors, and then accumulates the sub-sound vectors and sub-context vectors corresponding to the probability parameters of the same label based on multiple probability parameters, thereby generating at least two voiceprint features.

[0062] 204. Calculate a score based on the context vector, at least two voiceprint features, and text data to generate a semantic score;

[0063] The server calculates scores for the context vector, at least two voiceprint features, and text data to generate a semantic score.

[0064] Specifically, the server filters the context vector, text data and at least two voiceprint features to generate filtered context vectors, filtered text data and at least two filtered voiceprint features; the server inputs the filtered context vectors, filtered text data and at least two filtered voiceprint features into a pre-trained semantic model for calculation to generate a semantic score.

[0065] The server first filters out the first target text data corresponding to the repeatedly appearing words, as well as the context vectors and voiceprint features corresponding to the first target text data. At the same time, the server also filters out the second target text vectors corresponding to the words that appear too few times, as well as the context vectors and corresponding voiceprint features corresponding to the second target text vectors, thereby obtaining the filtered context vector, the filtered text data and the estimated at least two voiceprint features after filtering; the server queries the standard knowledge base for matching target answer sentence pairs based on the at least two filtered voiceprint features, the filtered text data and the filtered context vector, and then inputs the target answer sentence pairs, the at least two filtered voiceprint features, the filtered text data and the filtered context vector into a pre-trained semantic model for probability calculation to generate a similarity matching value, i.e., a semantic score.

[0066] 205. Calculate scores based on the context vector, at least two voiceprint features, and text data to generate key information pair scores;

[0067] The server calculates scores for the context vector, at least two voiceprint features, and text data to obtain key information pair scores.

[0068] Specifically, the server performs domain matching based on the text data to obtain the target question and answer domain; the server searches in the corresponding model library based on the target question and answer domain to obtain the target intent recognition model; the server inputs the filtered context vector, the filtered text data, and at least two filtered voiceprint features into the target intent recognition model for calculation to generate a key information pair score.

[0069] The server first performs domain matching in a preset domain library based on the text data to obtain the target domain, and then queries the corresponding target intent recognition algorithm model in the target domain based on the text data, and then inputs the target answer sentence pair used in step 204 into the target intent recognition algorithm model to obtain the target key information pair; then the filtered context vector, the filtered text data and at least two filtered voiceprint features are input into the target intent recognition algorithm model for calculation to obtain the customer service key information pair, and the information pair similarity between the target key information pair and the customer service key information pair is calculated to obtain the key information pair score.

[0070] 206. The scores are calculated based on the semantic scores and the key information to generate a target comprehensive score, which is used to represent the voice quality inspection results of the customer service personnel.

[0071] The server calculates a comprehensive score based on the semantic score and key information, and generates a target comprehensive score for representing the voice quality inspection result of the customer service staff.

[0072] In this embodiment, the server averages the semantic score and the key information pair score to obtain a target comprehensive score, which represents the voice quality inspection result of the customer service staff. In other embodiments, the semantic score and the key information pair score can also be combined with other weight parameters to calculate the target comprehensive score, which is not specifically limited in this solution.

[0073] In an embodiment of the present invention, a voice file is identified as at least two voiceprint features and text data, and then the voice data of the customer service personnel is scored from a semantic level and a key information level based on the at least two voiceprint features and the text data to complete the quality inspection of the customer service personnel's voice, thereby improving the accuracy of the scoring and thus improving the comprehensiveness and accuracy of the quality inspection of the customer service personnel's voice.

[0074] The above describes the voice quality inspection method of the customer service personnel in the embodiment of the present invention. The following describes the voice quality inspection device of the customer service personnel in the embodiment of the present invention. Figure 3 , an embodiment of the voice quality inspection device for customer service personnel in an embodiment of the present invention includes:

[0075] An acquisition module 301 is used to acquire a voice file of a customer service staff member, wherein the voice file is a complete voice dialogue;

[0076] A first recognition module 302, configured to perform sound recognition on the voice file to obtain a sound vector and a context vector;

[0077] A second recognition module 303, configured to perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data;

[0078] The calculation module 304 is used to calculate scores at the semantic level and the key information level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0079] In an embodiment of the present invention, a voice file is identified as at least two voiceprint features and text data, and then the voice data of the customer service personnel is scored from a semantic level and a key information level based on the at least two voiceprint features and the text data to complete the quality inspection of the customer service personnel's voice, thereby improving the accuracy of the scoring and thus improving the comprehensiveness and accuracy of the quality inspection of the customer service personnel's voice.

[0080] See also Figure 4 Another embodiment of the voice quality inspection device for customer service personnel in the embodiment of the present invention includes:

[0081] An acquisition module 301 is used to acquire a voice file of a customer service staff member, wherein the voice file is a complete voice dialogue;

[0082] A first recognition module 302, configured to perform sound recognition on the voice file to obtain a sound vector and a context vector;

[0083] A second recognition module 303, configured to perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data;

[0084] The calculation module 304 is used to calculate scores at the semantic level and the key information level based on the context vector, the at least two voiceprint features and the text data to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

[0085] Optionally, the first identification module 302 may also be specifically configured to:

[0086] Performing principal component analysis on the voice file to generate a principal component voice file;

[0087] The voice file is input into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

[0088] Optionally, the second identification module 303 may also be specifically configured to:

[0089] Inputting the sound vector into a pre-trained language model, performing text recognition in combination with a preset acoustic model, and generating text data;

[0090] The sound vector and the context vector are input into a pre-trained voiceprint recognition model to perform voiceprint recognition, and generate at least two voiceprint features.

[0091] Optionally, the calculation module 304 includes:

[0092] A semantic score calculation unit 3041, configured to perform score calculation according to the context vector, the at least two voiceprint features and the text data to generate a semantic score;

[0093] An information pair score calculation unit 3042, configured to perform score calculation based on the context vector, the at least two voiceprint features and the text data to generate a key information pair score;

[0094] The comprehensive score calculation unit 3043 is used to calculate the score based on the semantic score and the key information to generate a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service staff.

[0095] Optionally, the semantic score calculation unit 3041 may also be used for:

[0096] filtering the context vector, the text data, and the at least two voiceprint features to generate a filtered context vector, filtered text data, and at least two filtered voiceprint features;

[0097] The filtered context vector, the filtered text data and the filtered at least two voiceprint features are input into a pre-trained semantic model for calculation to generate a semantic score.

[0098] Optionally, the information pair score calculation unit 3042 may also be specifically used for:

[0099] Performing domain matching based on the text data to obtain a target question and answer domain;

[0100] Based on the target question-answering domain, searching in a preset model library to obtain a target intent recognition model;

[0101] The filtered context vector, the filtered text data and the at least two filtered voiceprint features are input into the target intent recognition model for calculation to generate a key information pair score.

[0102] In an embodiment of the present invention, a voice file is identified as at least two voiceprint features and text data, and then the voice data of the customer service personnel is scored from a semantic level and a key information level based on the at least two voiceprint features and the text data to complete the quality inspection of the customer service personnel's voice, thereby improving the accuracy of the scoring and thus improving the comprehensiveness and accuracy of the quality inspection of the customer service personnel's voice.

[0103] above Figure 3 and Figure 4 The voice quality inspection device for customer service personnel in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The voice quality inspection device for customer service personnel in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0104] Figure 51 is a schematic diagram of the structure of a voice quality inspection device for customer service personnel provided in an embodiment of the present invention. The voice quality inspection device 500 for customer service personnel may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 may be temporary storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the voice quality inspection device 500 for customer service personnel. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the voice quality inspection device 500 for customer service personnel.

[0105] The voice quality inspection device 500 for customer service personnel may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the voice quality inspection device for customer service personnel shown does not constitute a limitation on the voice quality inspection device for customer service personnel, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0106] The present invention also provides a voice quality inspection device for customer service personnel, wherein the computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the voice quality inspection method for customer service personnel in the above-mentioned embodiments.

[0107] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the customer service personnel's voice quality inspection method.

[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0109] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0110] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for voice quality inspection of customer service personnel, characterized in that: The voice quality inspection method of the customer service staff includes: Obtaining a voice file of a customer service representative, wherein the voice file is a complete set of voice conversations; Performing sound recognition on the voice file to obtain a sound vector and a context vector; Perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data; Based on the context vector, the at least two voiceprint features and the text data, score calculation is performed at the semantic level and the key information pair level to obtain a target comprehensive score, where the target comprehensive score is used to represent the voice quality inspection result of the customer service staff; The generating at least two voiceprint features and text data by performing recognition based on the sound vector and the context vector includes: Input the sound vector into a pre-trained language model, perform text recognition in combination with a preset acoustic model, and generate text data; Inputting the sound vector and the context vector into a pre-trained voiceprint recognition model to perform voiceprint recognition, and generating at least two voiceprint features; The step of inputting the sound vector and the context vector into a pre-trained voiceprint recognition model for voiceprint recognition includes: extracting each frame of Mel spectrum based on the sound vector and the context vector to obtain multiple frames of Mel spectrum; then inputting the multiple frames of Mel spectrum into the pre-trained voiceprint recognition model to obtain probability parameters corresponding to each frame of Mel spectrum, obtaining multiple probability parameters, and then regularizing the multiple probability parameters to generate multiple regularized probability parameters, adding the sub-sound vectors and sub-context vectors corresponding to the probability parameters belonging to the same label to generate at least two voiceprint features.

2. The voice quality inspection method for customer service personnel according to claim 1, characterized in that: The performing sound recognition on the voice file to obtain a sound vector and a context vector comprises: Performing principal component analysis on the voice file to generate a principal component voice file; The voice file is input into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

3. The voice quality inspection method for customer service personnel according to claim 1, characterized in that: The score calculation is performed based on the context vector, the at least two voiceprint features and the text data at the semantic level and the key information pair level to obtain a target comprehensive score, and the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel, including: Performing score calculation based on the context vector, the at least two voiceprint features, and the text data to generate a semantic score; Performing score calculation based on the context vector, the at least two voiceprint features, and the text data to generate a key information pair score; The scores are calculated based on the semantic scores and the key information to generate a target comprehensive score, where the target comprehensive score is used to represent the voice quality inspection result of the customer service personnel.

4. The method for voice quality inspection of customer service personnel according to claim 3, characterized in that: The generating a semantic score by performing score calculation according to the context vector, the at least two voiceprint features and the text data comprises: filtering the context vector, the text data, and the at least two voiceprint features to generate a filtered context vector, filtered text data, and at least two filtered voiceprint features; The filtered context vector, the filtered text data and the filtered at least two voiceprint features are input into a pre-trained semantic model for calculation to generate a semantic score.

5. The method for voice quality inspection of customer service personnel according to claim 4, characterized in that: The step of performing score calculation according to the context vector, the at least two voiceprint features and the text data to generate a key information pair score comprises: Performing domain matching based on the text data to obtain a target question and answer domain; Based on the target question-answering domain, searching in a preset model library to obtain a target intent recognition model; The filtered context vector, the filtered text data and the at least two filtered voiceprint features are input into the target intent recognition model for calculation to generate a key information pair score.

6. A voice quality inspection device for customer service personnel, characterized in that: The voice quality inspection device of the customer service staff includes: An acquisition module is used to acquire a voice file of a customer service staff member, wherein the voice file is a complete voice conversation; A first recognition module, used for performing sound recognition on the voice file to obtain a sound vector and a context vector; A second recognition module, configured to perform recognition based on the sound vector and the context vector to generate at least two voiceprint features and text data; A calculation module, configured to calculate scores at a semantic level and a key information pair level based on the context vector, the at least two voiceprint features, and the text data to obtain a target comprehensive score, wherein the target comprehensive score is used to represent a voice quality inspection result of a customer service staff member; The second recognition module is further used to input the sound vector into a pre-trained language model, perform text recognition in combination with a preset acoustic model, and generate text data; input the sound vector and the context vector into a pre-trained voiceprint recognition model to perform voiceprint recognition and generate at least two voiceprint features; The second recognition module is also used to extract each frame of Mel spectrum based on the sound vector and the context vector to obtain multiple frames of Mel spectrum; then input the multiple frames of Mel spectrum into a pre-trained voiceprint recognition model to obtain probability parameters corresponding to each frame of Mel spectrum, obtain multiple probability parameters, and then regularize the multiple probability parameters to generate multiple regularized probability parameters, add the sub-sound vectors and sub-context vectors corresponding to the probability parameters belonging to the same label, and generate at least two voiceprint features.

7. The voice quality inspection device for customer service personnel according to claim 6, characterized in that: The first identification module may also be specifically used for: Performing principal component analysis on the voice file to generate a principal component voice file; The voice file is input into a pre-trained unsupervised deep learning network for recognition to generate a sound vector and a context vector.

8. A voice quality inspection device for customer service personnel, characterized in that: The voice quality inspection device for customer service personnel includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory so that the voice quality inspection device for customer service personnel executes the voice quality inspection method for customer service personnel as described in any one of claims 1-5.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the voice quality inspection method for customer service personnel as described in any one of claims 1-5 is implemented.

Images