Offline network customer service quality inspection method and device, electronic equipment and storage medium

Abstract

The invention provides an offline network customer service quality inspection method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring offline customer service data; identifying the image data to extract facial expressions, performing sentiment analysis and keyword extraction on the voice dialogue data to obtain service items and voice sentiment analysis, and performing sentiment analysis based on the facial expressions and the voice sentiment analysis to obtain fused sentiment analysis; evaluating the fused sentiment analysis to obtain a first evaluation score, evaluating the service item by using a preset evaluation standard to obtain a second evaluation score, and fusing the first evaluation score and the second evaluation score to obtain a machine quality inspection result; performing manual sampling inspection on the voice dialogue data and the image data to obtain a manual quality inspection result; and fusing a machine quality inspection result and a manual quality inspection result, and obtaining a service quality inspection result based on a fusion result, thereby realizing accurate offline network customer service quality inspection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and specifically to an offline network customer service quality inspection method, device, electronic device and storage medium. Background Art

[0002] Online customer service can maintain relationships with customers, provide after-sales support, customer consultation, answers and dispute resolution, etc. Quality inspection of online customer service can timely discover problems in the customer service communication process, such as inappropriate language and improper attitude, thereby prompting customer service to improve and enhance the overall service quality. In related technologies, by using online customer service to serve customers, service quality inspection is also for online customer service, and quality inspection for offline customer service is relatively lacking, and the accuracy of quality inspection results is relatively low. Summary of the invention

[0003] The present application provides an offline network customer service quality inspection method, device, electronic device and storage medium, which can achieve relatively accurate offline network customer service quality inspection.

[0004] The technical solution of the embodiment of the present application is as follows: In a first aspect, an embodiment of the present application provides an offline network customer service quality inspection method, the method comprising: Acquire offline customer service data, wherein the offline customer service data includes multiple groups of voice conversation data and image data, and the voice conversation data corresponds to the image data; Recognize each of the image data, extract facial expressions, perform sentiment analysis and keyword extraction on each of the voice conversation data, obtain service items and voice sentiment analysis, perform sentiment analysis based on the facial expressions and the voice sentiment analysis, and obtain fusion sentiment analysis; Evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service item using a preset evaluation standard to obtain a second evaluation score, and fuse the first evaluation score and the second evaluation score to obtain a machine quality inspection result; Performing manual sampling inspection on each of the voice dialogue data and the image data to obtain manual quality inspection results; The machine quality inspection result and the manual quality inspection result are fused, and a service quality inspection result is obtained based on the fusion result.

[0005] In the above technical scheme, firstly, offline customer service data is obtained. The offline customer service data includes multiple groups of voice conversation data and image data. The voice conversation data corresponds to the image data. By obtaining the offline voice conversation data and image data, data support is provided for subsequent facial expression analysis, voice emotion analysis and project evaluation; each image data is recognized and facial expression is extracted, so that the quality inspection accuracy can be evaluated according to the facial expression in the future; emotion analysis and keyword extraction are performed on each voice conversation data respectively to obtain service items and voice emotion analysis, so that the quality inspection accuracy can be evaluated according to the voice emotion analysis in the future; emotion analysis is performed based on facial expression and voice emotion analysis to obtain fused emotion analysis, so that the emotion analysis is more accurate, thereby improving the quality inspection results. The accuracy of the results; evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service items using a preset evaluation standard to obtain a second evaluation score, fuse the first evaluation score and the second evaluation score to obtain a machine quality inspection result. Automatic quality inspection can not only improve the quality inspection speed, but also improve the accuracy of the quality inspection results, thus preparing for subsequent accurate quality inspection results; perform manual sampling on each voice conversation data and image data to obtain a manual quality inspection result, which has a high accuracy; fuse the machine quality inspection result and the manual quality inspection result, and obtain the service quality inspection result based on the fusion result. By fusing machine quality inspection and manual quality inspection, the diversity of quality inspection can be increased, and the accuracy of network customer service quality inspection can be improved.

[0006] In some embodiments of the present application, the performing of sentiment analysis based on the facial expression and the voice sentiment analysis to obtain a fused sentiment analysis includes: Analyzing the facial expression to obtain facial emotion analysis, and fusing the facial emotion analysis with the voice emotion analysis to obtain an initial emotion analysis; According to the progress of the service project, facial emotion prediction is performed based on the facial expression to obtain predicted facial emotion, and voice emotion prediction is performed based on the voice dialogue data to obtain predicted voice emotion; Fusing the predicted facial emotion and the predicted voice emotion to obtain a first predicted emotion analysis; The initial sentiment analysis is adjusted using the first predicted sentiment analysis to obtain the fused sentiment analysis.

[0007] In some embodiments of the present application, the adjusting the initial sentiment analysis by using the first predicted sentiment analysis to obtain the fused sentiment analysis includes: Predicting the initial sentiment analysis to obtain a second predicted sentiment analysis; The first predicted sentiment analysis is compared with the second predicted sentiment analysis, and when the first predicted sentiment analysis is consistent with the second predicted sentiment analysis, the initial sentiment analysis is used as the fused sentiment analysis.

[0008] In some embodiments of the present application, the fusing of the facial emotion analysis and the voice emotion analysis to obtain an initial emotion analysis includes: Establishing a facial emotion node according to the facial emotion analysis, establishing a voice emotion node according to the voice emotion analysis, and determining a first probability distribution corresponding to the facial emotion node and a second probability distribution corresponding to the voice emotion node; A fused emotion probability distribution is obtained according to the first probability distribution and the second probability distribution, and the initial emotion analysis is obtained according to the fused emotion probability distribution.

[0009] In some embodiments of the present application, the performing of sentiment analysis and keyword extraction on each of the voice dialogue data to obtain service items and voice sentiment analysis includes: Converting the voice conversation data into text conversation data, and extracting keywords from the text conversation data using preset project prompt words to obtain the service project, wherein the project prompt words are obtained by extracting keywords from offline network customer service; The voice conversation data is subjected to sentiment analysis using preset sentiment prompt words to obtain a voice sentiment analysis.

[0010] In some embodiments of the present application, the evaluation criteria include service quality criteria, service efficiency criteria, and service satisfaction criteria; The using of a preset evaluation standard to evaluate the service item to obtain a second evaluation score includes: The service items are evaluated respectively using the service quality standard, the service efficiency standard and the service satisfaction standard to obtain a quality score corresponding to the service quality standard, an efficiency score corresponding to the service efficiency standard and a satisfaction score corresponding to the service satisfaction standard; The quality score, the efficiency score and the satisfaction score are weightedly summed to obtain the second evaluation score.

[0011] In some embodiments of the present application, fusing the machine quality inspection result and the manual quality inspection result, and obtaining a service quality inspection result based on the fusion result, includes: Obtaining a preset first quality inspection weight and a preset second quality inspection weight, wherein the first quality inspection weight is less than the second quality inspection weight; Multiplying the first quality inspection weight and the machine quality inspection result to obtain a first result, and multiplying the second quality inspection weight and the manual quality inspection result to obtain a second result; The first result and the second result are added together to obtain a fusion result, and the fusion result is matched with a preset service level to obtain the service quality inspection result.

[0012] In a second aspect, an embodiment of the present application provides an offline network customer service quality inspection device, the device comprising: A data acquisition module, used to acquire offline customer service data, wherein the offline customer service data includes multiple groups of voice dialogue data and image data, and the voice dialogue data corresponds to the image data; A first data processing module is used to identify each of the image data, extract facial expressions, perform sentiment analysis and keyword extraction on each of the voice dialogue data, obtain service items and voice sentiment analysis, perform sentiment analysis based on the facial expressions and the voice sentiment analysis, and obtain fusion sentiment analysis; A first data fusion module is used to evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service item using a preset evaluation standard to obtain a second evaluation score, and fuse the first evaluation score with the second evaluation score to obtain a machine quality inspection result; A second data processing module is used to perform manual sampling inspection on each of the voice dialogue data and the image data to obtain manual quality inspection results; The second data fusion module is used to fuse the machine quality inspection result and the manual quality inspection result, and obtain the service quality inspection result based on the fusion result.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a user interface, a communication bus and a network interface, wherein the processor, the memory, the user interface and the network interface are respectively connected to the communication bus, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute instructions stored in the memory so that the electronic device executes an offline network customer service quality inspection method as described in any one of the items provided in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, the offline network customer service quality inspection method described in any one of the above-mentioned first aspects is executed.

[0015] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Since the offline customer service data is first obtained, the offline customer service data includes multiple groups of voice conversation data and image data. The voice conversation data corresponds to the image data. By obtaining the offline voice conversation data and image data, data support is provided for the subsequent facial expression analysis, voice emotion analysis and project evaluation; each image data is identified and facial expressions are extracted, so that the quality inspection accuracy can be evaluated based on facial expressions in the future. Emotion analysis and keyword extraction are performed on each voice conversation data to obtain service items and voice emotion analysis, so that the quality inspection accuracy can be evaluated based on voice emotion analysis in the future. Emotion analysis is performed based on facial expression and voice emotion analysis to obtain fused emotion analysis, which makes the emotion analysis more accurate, thereby improving the quality inspection results. The accuracy of the fusion sentiment analysis is evaluated to obtain a first evaluation score, and the service items are evaluated using a preset evaluation standard to obtain a second evaluation score. The first evaluation score and the second evaluation score are integrated to obtain a machine quality inspection result. Automatic quality inspection can not only improve the speed of quality inspection, but also improve the accuracy of quality inspection results, so as to prepare for the subsequent accurate quality inspection results. Manual sampling is performed on each voice dialogue data and image data to obtain a manual quality inspection result, which has a high accuracy. The machine quality inspection result and the manual quality inspection result are integrated, and the service quality inspection result is obtained based on the integrated result. By integrating machine quality inspection and manual quality inspection, the diversity of quality inspection can be increased, and the accuracy of network customer service quality inspection can be improved. Therefore, the problem of low accuracy of offline quality inspection results in related technologies is effectively solved.

[0016] 2. By analyzing facial expressions and predicting facial emotion analysis, accurate facial emotion analysis can be obtained to achieve quality inspection of customer service attitude.

[0017] 3. Using probability distribution for sentiment analysis fusion can increase the expression of emotions so that accurate service quality inspection can be performed based on emotions later. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of an offline network customer service quality inspection method provided by an embodiment of the present application; Figure 2 yes Figure 1 A schematic flow chart of a sub-step of step S200; Figure 3 yes Figure 2 A schematic flow chart of a sub-step of step S210; Figure 4 yes Figure 2 A schematic flow chart of a sub-step of step S240; Figure 5 It is a structural diagram of an offline network customer service quality inspection device provided by an embodiment of the present application; Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0021] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0022] The embodiment of the present application provides an offline network customer service quality inspection method, device, electronic device and readable storage medium. The offline network customer service quality inspection method first obtains offline customer service data, and the offline customer service data includes multiple groups of voice dialogue data and image data. The voice dialogue data corresponds to the image data. By obtaining the offline voice dialogue data and image data, data support is provided for subsequent facial expression analysis, voice emotion analysis and project evaluation; each image data is identified and facial expressions are extracted so that the accuracy of quality inspection can be evaluated according to the facial expressions in the subsequent quality inspection; emotion analysis and keyword extraction are performed on each voice dialogue data respectively to obtain service items and voice emotion analysis so that the accuracy of quality inspection can be evaluated according to the voice emotion analysis in the subsequent quality inspection; emotion analysis is performed based on facial expressions and voice emotion analysis to obtain fusion emotion Analysis makes sentiment analysis more accurate, thereby improving the accuracy of quality inspection results; evaluating the integrated sentiment analysis to obtain a first evaluation score, using a preset evaluation standard to evaluate the service item to obtain a second evaluation score, fusing the first evaluation score and the second evaluation score to obtain a machine quality inspection result. Automatic quality inspection can not only improve the speed of quality inspection, but also improve the accuracy of quality inspection results, thus preparing for subsequent accurate quality inspection results; manually sampling each voice conversation data and image data to obtain a manual quality inspection result, which has a high accuracy; fusing the machine quality inspection result and the manual quality inspection result, and obtaining a service quality inspection result based on the fusion result. By fusing machine quality inspection and manual quality inspection, the diversity of quality inspection can be increased, and the accuracy of network customer service quality inspection can be improved.

[0023] It should be noted that the offline network customer service quality inspection method is mainly used for offline network customer service quality inspection, providing network consultation, after-sales service and dispute resolution at various outlets or business halls. Offline network customer service services include e-commerce, e-government, electronicization of corporate business processes and other offline customer service services.

[0024] The technical solution provided in the embodiments of the present application is further described below in conjunction with the accompanying drawings.

[0025] Reference Figure 1 , Figure 1 1 is a flow chart of an offline network customer service quality inspection method provided in an embodiment of the present application. The offline network customer service quality inspection method is applied to an offline network customer service quality inspection device, and the offline network customer service quality inspection method is executed by a processor in an electronic device or a readable storage medium. The offline network customer service quality inspection method includes steps S100, S200, S300, S400, and S500.

[0026] Step S100, obtaining offline customer service data, the offline customer service data including multiple groups of voice dialogue data and image data, the voice dialogue data corresponding to the image data.

[0027] In one embodiment, offline customer service data can be collected by collecting image data through cameras installed in offline network sites or business hall sites, and voice conversation data can be collected using voice recording equipment. When providing services, it is necessary to communicate with users, which will generate multiple groups of voice conversations. Therefore, when using voice equipment to collect voice conversation data, the camera is aimed at the service personnel to continuously record the facial images and service conversations of the service personnel, and voice conversation data and image data are collected at the same time to ensure that the voice conversation data corresponds to the image data, thereby providing data support for subsequent facial expression analysis, voice emotion analysis and project evaluation.

[0028] Step S200, recognize each image data, extract facial expressions, perform sentiment analysis and keyword extraction on each voice dialogue data, obtain service items and voice sentiment analysis, perform sentiment analysis based on facial expressions and voice sentiment analysis, and obtain fused sentiment analysis.

[0029] In one embodiment, for the collected image data, a face image is first identified, and then expression recognition is performed on the face image to obtain facial expression, so as to evaluate the accuracy of quality inspection based on facial expression later. A preset face analysis algorithm can be used to identify each image data, and the face analysis algorithm includes a face recognition algorithm and a facial expression recognition algorithm. The face recognition algorithm is used to perform face recognition on each image data to obtain a face image, and the facial expression recognition algorithm is used to perform expression recognition on the face image to obtain facial expression. Among them, the face recognition algorithm can be a module matching algorithm, an Adaboost algorithm, etc., and the facial expression recognition algorithm can be a neural network, a decision tree, etc.

[0030] It should be noted that, since service personnel are required to face the camera during data collection, it is relatively easy to extract facial images and perform expression analysis, and the accuracy of facial recognition is relatively high. In addition, service quality inspection can build a facial image library of on-the-job personnel for service personnel, which can further improve the accuracy of facial recognition and expression recognition.

[0031] In one embodiment, sentiment analysis and keyword extraction are performed on each voice conversation data to obtain service items and voice sentiment analysis, including but not limited to: converting the voice conversation data into text conversation data, performing keyword extraction on the text conversation data using preset project prompt words to obtain service items, where the project prompt words are obtained by performing keyword extraction on offline network customer service; performing sentiment analysis on the voice conversation data using preset emotion prompt words to obtain voice sentiment analysis.

[0032] Specifically, the deep learning model is used to convert the voice conversation data into text conversation data. The text conversion is conducive to the subsequent keyword extraction. Construct a preset project prompt word, which is obtained by extracting keywords from offline network customer service, summarizing customer service projects by professionals, or extracting keywords from historical offline network customer service projects using a neural network model, and then manually correcting them to obtain project prompt words. The project prompt word is used to extract keywords from text conversation data as follows: first, a recurrent neural network is used to extract keywords from text conversation data, and the extracted keywords are matched with the project prompt word. If the match is successful, the service project is obtained. It is conducive to the subsequent judgment of service quality according to the service project, and whether the voice emotion and facial emotion meet the requirements, thereby improving the accuracy of quality inspection. The preset emotional prompt word is a vocabulary set in advance according to the service project. The preset natural language processing algorithm is used to perform emotional analysis on the voice conversation data to obtain emotional prompt words, and the emotional prompt word is matched with the preset emotional prompt word. If the match is successful, the voice emotional analysis is obtained, so that the accuracy of quality inspection can be evaluated according to the voice emotional analysis.

[0033] like Figure 2 As shown, sentiment analysis is performed based on facial expression and voice sentiment analysis to obtain fusion sentiment analysis, including: Step S210, analyzing facial expressions to obtain facial emotion analysis, and integrating facial emotion analysis with voice emotion analysis to obtain initial emotion analysis.

[0034] In one embodiment, the extracted facial expression is matched with a preset facial emotion analysis, and the facial emotion analysis is obtained according to the matching result. For example, the emotional state can be obtained by matching the shape of eyebrows. When the eyebrows are furrowed, it may be annoyed or angry, so that the facial emotion can be reflected to observe the service attitude.

[0035] like Figure 3 As shown, facial emotion analysis and voice emotion analysis are integrated to obtain initial emotion analysis, including but not limited to the following steps: Step S211, establish a facial emotion node based on facial emotion analysis, establish a speech emotion node based on speech emotion analysis, and determine a first probability distribution corresponding to the facial emotion node and a second probability distribution corresponding to the speech emotion node.

[0036] In some possible embodiments of the present application, when performing facial emotion analysis, corresponding facial emotion features are obtained through step S210, and facial emotion nodes corresponding to feature points are established based on the extracted features, and in the above-mentioned voice dialogue data analysis, corresponding voice emotion features are obtained, and voice emotion nodes corresponding to feature points are established based on the extracted features. The initial probability distribution corresponding to the facial emotion nodes and the initial probability distribution corresponding to the voice emotion nodes are set based on experience, and the initial probability distribution is adjusted through a large amount of data to obtain a first probability distribution corresponding to the facial emotion nodes and a second probability distribution corresponding to the voice emotion nodes, so that a fused emotion probability distribution can be obtained later based on the first probability distribution and the second probability distribution.

[0037] Step S212, obtaining a fused emotion probability distribution according to the first probability distribution and the second probability distribution, and obtaining an initial emotion analysis according to the fused emotion probability distribution.

[0038] In some possible embodiments of the present application, the first probability distribution and the second probability distribution are calculated by a Bayesian model, and the calculated probability is a fused emotion probability distribution, which can reflect the proportion of the fused emotion in the first probability distribution and the second probability distribution, thereby obtaining a more accurate emotion analysis. The emotion analysis category is determined based on the fused emotion probability distribution to obtain an initial emotion analysis, so that a fused emotion analysis can be obtained based on the initial emotion analysis later.

[0039] Step S220, according to the progress of the service project, facial emotion prediction is performed based on facial expressions to obtain predicted facial emotions, and voice emotion prediction is performed based on voice dialogue data to obtain predicted voice emotions.

[0040] In one embodiment, according to the different stages of the service project, the service personnel may have emotional changes. The emotions of the project progress in the next stage are predicted based on the emotions of the project progress in the previous stage, which can reflect whether the subsequent service attitude meets the quality inspection requirements. Under the progress of the service project, a preset prediction algorithm is used to predict facial emotions for facial expressions over a period of history to obtain predicted facial emotions. By performing facial emotion prediction, the expression trend can be estimated, thereby obtaining a more accurate service quality inspection, wherein the prediction algorithm can be a convolutional neural network and its variants. Under the progress of the service project, a preset recurrent neural network is used to predict voice emotions for voice dialogue data over a period of history to obtain predicted voice emotions. By performing voice emotion prediction, the trend of tone, intonation, etc. can be estimated, thereby obtaining a more accurate service quality inspection.

[0041] Step S230, fusing the predicted facial emotion and the predicted voice emotion to obtain a first predicted emotion analysis.

[0042] In one embodiment, according to the process of step S211 and step S212, the predicted facial emotion and the predicted voice emotion are fused to obtain a first predicted emotion analysis, so that the emotion adjustment is subsequently performed according to the first predicted emotion analysis to obtain an accurate emotion analysis. The specific fusion process is not described in detail here.

[0043] Step S240, using the first predicted sentiment analysis to adjust the initial sentiment analysis to obtain a fused sentiment analysis.

[0044] like Figure 4 As shown, the initial sentiment analysis is adjusted using the first predicted sentiment analysis to obtain a fused sentiment analysis, including but not limited to the following steps: Step S241, predicting the initial sentiment analysis to obtain a second predicted sentiment analysis.

[0045] In some possible embodiments of the present application, the initial sentiment analysis is a sentiment analysis of the historical and current service project stages. Whether there will be a sentiment change in future service projects will also affect the quality inspection results of customer service. Therefore, a convolutional neural network or a variant of a convolutional neural network is used to predict the initial sentiment analysis to obtain a second predicted sentiment analysis, so that the second predicted sentiment analysis can be used to determine the fused sentiment analysis. It should be noted that the convolutional neural network or the variant of the convolutional neural network are all trained models, which can be used by modifying the corresponding parameters to meet the network input format.

[0046] Step S242: compare the first predicted sentiment analysis with the second predicted sentiment analysis. If the first predicted sentiment analysis is consistent with the second predicted sentiment analysis, use the initial sentiment analysis as the fused sentiment analysis.

[0047] In some possible embodiments of the present application, comparing the first predicted sentiment analysis with the second predicted sentiment analysis is specifically to match the first predicted sentiment analysis with the second predicted sentiment analysis through a character matching algorithm. When the first predicted sentiment analysis is consistent with the second predicted sentiment analysis, it indicates that the prediction results of the first predicted sentiment analysis and the second predicted sentiment analysis are the same. Taking the initial sentiment analysis as a fused sentiment analysis can accurately represent the current service attitude.

[0048] In some other possible embodiments of the present application, when the first predicted sentiment analysis is consistent with the second predicted sentiment analysis, it indicates that there may be errors in the initial sentiment analysis, and the second predicted sentiment analysis is quantified with the first predicted sentiment analysis, and the error between the first predicted sentiment analysis and the first predicted sentiment analysis is compared. When the error is less than a preset threshold, the initial sentiment analysis is used as a fusion sentiment analysis, which can accurately represent the current service attitude. When the error is greater than or equal to the preset threshold, the initial sentiment analysis is discarded and recalculated.

[0049] Step S300, evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service item using a preset evaluation standard to obtain a second evaluation score, and fuse the first evaluation score and the second evaluation score to obtain a machine quality inspection result.

[0050] In one embodiment, the fused sentiment analysis is evaluated, specifically, the fused sentiment analysis is quantified, and reference may be made to a preset sentiment analysis evaluation table, wherein the sentiment analysis evaluation table includes sentiments corresponding to different projects and different project progresses, and scores corresponding to the sentiments. By referring to the sentiment analysis evaluation table, a first evaluation score can be matched to obtain a first evaluation score, so that a machine quality inspection result can be obtained subsequently based on the first evaluation score.

[0051] In another embodiment, the evaluation criteria include service quality criteria, service efficiency criteria, and service satisfaction criteria; evaluating the service items using the preset evaluation criteria to obtain a second evaluation score includes, but is not limited to, the following steps: Step S310, using the service quality standard, service efficiency standard and service satisfaction standard to evaluate the service items respectively, to obtain a quality score corresponding to the service quality standard, an efficiency score corresponding to the service efficiency standard and a satisfaction score corresponding to the service satisfaction standard.

[0052] In some possible embodiments of the present application, the service quality standard includes multiple service items and quality scores corresponding to the service items, and the service items are matched with the service items in the service quality standard to obtain the corresponding quality scores, so that the second evaluation score can be obtained according to the quality scores later. The service efficiency standard includes multiple service items and efficiency scores corresponding to the service items, and the service items are matched with the service items in the service quality standard to obtain the corresponding efficiency scores, so that the second evaluation score can be obtained according to the efficiency scores later. The service satisfaction includes multiple service items and satisfaction scores corresponding to the service items, and the service items are matched with the service items in the service quality standard to obtain the corresponding satisfaction scores, so that the second evaluation score can be obtained according to the satisfaction scores later.

[0053] Step S320, performing weighted summation on the quality score, the efficiency score and the satisfaction score to obtain a second evaluation score.

[0054] In some possible embodiments of the present application, the quality score, efficiency score, and satisfaction score are weighted and summed, and the weights corresponding to the quality score, efficiency score, and satisfaction score are set to 1, respectively, that is, they are directly added to obtain a second evaluation score, so that the machine quality inspection result can be obtained subsequently based on the evaluation score.

[0055] In another embodiment, the first evaluation score and the second evaluation score are weighted and summed, and the first evaluation score and the second evaluation score are assigned a corresponding ratio, which can be 4:6, that is, the first evaluation score accounts for 40% and the second evaluation score accounts for 60%, or it can be 3:7. Since the second evaluation score corresponds to service quality, service efficiency and service satisfaction, it can better reflect the effect of network customer service. Therefore, the proportion of the second evaluation score is set to be greater than the proportion of the first evaluation score. By fusion, the machine quality inspection results are obtained, which can increase the multi-faceted feedback of the quality inspection results and improve the accuracy of the quality inspection results.

[0056] Step S400: Perform manual sampling inspection on each voice dialogue data and image data to obtain manual quality inspection results.

[0057] In one embodiment, each voice dialogue data and image data may be manually sampled. Manual quality inspection has high accuracy. The results of manual evaluation are recorded to obtain manual quality inspection results, which are also expressed in the form of scores for subsequent fusion calculations.

[0058] Step S500: fuse the machine quality inspection result and the manual quality inspection result, and obtain the service quality inspection result based on the fusion result.

[0059] In one embodiment, the machine quality inspection result and the manual quality inspection result are integrated, and a service quality inspection result is obtained based on the integrated result, including but not limited to the following steps: Step S510, obtaining a preset first quality inspection weight and a preset second quality inspection weight, wherein the first quality inspection weight is smaller than the second quality inspection weight.

[0060] In some possible embodiments of the present application, the preset first quality inspection weight and the preset second quality inspection weight are both set by professionals based on experience, the first quality inspection weight corresponds to the machine quality inspection result, and the second quality inspection weight corresponds to the manual quality inspection result. Since the manual quality inspection has a higher accuracy, the first quality inspection weight is set to be less than the second quality inspection weight, which can increase the accuracy of the quality inspection result. The preset first quality inspection weight and the preset second quality inspection weight can be obtained through a preset reading function to facilitate the subsequent calculation of the first result and the second result. The second quality inspection weight can also correspond to the machine quality inspection result, the first quality inspection weight corresponds to the manual quality inspection result, and the first quality inspection weight is set to be greater than the second quality inspection weight to increase the accuracy of the quality inspection result.

[0061] Step S520, multiply the first quality inspection weight and the machine quality inspection result to obtain a first result, and multiply the second quality inspection weight and the manual quality inspection result to obtain a second result.

[0062] In some possible embodiments of the present application, since the weights and quality inspection results are both in fractional form, the first quality inspection weight and the machine quality inspection result are multiplied to obtain the first result, and the second quality inspection weight and the manual quality inspection result are multiplied to obtain the second result for subsequent fusion.

[0063] Step S530, adding the first result and the second result to obtain a fusion result, and matching the fusion result with a preset service level to obtain a service quality inspection result.

[0064] In some possible embodiments of the present application, the first result and the second result are added to obtain a fusion result, and the fusion result is expressed as a score. The first result and the second result may also be redistributed with adjustment weights, which will not be described in detail here. The preset service level includes a fusion score interval and a corresponding quality inspection level. The fusion result is compared with the fusion score interval. When the fusion result is in the corresponding fusion score interval, the corresponding quality inspection level is obtained, and the quality inspection level is used as the service quality inspection result. It should be noted that the quality inspection level may include three levels, which are divided into excellent, good, medium and poor. It may also include four or five levels, which will not be described here.

[0065] like Figure 5As shown, the embodiment of the present application provides an offline network customer service quality inspection device 100, which obtains offline customer service data through a data acquisition module 110. The offline customer service data includes multiple groups of voice dialogue data and image data. The voice dialogue data corresponds to the image data. By obtaining the offline voice dialogue data and image data, data support is provided for subsequent facial expression analysis, voice emotion analysis and project evaluation; then the first data processing module 120 is used to identify each image data and extract facial expressions, so that the accuracy of quality inspection can be evaluated according to the facial expressions in the subsequent data processing. Emotion analysis and keyword extraction are performed on each voice dialogue data to obtain service items and voice emotion analysis, so that the accuracy of quality inspection can be evaluated according to the voice emotion analysis in the subsequent data processing. Emotion analysis is performed based on facial expressions and voice emotion analysis to obtain fused emotion analysis, so that the emotion analysis is more accurate. The first data fusion module 130 is used to evaluate the fused sentiment analysis to obtain a first evaluation score, and the service item is evaluated using a preset evaluation standard to obtain a second evaluation score. The first evaluation score and the second evaluation score are fused to obtain a machine quality inspection result. Automatic quality inspection can not only improve the speed of quality inspection, but also improve the accuracy of quality inspection results, so as to prepare for obtaining accurate quality inspection results in the future. The second data processing module 140 is used to perform manual sampling on each voice dialogue data and image data to obtain a manual quality inspection result, which has a high accuracy. Finally, the second data fusion module 150 is used to fuse the machine quality inspection result and the manual quality inspection result, and the service quality inspection result is obtained based on the fusion result. By fusing machine quality inspection and manual quality inspection, the diversity of quality inspection can be increased, and the accuracy of network customer service quality inspection can be improved.

[0066] It should be noted that the data acquisition module 110 is connected to the first data processing module 120, the first data processing module 120 is connected to the first data fusion module 130, the first data fusion module 130 is connected to the second data processing module 140, and the second data processing module 140 is connected to the second data fusion module 150. The above-mentioned offline network customer service quality inspection method is applied to an offline network customer service quality inspection device 100, and the offline network customer service quality inspection device 100 obtains offline customer service data, and the offline customer service data includes multiple groups of voice conversation data and image data. The voice conversation data corresponds to the image data. By obtaining offline voice conversation data and image data, data support is provided for subsequent facial expression analysis, voice emotion analysis and project evaluation; each image data is identified and facial expressions are extracted so that the quality inspection accuracy can be evaluated according to the facial expressions in the subsequent quality inspection, and emotion analysis and keyword extraction are performed on each voice conversation data to obtain service items and voice emotion analysis, so that the quality inspection accuracy can be evaluated according to the voice emotion analysis in the subsequent quality inspection, and emotion analysis is performed based on facial expressions and voice emotion analysis to obtain fused emotion analysis. Analysis makes sentiment analysis more accurate, thereby improving the accuracy of quality inspection results; evaluating the integrated sentiment analysis to obtain a first evaluation score, using a preset evaluation standard to evaluate the service item to obtain a second evaluation score, fusing the first evaluation score and the second evaluation score to obtain a machine quality inspection result. Automatic quality inspection can not only improve the speed of quality inspection, but also improve the accuracy of quality inspection results, thereby preparing for subsequent accurate quality inspection results; performing manual sampling on each voice conversation data and image data to obtain a manual quality inspection result, which has a high accuracy; fusing the machine quality inspection result with the manual quality inspection result, and obtaining a service quality inspection result based on the fusion result. By fusing machine quality inspection and manual quality inspection, the diversity of quality inspection can be increased, and the accuracy of network customer service quality inspection can be improved.

[0067] It should also be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0068] The present application also discloses an electronic device. Figure 6 , Figure 6 The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0069] The communication bus 502 is used to realize the connection and communication between these components.

[0070] The user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0071] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0072] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.

[0073] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. Refer to Figure 6 , the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of an offline network customer service quality inspection method.

[0074] exist Figure 6 In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 501 can be used to call the application program storing an offline network customer service quality inspection method in the memory 505, and when executed by one or more processors 501, the electronic device 500 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0075] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0079] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0080] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0081] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for quality inspection of offline customer service, characterized in that: The method comprises: Acquire offline customer service data, wherein the offline customer service data includes multiple groups of voice conversation data and image data, and the voice conversation data corresponds to the image data; Recognize each of the image data, extract facial expressions, perform sentiment analysis and keyword extraction on each of the voice conversation data, obtain service items and voice sentiment analysis, perform sentiment analysis based on the facial expressions and the voice sentiment analysis, and obtain fused sentiment analysis; Evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service item using a preset evaluation standard to obtain a second evaluation score, and fuse the first evaluation score and the second evaluation score to obtain a machine quality inspection result; Performing manual sampling inspection on each of the voice dialogue data and the image data to obtain manual quality inspection results; The machine quality inspection result and the manual quality inspection result are fused, and a service quality inspection result is obtained based on the fusion result.

2. The offline network customer service quality inspection method according to claim 1, characterized in that: The performing of sentiment analysis based on the facial expression and the voice sentiment analysis to obtain a fusion sentiment analysis includes: Analyzing the facial expression to obtain facial emotion analysis, and fusing the facial emotion analysis with the voice emotion analysis to obtain an initial emotion analysis; According to the progress of the service project, facial emotion prediction is performed based on the facial expression to obtain predicted facial emotion, and voice emotion prediction is performed based on the voice dialogue data to obtain predicted voice emotion; Fusing the predicted facial emotion and the predicted voice emotion to obtain a first predicted emotion analysis; The initial sentiment analysis is adjusted using the first predicted sentiment analysis to obtain the fused sentiment analysis.

3. The offline network customer service quality inspection method according to claim 2, characterized in that: The using the first predicted sentiment analysis to adjust the initial sentiment analysis to obtain the fused sentiment analysis includes: Predicting the initial sentiment analysis to obtain a second predicted sentiment analysis; The first predicted sentiment analysis is compared with the second predicted sentiment analysis, and when the first predicted sentiment analysis is consistent with the second predicted sentiment analysis, the initial sentiment analysis is used as the fused sentiment analysis.

4. The offline network customer service quality inspection method according to claim 2, characterized in that: The step of fusing the facial emotion analysis and the voice emotion analysis to obtain an initial emotion analysis includes: Establishing a facial emotion node according to the facial emotion analysis, establishing a voice emotion node according to the voice emotion analysis, and determining a first probability distribution corresponding to the facial emotion node and a second probability distribution corresponding to the voice emotion node; A fused emotion probability distribution is obtained according to the first probability distribution and the second probability distribution, and the initial emotion analysis is obtained according to the fused emotion probability distribution.

5. The offline network customer service quality inspection method according to claim 1, characterized in that: The sentiment analysis and keyword extraction are performed on each of the voice dialogue data to obtain service items and voice sentiment analysis, including: Converting the voice conversation data into text conversation data, and extracting keywords from the text conversation data using preset project prompt words to obtain the service project, wherein the project prompt words are obtained by extracting keywords from offline network customer service; The voice conversation data is subjected to sentiment analysis using preset sentiment prompt words to obtain a voice sentiment analysis.

6. The offline network customer service quality inspection method according to claim 1, characterized in that: The evaluation criteria include service quality standards, service efficiency standards and service satisfaction standards; The using of a preset evaluation standard to evaluate the service item to obtain a second evaluation score includes: The service items are evaluated respectively using the service quality standard, the service efficiency standard and the service satisfaction standard to obtain a quality score corresponding to the service quality standard, an efficiency score corresponding to the service efficiency standard and a satisfaction score corresponding to the service satisfaction standard; The quality score, the efficiency score and the satisfaction score are weightedly summed to obtain the second evaluation score.

7. The offline network customer service quality inspection method according to claim 1, characterized in that: The fusing the machine quality inspection result and the manual quality inspection result, and obtaining a service quality inspection result based on the fusion result, includes: Obtaining a preset first quality inspection weight and a preset second quality inspection weight, wherein the first quality inspection weight is less than the second quality inspection weight; Multiplying the first quality inspection weight and the machine quality inspection result to obtain a first result, and multiplying the second quality inspection weight and the manual quality inspection result to obtain a second result; The first result and the second result are added together to obtain a fusion result, and the fusion result is matched with a preset service level to obtain the service quality inspection result.

8. An offline network customer service quality inspection device, characterized in that: The device comprises: A data acquisition module (110) is used to acquire offline customer service data, wherein the offline customer service data includes multiple groups of voice dialogue data and image data, and the voice dialogue data corresponds to the image data; A first data processing module (120) is used to identify each of the image data, extract facial expressions, perform sentiment analysis and keyword extraction on each of the voice dialogue data, obtain service items and voice sentiment analysis, perform sentiment analysis based on the facial expressions and the voice sentiment analysis, and obtain fusion sentiment analysis; A first data fusion module (130) is used to evaluate the fused sentiment analysis to obtain a first evaluation score, evaluate the service item using a preset evaluation standard to obtain a second evaluation score, and fuse the first evaluation score and the second evaluation score to obtain a machine quality inspection result; A second data processing module (140) is used to perform manual sampling inspection on each of the voice dialogue data and the image data to obtain manual quality inspection results; The second data fusion module (150) is used to fuse the machine quality inspection result and the manual quality inspection result, and obtain a service quality inspection result based on the fusion result.

9. An electronic device, characterized in that: The electronic device (500) comprises a processor (501), a memory (505), a user interface (503), a communication bus (502) and a network interface (504), wherein the processor (501), the memory (505), the user interface (503) and the network interface (504) are respectively connected to the communication bus (502), the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device (500) executes the offline network customer service quality inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the offline network customer service quality inspection method according to any one of claims 1 to 7 is executed.

Images