Customer service evaluation method and device based on improved NLP model, medium and equipment

Abstract

The invention provides a customer service evaluation method and device based on an improved NLP model, a medium and equipment, and relates to the technical field of natural language processing. Through an improved NLP model EUEM based on a Transform architecture, deep learning and natural language processing are performed on historical interaction data, intelligent analysis of interaction quality and emotional tendency is realized, compared with traditional manual evaluation, an evaluation result can be given more quickly and more accurately, and labor cost is reduced; a score calculation formula is synthesized, weight parameters are adjusted, and the weights of the interaction quality and the emotional tendency in evaluation can be balanced according to actual requirements, so that the evaluation result better meets the actual requirements; the dynamic emotion influence scoring mechanism can consider real feelings of clients in different interaction stages and emotion states, so that the evaluation of emotion scores is more flexible and accurate; automatic, efficient and accurate evaluation of customer services is achieved, and therefore the customer satisfaction degree is improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a customer service evaluation method, device, medium and equipment based on an improved NLP model. Background Art

[0002] In order to provide customers with a higher level of service, it is usually necessary to evaluate the service quality of customer service. At present, customer service evaluation mainly relies on manual evaluation or automated systems based on keyword and pattern matching. Manual evaluation requires a lot of human resources, and due to individual differences, the evaluation results are often subjective and inconsistent, with high costs and low evaluation efficiency. Although automated systems based on keyword and pattern matching reduce the demand for human resources, they can usually only match simple language patterns and cannot accurately understand and judge complex language interactions and emotional factors, resulting in limited accuracy and comprehensiveness of the evaluation. Summary of the invention

[0003] The present application provides a customer service evaluation method, apparatus, medium and equipment based on an improved NLP model, which can realize automated, efficient and accurate evaluation of customer service, reduce evaluation costs, improve the consistency and comprehensiveness of evaluation, and ultimately improve customer satisfaction.

[0004] According to a first aspect of the present application, a customer service evaluation method based on an improved NLP model is provided, and the customer service evaluation method based on the improved NLP model includes: Obtain historical interaction data between customer service and users; Preprocessing the historical interaction data, wherein the preprocessing includes data cleaning, labeling, and word segmentation; Use the preprocessed historical interaction data to train the improved NLP model EUEM based on the Transformer architecture to obtain the trained EUEM model; The interaction data to be evaluated is input into the trained EUEM model, and the interaction quality score and sentiment score are output, and the comprehensive score is calculated based on these two scores.

[0005] In one embodiment, the training step of the EUEM model includes: Collecting pre-processed historical interaction data; the pre-processed interaction data has been labeled with interaction quality categories and emotion categories; The EUEM model is iteratively trained using the preprocessed historical interaction data. In each iteration, the error between the interaction quality score and sentiment score predicted by the model and the interaction quality category and sentiment category actually labeled in the training data is calculated, and the model parameters are adjusted according to the error; training is terminated until the loss function converges.

[0006] In one possible implementation, the calculation formula of the interaction quality score is: Q=∑P i S i ; where Q is the interaction quality score, P i is the probability of the i-th interaction quality category, S i is the score of the i-th interaction quality category; The calculation formula of the sentiment score is: E=∑P j S j ; where E is the sentiment score, P j is the probability of the jth emotion category, S j is the score of the jth sentiment category; Based on the interaction quality score and the sentiment score, a comprehensive score is calculated using the following formula: T=α×Q+β×E; wherein T is the comprehensive score, and α and β are weight parameters.

[0007] In one embodiment, when evaluating the sentiment score E, a dynamic sentiment impact scoring mechanism is also included, which includes the following steps: For multiple emotion categories, initialize the emotion weights separately; Determining trigger conditions for adjusting the sentiment weight according to the interaction context, wherein the interaction context includes the interaction stage, the customer's problem level, and the customer's historical response; When the triggering conditions are met, the emotional weight is adjusted dynamically and the dynamic emotional impact score is calculated; The calculation formula is: S t =E t ×C t ; S t represents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the stage of interaction, the severity of the problem, and the customer’s historical response; The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E.

[0008] According to a second aspect of the present application, a customer service evaluation device based on an improved NLP model is provided, and the customer service evaluation device based on the improved NLP model comprises: The acquisition module is used to obtain historical interaction data between customer service and users; A preprocessing module, used for preprocessing the historical interaction data, wherein the preprocessing includes data cleaning, labeling and word segmentation; The training module is used to train the improved NLP model EUEM based on the Transformer architecture using the preprocessed historical interaction data to obtain a trained EUEM model; The calculation module is used to input the interaction data to be evaluated into the trained EUEM model, output the interaction quality score and sentiment score, and calculate the comprehensive score based on these two scores.

[0009] In one embodiment, the training module includes: A collecting unit, used to collect pre-processed historical interaction data; the pre-processed interaction data has been annotated with interaction quality categories and emotion categories; The iterative training unit is used to iteratively train the EUEM model using the preprocessed historical interaction data. In each iteration, the error between the interaction quality score and sentiment score predicted by the model and the interaction quality category and sentiment category actually marked in the training data is calculated, and the model parameters are adjusted according to the error; until the loss function converges, the training is terminated.

[0010] In one possible implementation, the calculation formula of the interaction quality score is: Q=∑P i S i ; where Q is the interaction quality score, P i is the probability of the i-th interaction quality category, S i is the score of the i-th interaction quality category; The calculation formula of the sentiment score is: E=∑P j S j ; where E is the sentiment score, P j is the probability of the jth emotion category, S j is the score of the jth sentiment category; Based on the interaction quality score and the sentiment score, a comprehensive score is calculated using the following formula: T=α×Q+β×E; wherein T is the comprehensive score, and α and β are weight parameters.

[0011] In one possible implementation, the calculation module further includes a dynamic adjustment unit, which is used to: For multiple emotion categories, initialize the emotion weights separately; Determining trigger conditions for adjusting the sentiment weight according to the interaction context, wherein the interaction context includes the interaction stage, the customer's problem level, and the customer's historical response; When the triggering conditions are met, the emotional weight is adjusted dynamically and the dynamic emotional impact score is calculated; The calculation formula is: S t =E t ×C t ; S trepresents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the stage of interaction, the severity of the problem, and the customer’s historical response; The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E.

[0012] According to a third aspect of the present application, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the customer service evaluation method based on the improved NLP model described in the present application.

[0013] According to a third aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the customer service evaluation method based on the improved NLP model described in the present application.

[0014] The present application discloses a customer service evaluation method, device, medium and equipment based on an improved NLP model. Through the improved NLP model EUEM based on the Transformer architecture, deep learning and natural language processing are performed on historical interaction data to realize intelligent analysis of interaction quality and emotional tendency. Compared with traditional manual evaluation, evaluation results can be given more quickly and accurately, reducing labor costs. The comprehensive scoring calculation formula can balance the weights of interaction quality and emotional tendency in the evaluation according to actual needs by adjusting the weight parameters, so that the evaluation results are more in line with actual needs. The dynamic emotional impact scoring mechanism can take into account the real feelings of customers at different interaction stages and emotional states, making the evaluation of emotional scores more flexible and accurate. It realizes automated, efficient and accurate evaluation of customer service, thereby improving customer satisfaction.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0017] Figure 1 A flowchart of a customer service evaluation method based on an improved NLP model provided in Example 1 of the present application is shown; Figure 2 A structural diagram of a customer service evaluation device based on an improved NLP model provided in Example 2 of the present application is shown; Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present application is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0019] In order to solve the above-mentioned problems existing in the prior art, the embodiments of the present application provide a method, medium and device for evaluating customer service based on an improved NLP model, which are described in detail below.

[0020] Embodiment 1: like Figure 1 As shown, Figure 1 A flowchart of a customer service evaluation method based on an improved NLP model provided in an embodiment of the present application, the method comprising the following steps: Get historical interaction data between customer service and users.

[0021] Export historical interaction data between customer service and users from the customer service system, including text chat records, voice chat records (need to be converted to text), phone recording records, and email records.

[0022] Preprocess the historical interaction data, including data cleaning, labeling and word segmentation.

[0023] Preprocess historical interaction data. Data cleaning can remove irrelevant information and noise. Annotation processing can label the data with interaction quality and sentiment categories. Word segmentation can split text data into independent words or phrases.

[0024] The preprocessed historical interaction data is used to train the improved NLP model EUEM (Emotion Understanding and Evaluation Model) based on the Transformer architecture to obtain the trained EUEM model.

[0025] Collect preprocessed historical interaction data and ensure that the data is labeled with interaction quality categories (such as excellent, good, average, poor) and sentiment categories (such as satisfied, neutral, dissatisfied, angry).

[0026] The improved NLP model EUEM based on the Transformer architecture is iteratively trained using preprocessed historical interaction data. In each iteration, the model predicts the interaction quality score and sentiment score, and compares them with the interaction quality category and sentiment category actually marked in the training data to calculate the error.

[0027] Adjust model parameters according to the error to optimize model performance. Repeat iterative training until the loss function converges, that is, the model performance no longer improves significantly, and then end the training.

[0028] The EUEM model can capture deep semantic information in text and accurately identify the interaction quality and sentiment categories between customer service and users. Through iterative training and optimization, the model performance is continuously improved until the loss function converges, thus ensuring the accuracy and objectivity of the evaluation of interaction data. Compared with traditional manual evaluation, this evaluation method based on big data and machine learning can give evaluation results faster and more accurately, providing strong support for improving customer service levels.

[0029] The interaction data to be evaluated is input into the trained EUEM model, and the interaction quality score and sentiment score are output, and the comprehensive score is calculated based on these two scores.

[0030] For the interaction data to be evaluated, it is input into the trained EUEM model, and the model will output the interaction quality score and sentiment score.

[0031] The interaction quality score reflects the fluency, efficiency, and accuracy of the customer service interaction process, while the emotion score reflects the customer service staff's emotional processing ability and user satisfaction.

[0032] First, the model converts the text into word vectors, then performs information fusion and abstraction through a series of Transformer layers, and finally outputs the probability of each interaction quality category through the fully connected layer and the Softmax layer. The calculation formula for the interaction quality score is: Q=∑P i S i , where P i is the probability of the i-th interaction quality category, Si is the score of the i-th interaction quality category and can be set according to actual needs.

[0033] The model also converts text into word vectors, then passes through another series of Transformer layers for information fusion and abstraction, and finally outputs the probability of each sentiment category through a fully connected layer and a Softmax layer. The formula for calculating the sentiment score is: E=∑P j S j , where P j is the probability of the jth emotion category, S j is the score of the jth emotion category, and the score of the emotion category can also be set according to actual needs.

[0034] Based on the interaction quality score and sentiment score, the comprehensive score is calculated using the formula: T = α × Q + β × E, where T is the comprehensive score and α and β are weight parameters (which can be adjusted according to actual needs).

[0035] Through quantitative evaluation, the influence of subjective factors on the evaluation results can be eliminated, making the evaluation more objective and fair. The calculation formulas for the interaction quality score and the emotional score take into account the probabilities and scores of different interaction quality categories and emotional categories, and can more comprehensively reflect the interaction between customer service and users. The comprehensive score calculation formula can balance the weights of interaction quality and emotional tendency in the evaluation according to actual needs by adjusting the weight parameters, so that the evaluation results are more in line with actual needs.

[0036] The same emotion expressed by customers at different stages of an interaction may have different influences depending on the context. For example, suppose a customer has multiple interactions with customer service after purchasing a new TV. In the first interaction, the customer expressed an emotion of "unsatisfied" due to a delivery problem with the TV. Specifically, the customer was expecting to receive the TV before the weekend so that he could watch the weekend sports events, but the delivery was delayed and the customer was unable to receive the TV within the expected time. In this case, the customer's "unsatisfied" emotion has a significant negative impact on the evaluation of the quality of the customer service interaction because it is directly related to the customer's shopping experience and expectations.

[0037] However, in the second interaction, the customer also expressed the emotion of "dissatisfaction", but this time the reason had nothing to do with the quality or delivery of the TV. It turned out that the customer found some functional designs that he didn't like while watching the TV and expressed his dissatisfaction when communicating with the customer service. Although the customer used the word "dissatisfied" again, this emotional expression had a relatively small impact on the evaluation of the quality of the customer service interaction. This is because the dissatisfaction this time was more based on personal preferences and subjective feelings, rather than mistakes or shortcomings in the customer service in providing services.

[0038] More importantly, after the customer service learned the context of the customer's second expression of "dissatisfaction", they adopted a positive communication strategy, explained the functional design principles of the TV to the customer, and provided some usage suggestions. Through effective communication, the customer service not only alleviated the customer's dissatisfaction, but also enhanced the customer's trust and loyalty to the brand.

[0039] This example demonstrates the necessity of dynamically adjusting sentiment weights based on the interaction context. In the first interaction, the customer’s “dissatisfied” sentiment directly reflects the inadequacy of the delivery service, which has a significant negative impact on the evaluation of the customer service interaction quality. In the second interaction, although the customer also expressed the sentiment of “dissatisfied”, the influence of this sentiment expression was relatively small due to the different context (i.e., the reason for dissatisfaction was not related to the customer service). Through effective communication and explanation, customer service can even transform the customer’s dissatisfaction into a positive brand experience.

[0040] Therefore, in customer service evaluation, we need to comprehensively consider the different meanings and influences of the same emotions expressed by customers at different stages of interaction to ensure the accuracy and fairness of the evaluation results.

[0041] In this regard, the embodiment of the present application introduces a dynamic emotion impact scoring mechanism when evaluating the emotion score E. First, for multiple emotion categories (such as satisfaction, neutrality, dissatisfaction, anger), the emotion weights are initialized respectively.

[0042] The triggering conditions for sentiment weight adjustment are determined based on the interaction context (including interaction stage, customer problem level, customer historical response, etc.). For example, when a customer expresses "dissatisfied" sentiment in the feedback stage after the problem is solved and the problem level is high, it may trigger a weight adjustment.

[0043] When the triggering conditions are met, the sentiment weight is adjusted dynamically and the dynamic sentiment impact score is calculated.

[0044] The calculation formula is: t =E t ×C t , where S t represents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the interaction stage, problem severity, and customer’s historical response.

[0045] The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E. The emotional score E calculated after dynamic weight adjustment can more truly reflect the customer's emotions and improve the accuracy of customer service evaluation.

[0046] The dynamic emotional impact scoring mechanism can take into account the real feelings of customers at different stages of interaction and emotional states, making the evaluation of emotional scores more flexible and accurate. By determining the trigger conditions for adjusting the emotional weight based on factors such as the interaction stage, customer problem level, and customer historical response, and calculating the dynamic emotional impact score, it can more comprehensively reflect the emotional changes of customers during the interaction process. This dynamic and flexible evaluation method can better adapt to the needs of different customers and different interaction scenarios, and provide a strong guarantee for improving customer service experience.

[0047] Through the customer service evaluation method based on the improved NLP model provided in Example 1 of the present application, it is possible to realize intelligent analysis of the interaction data between customer service and users, and accurately evaluate the interaction quality and emotional tendency. This method not only improves the accuracy and objectivity of customer service evaluation, but also provides strong data support for improving customer service levels. At the same time, the introduction of a dynamic emotional impact scoring mechanism makes the evaluation of emotional scores more flexible and accurate, and can better reflect the true feelings of customers at different stages of interaction and emotional states.

[0048] Embodiment 2: like Figure 2 As shown, Figure 2 A structural diagram of a customer service evaluation device based on an improved NLP model provided in an embodiment of the present application, the device comprising: The acquisition module is used to obtain historical interaction data between customer service and users; The preprocessing module is used to preprocess the historical interaction data, including data cleaning, labeling and word segmentation; The training module is used to train the improved NLP model EUEM based on the Transformer architecture using the preprocessed historical interaction data to obtain a trained EUEM model; The calculation module is used to input the interaction data to be evaluated into the trained EUEM model, output the interaction quality score and sentiment score, and calculate the comprehensive score based on these two scores.

[0049] Specifically, the training modules include: A collection unit, used to collect pre-processed historical interaction data; the pre-processed interaction data has been annotated with interaction quality categories and sentiment categories; The iterative training unit is used to iteratively train the EUEM model using the preprocessed historical interaction data. In each iteration, the error between the interaction quality score and sentiment score predicted by the model and the interaction quality category and sentiment category actually marked in the training data is calculated, and the model parameters are adjusted according to the error; until the loss function converges, the training is terminated.

[0050] Specifically, the calculation formula for the interaction quality score is: Q=∑P i S i ; where Q is the interaction quality score, P i is the probability of the i-th interaction quality category, S i is the score of the i-th interaction quality category; The calculation formula of sentiment score is: E=∑P j S j ; where E is the sentiment score, P j is the probability of the jth emotion category, S j is the score of the jth sentiment category; Based on the interaction quality score and the sentiment score, the comprehensive score is calculated using the formula: T = α × Q + β × E; where T is the comprehensive score, and α and β are weight parameters.

[0051] Specifically, the calculation module also includes a dynamic adjustment unit, which is used to: For multiple emotion categories, initialize the emotion weights separately; Determine the triggering conditions for adjusting the sentiment weight based on the interaction context, which includes the interaction stage, customer problem level, and customer historical response; When the triggering conditions are met, the emotional weight is adjusted dynamically and the dynamic emotional impact score is calculated; The calculation formula is: S t =E t ×C t ; S t represents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the stage of interaction, the severity of the problem, and the customer’s historical response; The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E.

[0052] Through the customer service evaluation method based on the improved NLP model provided in Example 2 of this application, it is possible to realize intelligent analysis of the interaction data between customer service and users, and accurately evaluate the interaction quality and emotional tendency. This method not only improves the accuracy and objectivity of customer service evaluation, but also provides strong data support for improving customer service levels. At the same time, the introduction of a dynamic emotional impact scoring mechanism makes the evaluation of emotional scores more flexible and accurate, and can better reflect the true feelings of customers at different stages of interaction and emotional states.

[0053] Embodiment three: According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0054] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0055] like Figure 3 As shown, the device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0056] A number of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0057] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as a customer service evaluation method based on an improved NLP model. For example, in some embodiments, the customer service evaluation method based on the improved NLP model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the customer service evaluation method based on the improved NLP model described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the customer service evaluation method based on the improved NLP model in any other appropriate manner (eg, by means of firmware).

[0058] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0059] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0060] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0061] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0062] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0063] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0064] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0065] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0066] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A customer service evaluation method based on an improved NLP model, characterized in that: The customer service evaluation method based on the improved NLP model includes: Obtain historical interaction data between customer service and users; Preprocessing the historical interaction data, wherein the preprocessing includes data cleaning, labeling, and word segmentation; Use the preprocessed historical interaction data to train the improved NLP model EUEM based on the Transformer architecture to obtain the trained EUEM model; The interaction data to be evaluated is input into the trained EUEM model, and the interaction quality score and sentiment score are output, and the comprehensive score is calculated based on these two scores.

2. The customer service evaluation method based on the improved NLP model according to claim 1 is characterized in that: The training steps of the EUEM model include: Collecting pre-processed historical interaction data; the pre-processed interaction data has been labeled with interaction quality categories and emotion categories; The EUEM model is iteratively trained using the preprocessed historical interaction data. In each iteration, the error between the interaction quality score and sentiment score predicted by the model and the interaction quality category and sentiment category actually labeled in the training data is calculated, and the model parameters are adjusted according to the error; training is terminated until the loss function converges.

3. The customer service evaluation method based on the improved NLP model according to claim 2 is characterized in that: The calculation formula of the interaction quality score is: Q=∑P i S i ; where Q is the interaction quality score, P i is the probability of the i-th interaction quality category, S i is the score of the i-th interaction quality category; The calculation formula of the sentiment score is: E=∑P j S j ; where E is the sentiment score, P j is the probability of the jth emotion category, S j is the score of the jth sentiment category; Based on the interaction quality score and the sentiment score, a comprehensive score is calculated using the following formula: T=α×Q+β×E; wherein T is the comprehensive score, and α and β are weight parameters.

4. The customer service evaluation method based on the improved NLP model according to claim 3 is characterized in that: When evaluating the sentiment score E, a dynamic sentiment impact scoring mechanism is also included, which includes the following steps: For multiple emotion categories, initialize the emotion weights separately; Determining trigger conditions for adjusting the sentiment weight according to the interaction context, wherein the interaction context includes the interaction stage, the customer's problem level, and the customer's historical response; When the triggering conditions are met, the emotional weight is adjusted dynamically and the dynamic emotional impact score is calculated; The calculation formula is: S t =E t ×C t ; S t represents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the stage of interaction, the severity of the problem, and the customer’s historical response; The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E.

5. A customer service evaluation device based on an improved NLP model, characterized in that: The improved NLP model-based customer service evaluation device comprises: The acquisition module is used to obtain historical interaction data between customer service and users; A preprocessing module, used for preprocessing the historical interaction data, wherein the preprocessing includes data cleaning, labeling and word segmentation; The training module is used to train the improved NLP model EUEM based on the Transformer architecture using the preprocessed historical interaction data to obtain a trained EUEM model; The calculation module is used to input the interaction data to be evaluated into the trained EUEM model, output the interaction quality score and sentiment score, and calculate the comprehensive score based on these two scores.

6. The customer service evaluation device based on the improved NLP model according to claim 5 is characterized in that: The training module includes: A collecting unit, used to collect pre-processed historical interaction data; the pre-processed interaction data has been annotated with interaction quality categories and emotion categories; The iterative training unit is used to iteratively train the EUEM model using the preprocessed historical interaction data. In each iteration, the error between the interaction quality score and sentiment score predicted by the model and the interaction quality category and sentiment category actually marked in the training data is calculated, and the model parameters are adjusted according to the error; until the loss function converges, the training is terminated.

7. The customer service evaluation device based on the improved NLP model according to claim 6 is characterized in that: The calculation formula of the interaction quality score is: Q=∑P i S i ; where Q is the interaction quality score, P i is the probability of the i-th interaction quality category, S i is the score of the i-th interaction quality category; The calculation formula of the sentiment score is: E=∑P j S j ; where E is the sentiment score, P j is the probability of the jth emotion category, S j is the score of the jth sentiment category; Based on the interaction quality score and the sentiment score, a comprehensive score is calculated using the following formula: T=α×Q+β×E; wherein T is the comprehensive score, and α and β are weight parameters.

8. The customer service evaluation device based on the improved NLP model according to claim 7 is characterized in that: The calculation module also includes a dynamic adjustment unit, which is used to: For multiple emotion categories, initialize the emotion weights separately; Determining trigger conditions for adjusting the sentiment weight according to the interaction context, wherein the interaction context includes the interaction stage, the customer's problem level, and the customer's historical response; When the triggering conditions are met, the emotional weight is adjusted dynamically and the dynamic emotional impact score is calculated; The calculation formula is: S t =E t ×C t ; S t represents the dynamic emotional impact score, E t is the sentiment score at time t, C t is a context-based weighting factor, C t It is determined by the stage of interaction, the severity of the problem, and the customer’s historical response; The dynamic emotional impact scores at each time t are weighted averaged to obtain the emotional score E.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images