Service evaluation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a service evaluation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the following steps: obtaining the current working process record of the current customer service work of a first user, and determining the type of a first problem requested to be solved; setting a service result corresponding to the current customer service work as a first preset value for indicating problem solving; searching first customer service work serving the first user and requesting to solve the first problem type, and setting a service result corresponding to the first customer service work as a second preset value indicating that the problem is not solved; and obtaining the number of customer service workers participating in the current evaluation period and the number of all customer service workers participating in the current evaluation period, and determining customer service work quality scores corresponding to the customer service workers, wherein the service result of the customer service workers is a first preset value, and obtaining the number of all the customer service workers participating in the current evaluation period. By applying the technical scheme provided by the embodiment of the invention, service quality evaluation is carried out on the customer service work participated by the customer service staff.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a service evaluation method, device, electronic device and storage medium. Background Art

[0002] The content of customer service work is to handle various user issues in order to maintain a good relationship between the enterprise and the user, so that the user can feel the attention and care of the enterprise. For example, the enterprise can solve the user's problem through customer service work, and even if the user has obvious negative emotions, it can also appease the user's emotions, thereby improving user satisfaction. Therefore, it is necessary to evaluate the service quality of the customer service work in which the customer service personnel participate. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a service evaluation method, device, electronic device and storage medium to evaluate the service quality of customer service work participated in by customer service personnel. The specific technical solution is as follows:

[0004] In a first aspect of the implementation of the present application, a service evaluation method is first provided, the method comprising:

[0005] Obtain the current work process record of the current customer service work serving the first user;

[0006] Determine, according to the current work process record, the first problem type requested to be solved in the current customer service work;

[0007] After the current customer service work is completed, setting the service result corresponding to the current customer service work to a first preset value, wherein the first preset value indicates that the problem requested to be solved in the customer service work has been solved;

[0008] Retrieving, from all customer service jobs that have been completed in the current evaluation cycle, a first customer service job that serves the first user and requests a solution to the first problem type;

[0009] If the first customer service work is retrieved, setting the service result corresponding to the first customer service work to a second preset value, wherein the second preset value indicates that the problem requested to be solved in the customer service work is not solved;

[0010] After the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value is obtained during the current evaluation cycle, and a second number of all customer service tasks in which the target customer service personnel participated during the current evaluation cycle is obtained;

[0011] A customer service work quality score corresponding to the target customer service personnel is determined based on the first number and the second number.

[0012] In some embodiments, the step of determining the first problem type requested to be solved in the current customer service work according to the current work process record includes:

[0013] Matching the current work process record with the sample work process records included in the sample library to obtain a first sample work process record with the highest similarity to the current work process record;

[0014] The problem type marked in the first sample work process record is extracted as the first problem type requested to be solved in the current customer service work.

[0015] In some embodiments, the step of determining the first problem type requested to be solved in the current customer service work according to the current work process record includes:

[0016] Matching the current work process record with sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0017] The problem types marked in the current work process record and the preset number of second sample work process records are input into the language model to obtain the first problem type requested to be solved in the current customer service work.

[0018] In some embodiments, the step of determining the first problem type requested to be solved in the current customer service work according to the current work process record includes:

[0019] Matching the current work process record with sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0020] Inputting the question types marked by the current work process record and the preset number of second sample work process records into the language macro model to obtain the similarity between the second question type and the second question type;

[0021] If the similarity of the second question type is greater than the similarity of the third sample work process record, the second question type is used as the first question type requested to be solved in the current customer service work, and the third sample work process record is used as the second sample work process record with the highest similarity;

[0022] If the similarity of the second problem type is less than or equal to the similarity of the third sample work process record, the problem type marked in the third sample work process record is used as the first problem type requested to be solved in the current customer service work.

[0023] In some embodiments, the step of matching the current work process record with the sample work process record included in the sample library includes:

[0024] Converting the current work process record into text data;

[0025] Performing vectorization processing on the text data to obtain a current feature vector corresponding to the text data;

[0026] The current feature vector is matched with a sample feature vector corresponding to a sample work process record included in the sample library.

[0027] In some embodiments, the first question type includes a first question main type and a first question subtype;

[0028] The step of retrieving a first customer service job that serves the first user and requests to solve the first problem type includes:

[0029] A first customer service job that serves the first user and requests a solution to the first problem subtype under the first problem main type is retrieved.

[0030] In some embodiments, the method further comprises:

[0031] receiving a modification operation instruction, wherein the modification operation instruction includes a third problem type requested to be solved in a second customer service work serving a second user;

[0032] Modify the fourth problem type requested to be solved in the second customer service work to the third problem type;

[0033] Retrieving a third customer service job that serves the second user and requests to solve the third problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0034] If the third customer service task is retrieved, the service result corresponding to the third customer service task is updated.

[0035] In some embodiments, the method further comprises:

[0036] Retrieving a fourth customer service job that serves the second user and requests resolution of a fourth problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0037] If the fourth customer service task is retrieved, the service result corresponding to the fourth customer service task is updated.

[0038] In a second aspect of the implementation of the present application, a service evaluation device is also provided, the device comprising:

[0039] A first acquisition module is used to acquire a current work process record of a current customer service work serving a first user;

[0040] A first obtaining module is used to determine the first problem type requested to be solved in the current customer service work according to the current work process record;

[0041] A first setting module is used to set the service result corresponding to the current customer service work to a first preset value after the current customer service work is completed, and the first preset value indicates that the problem requested to be solved in the customer service work has been solved;

[0042] A retrieval module, configured to retrieve, from among all customer service jobs that have been completed in a current evaluation cycle, a first customer service job that serves the first user and requests a solution to the first problem type;

[0043] A second setting module is used to set the service result corresponding to the first customer service work to a second preset value if the first customer service work is retrieved, and the second preset value indicates that the problem requested to be solved in the customer service work is not solved;

[0044] A second acquisition module is used to acquire, after the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value in the current evaluation cycle, and to acquire a second number of all customer service tasks in which the target customer service personnel participated in the current evaluation cycle;

[0045] The second obtaining module is used to determine the customer service work quality score corresponding to the target customer service personnel according to the first number and the second number.

[0046] In some embodiments, the first obtaining module is specifically used to:

[0047] Matching the current work process record with the sample work process records included in the sample library to obtain a first sample work process record with the highest similarity to the current work process record;

[0048] The problem type marked in the first sample work process record is extracted as the first problem type requested to be solved in the current customer service work.

[0049] In some embodiments, the first obtaining module is specifically used to:

[0050] Matching the current work process record with sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0051] The problem types marked in the current work process record and the preset number of second sample work process records are input into the language model to obtain the first problem type requested to be solved in the current customer service work.

[0052] In some embodiments, the first obtaining module is specifically used to:

[0053] Matching the current work process record with sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0054] Inputting the question types marked by the current work process record and the preset number of second sample work process records into the language macro model to obtain the similarity between the second question type and the second question type;

[0055] If the similarity of the second question type is greater than the similarity of the third sample work process record, the second question type is used as the first question type requested to be solved in the current customer service work, and the third sample work process record is used as the second sample work process record with the highest similarity;

[0056] If the similarity of the second problem type is less than or equal to the similarity of the third sample work process record, the problem type marked in the third sample work process record is used as the first problem type requested to be solved in the current customer service work.

[0057] In some embodiments, the first obtaining module is specifically used to:

[0058] Converting the current work process record into text data;

[0059] Performing vectorization processing on the text data to obtain a current feature vector corresponding to the text data;

[0060] The current feature vector is matched with a sample feature vector corresponding to a sample work process record included in the sample library.

[0061] In some embodiments, the first question type includes a first question main type and a first question subtype; the retrieval module is specifically used to:

[0062] A first customer service job that serves the first user and requests a solution to the first problem subtype under the first problem main type is retrieved.

[0063] In some embodiments, the apparatus further comprises: a modification module; the modification module is configured to:

[0064] receiving a modification operation instruction, wherein the modification operation instruction includes a third problem type requested to be solved in a second customer service work serving a second user;

[0065] Modify the fourth problem type requested to be solved in the second customer service work to the third problem type;

[0066] Retrieving a third customer service job that serves the second user and requests to solve the third problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0067] If the third customer service task is retrieved, the service result corresponding to the third customer service task is updated.

[0068] In some embodiments, the modification module is further used to:

[0069] Retrieving a fourth customer service job that serves the second user and requests resolution of a fourth problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0070] If the fourth customer service task is retrieved, the service result corresponding to the fourth customer service task is updated.

[0071] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned service evaluation methods is implemented.

[0072] In another aspect of the implementation of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned service evaluation methods.

[0073] In the technical solution provided in the embodiment of the present application, the type of problem that the current customer service work requests to solve is determined based on the current work process record of the current customer service work. After the current customer service work is completed, the service result corresponding to the current customer service work indicating whether the user problem is solved is set, and the service result corresponding to the first customer service work that serves the same user and solves the same problem is updated. By counting the first number of customer service work that solves user problems and the second number of all customer service work that the target customer service personnel participate in during the evaluation period, the quality of customer service work provided by the customer service personnel is evaluated based on the first number and the second number. Through the technical solution provided in the embodiment of the present application, the service quality of the customer service work of the customer service personnel is evaluated based on the service results, and then the quality of the customer service work of the customer service personnel is evaluated, which reflects the working ability of the customer service personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0075] Figure 1 A first flow chart of the service evaluation method provided in the embodiment of the present application;

[0076] Figure 2 A first detailed schematic diagram of step S12 provided in an embodiment of the present application;

[0077] Figure 3 A second detailed schematic diagram of step S12 provided in an embodiment of the present application;

[0078] Figure 4 A third detailed schematic diagram of step S12 provided in an embodiment of the present application;

[0079] Figure 5 A schematic diagram of a first process for modifying a service result provided in an embodiment of the present application;

[0080] Figure 6 A second flow chart of modifying service results provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of a service evaluation provided in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of setting a work order provided in an embodiment of the present application;

[0083] Fig. 9 A schematic diagram of modifying one-time solution information provided in an embodiment of the present application;

[0084] Fig.10 A schematic diagram of the structure of a service evaluation device provided in an embodiment of the present application;

[0085] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0087] The content of customer service work is to handle various user problems in order to maintain a good relationship between the enterprise and the user, so that the user can feel the attention and care of the enterprise. For example, the enterprise can solve the user's problem through customer service work, and even if the user has obvious negative emotions, it can also appease the user's emotions, thereby improving user satisfaction. Therefore, it is necessary to evaluate the service quality of the customer service work process of customer service personnel.

[0088] In order to solve the above problems, an embodiment of the present application provides a service evaluation method, which is applied to electronic devices such as computers and servers. For the convenience of description, the electronic devices are used as the execution subjects in the following, and it does not serve as a limitation.

[0089] See also Figure 1 , Figure 1 A first flow chart of a service evaluation method provided in an embodiment of the present application is applied to an electronic device. The service evaluation method includes the following steps.

[0090] Step S11, obtaining a current work process record of the current customer service work serving the first user.

[0091] Step S12, determining the first problem type requested to be solved in the current customer service work according to the current work process record.

[0092] Step S13, after the current customer service work is completed, the service result corresponding to the current customer service work is set to a first preset value, and the first preset value indicates that the problem requested to be solved in the customer service work has been solved.

[0093] Step S14, searching for a first customer service job that serves a first user and requests a solution to a first problem type from among all customer service jobs that have been completed in the current evaluation cycle.

[0094] Step S15: If the first customer service work is retrieved, the service result corresponding to the first customer service work is set to a second preset value, and the second preset value indicates that the problem requested to be solved in the customer service work is not solved.

[0095] Step S16, after the current evaluation cycle ends, obtain a first number of customer service tasks in which the target customer service personnel participate and whose service results are a first preset value during the current evaluation cycle, and obtain a second number of all customer service tasks in which the target customer service personnel participate during the current evaluation cycle.

[0096] Step S17: Determine the customer service work quality score corresponding to the target customer service personnel according to the first quantity and the second quantity.

[0097] In the technical solution provided in the embodiment of the present application, the type of problem that the current customer service work requests to solve is determined based on the current work process record of the current customer service work. After the current customer service work is completed, the service result corresponding to the current customer service work indicating whether the user problem is solved is set, and the service result corresponding to the first customer service work that serves the same user and solves the same problem is updated. By counting the first number of customer service work that solves user problems and the second number of all customer service work that the target customer service personnel participate in during the evaluation period, the quality of customer service work provided by the customer service personnel is evaluated based on the first number and the second number. Through the technical solution provided in the embodiment of the present application, the service quality of the customer service work of the customer service personnel is evaluated based on the service results, and then the quality of the customer service work of the customer service personnel is evaluated, which reflects the working ability of the customer service personnel.

[0098] In the above step S11, the customer service personnel provide services to users during the customer service work process. The current customer service work is the customer service work currently being performed. The customer service personnel participating in the current customer service work (such as the first customer service personnel) provide services to users (such as the first user) during the current customer service work process.

[0099] The work process record is a record of the services provided by customer service personnel to users during the customer service work process. The work process record may include the conversation content between the customer service personnel and the user and the operation content of the customer service personnel, etc. The work process record may also only include the conversation content or the operation content. The following explanation will be given using the work process record including the conversation content and the operation content as an example, which does not serve as a limitation.

[0100] The current work process record is a service record of the first customer service staff providing services to the first user during the current customer service work process. The current work process record includes the current conversation content between the first customer service staff and the first user and the current operation content of the first customer service staff.

[0101] In an embodiment of the present application, during the current customer service work process, the electronic device can obtain multiple conversation records between the first customer service staff and the first user through a collection device (such as a first collection device) as the current conversation content. The first collection device can be a terminal device for the first customer service staff to have a conversation with the first user, such as a mobile phone device, a computer device, etc. The electronic device and the first collection device can also be the same device, in which case the electronic device can obtain the current conversation content by itself. The first collection device is not limited here.

[0102] The first customer service staff can have an online conversation, a real-time voice conversation, a video conversation, etc. with the first user. Accordingly, the conversation content obtained by the electronic device may be an online conversation content, a real-time voice conversation content, a video conversation content, etc. The manner in which the first customer service staff has a conversation with the first user and the form of the conversation content are not limited herein.

[0103] In an embodiment of the present application, during the current customer service work process, the electronic device can obtain multiple operation records performed by the first customer service staff through a collection device (such as a second collection device) as the current operation content. The second collection device can be a terminal device operated by the first customer service staff, such as a mobile phone device, a computer device, etc. The first collection device and the second collection device can be the same or different. The electronic device and the second collection device can also be the same device, in which case the electronic device can obtain the current operation content by itself. The second collection device is not limited here.

[0104] In the above step S12, the first problem type is the type of problem that the first user requests to be solved during the current customer service work process. During the current customer service work process, the electronic device analyzes the current work process record to determine the first problem type.

[0105] In the embodiment of the present application, the electronic device may execute step S12 after acquiring each conversation record between the first customer service staff and the first user and / or each operation record of the first customer service staff, determine the first question type once, update the first question type, and improve the accuracy of determining the first question type. The electronic device may also execute step S12 after acquiring all conversation records and / or all operation records to determine the first question type. The time for the electronic device to determine the first question type is not limited here.

[0106] In an embodiment of the present application, the electronic device may also obtain service description information, extract the problem type from the problem type field in the service description information, and use it as the first problem type. The service description information may be a service document, a service ticket, etc. The service description information may include the following content: problem type, problem description, and solution description, etc. The service description information may be filled in by the customer service staff according to the content of the conversation, or it may be generated by an artificial intelligence (AI) model based on the content of the conversation. The form of the service description information, the content included, and the generation method are not limited here.

[0107] In the above step S13, the electronic device may determine that the current customer service work has ended after determining that the conversation between the first customer service staff and the first user has ended. For example, the electronic device may perform natural language analysis (NLP) on the acquired current conversation content, and determine that the conversation has ended when determining that the current conversation content includes a closing statement. The manner in which the electronic device determines that the conversation has ended is not limited herein.

[0108] The electronic device may also determine that the current customer service work has ended after determining that the operation of the first customer service staff has ended. For example, the electronic device may receive a service end instruction sent by the terminal device operated by the first customer service staff to determine that the current customer service work has ended. The manner in which the electronic device determines that the operation of the first customer service staff has ended is not limited herein.

[0109] The electronic device may also determine that the current customer service work has ended after obtaining the service description information. The end time of the current customer service work is not limited here.

[0110] In the embodiment of the present application, the service result indicates the resolution of the user's problem in the corresponding customer service work. The service result can be a first preset value / a second preset value, indicating that the problem requested by the user in the customer service work has been resolved / unresolved. The first preset value can be 1, and the second preset value can be 0, or the first preset value can be 0, and the second preset value can be 1. The first preset value and the second preset value are not limited here.

[0111] After the current customer service work is completed, the electronic device can determine that the problem requested by the first user in the current customer service work has been solved, and set the corresponding service result to the first preset value.

[0112] In the embodiment of the present application, the electronic device can create the current customer service work when the current customer service work starts, and record relevant information in the current customer service work. For example, the electronic device can record the user ID of the first user, the current work process record, the first problem type, the service description information and the service result in the current customer service work, and the user ID and the service result can be recorded in the service description information. The content recorded in the customer service work is not limited here.

[0113] In the above step S14, the electronic device can evaluate the customer service work quality of the customer service personnel according to the evaluation cycle, and the current evaluation cycle is the evaluation cycle of the current customer service work. The duration of the evaluation cycle can be 1 day, 1 week, 1 month, etc., and the duration of the evaluation cycle is not limited here.

[0114] The first customer service work is all the customer service work that ends within the time period from the start time of the current evaluation cycle to the end time of the current customer service work (such as the first time period), and it serves the same users (that is, the service users are all the first users) and solves the same problems (that is, the problem types are all the first problem type) as the current customer service work.

[0115] In the embodiment of the present application, the electronic device retrieves the first customer service work from all the customer service work that ends within the first time period. The electronic device may first retrieve the customer service work whose user identifier is the user identifier of the first user from all the customer service work that ends within the first time period, and then retrieve the customer service work whose problem type is the first problem type from the retrieved customer service work as the first customer service work. The electronic device may also first retrieve the customer service work whose problem type is the first problem type, and then retrieve the customer service work whose user identifier is the first user identifier. The manner in which the electronic device retrieves the first customer service work is not limited herein.

[0116] In an embodiment of the present application, the category to which the first customer service staff belongs can also be recorded in the current customer service work, such as the category to which the first customer service staff belongs can be recorded in the service description information. The category represents the skill group information of the customer service staff. Furthermore, the electronic device can retrieve the first customer service work based on the category to which the customer service staff belongs. That is, the electronic device can first retrieve the customer service work with the user identifier being the first user identifier, then retrieve the customer service work with the customer service staff belonging to the category of the first customer service staff belonging to, and then retrieve the customer service work with the problem type being the first problem type, and the order in which the electronic device searches based on the user identifier, the category to which the customer service staff belongs, and the problem type is not limited. The category to which the first customer service staff belongs can be recorded

[0117] In the above step S15, after the electronic device retrieves the first customer service work, since the current customer service work is generated to solve the same problem for the same user, it can be determined that the problem requested by the first user in the first customer service work is not solved, and the service result corresponding to the first customer service work is set to the second preset value.

[0118] In the embodiment of the present application, if the electronic device does not retrieve the first customer service job, no processing is performed.

[0119] In the above step S16, the target customer service personnel is any customer service personnel, and the first number represents the number of times the target customer service personnel solves the user's problems in the current evaluation period.

[0120] At the end of the current evaluation cycle, the electronic device determines all customer service tasks in which the target customer service personnel participate in the current evaluation cycle, and counts the number of customer service tasks with service results of the first preset value among all customer service tasks as the first number. The electronic device also obtains the number of all customer service tasks as the second number.

[0121] In the above step S17, the electronic device can evaluate the work quality of the target customer service personnel according to the first quantity and the second quantity, and calculate the corresponding customer service work quality score. In the embodiment of the present application, the electronic device can calculate the ratio of the first quantity to the second quantity, and use the calculated ratio as the customer service work quality score corresponding to the target customer service personnel. The electronic device can also convert the first quantity and the second quantity into corresponding scores according to the pre-set correspondence between the first quantity and the second quantity and the score, and obtain the customer service work quality score corresponding to the target customer service personnel. The manner in which the electronic device determines the customer service work quality score according to the first quantity and the second quantity is not limited here.

[0122] In some embodiments, see Figure 2 , which is a first detailed schematic diagram of step S12 provided in the embodiment of the present application. The above step S12 may include the following steps.

[0123] Step S21, matching the current work process record with the sample work process records included in the sample library to obtain a first sample work process record having the highest similarity to the current work process record.

[0124] Step S22: extract the problem type marked in the first sample work process record as the first problem type requested to be solved in the current customer service work.

[0125] In the technical solution provided in the embodiment of the present application, the electronic device determines the first question type based on the sample work process records included in the pre-built sample library, and improves the accuracy of determining the question type through matching.

[0126] In the above step S21, the sample library includes a plurality of sample work process records and labels of the plurality of sample work process records. The label may be the problem type marked by the sample work process record, that is, the actual problem type corresponding to the sample work process record.

[0127] The electronic device matches the current work process record with multiple sample work process records to obtain the similarity between the current work process record and the multiple sample work process records. In the embodiment of the present application, the similarity between the current work process record and the sample work process records is used to represent the probability that the marked question type is correct. The electronic device selects the sample work process record with the highest similarity as the first sample work process record.

[0128] In the embodiment of the present application, when the current work process is recorded as the current conversation content or the current operation content, the sample work process is recorded as the sample conversation content or the sample operation content. The electronic device can match the current conversation content or the current operation content with the sample conversation content or the sample operation content to obtain the first sample conversation content or the first sample operation content as the first sample work process record.

[0129] In the case where the current work process record includes the current conversation content and the current operation content, the sample work process record includes the sample conversation content and the sample operation content. The electronic device may match the current conversation content with the sample conversation content to obtain the first sample conversation content with the highest similarity, match the current operation content with the sample operation content to obtain the first sample operation content with the highest similarity, and use the first sample conversation content and the first sample operation content as the first sample work process record.

[0130] In the above step S22, the electronic device extracts the label of the first sample work process record to obtain the marked problem type as the first problem type requested to be solved in the current customer service work.

[0131] In some embodiments, see Figure 3 , which is a second detailed schematic diagram of step S12 provided in the embodiment of the present application. The above step S12 may include the following steps.

[0132] Step S31, matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records having the highest similarity with the current work process record.

[0133] Step S32, input the problem types marked in the current work process record and a preset number of second sample work process records into the language model to obtain the first problem type requested to be solved in the current customer service work.

[0134] In the technical solution provided in the embodiment of the present application, the electronic device determines the question types annotated by multiple sample work process records with the highest similarity based on the sample work process records included in the pre-built sample library, that is, determines multiple question types with the highest accuracy, and determines the first question type from a preset number of question types in combination with the pre-trained language large model, thereby further improving the accuracy of the determined first question type.

[0135] In the above step S31, the preset number can be any number, such as 5, 10, etc., and can be set according to actual conditions and is not limited here.

[0136] The electronic device matches the current work process record with the multiple sample work process records, obtains the similarity between the current work process record and the multiple sample work process records, and uses the preset number of sample work process records with the highest similarity as the second sample work process record. Similar to the above step S21, please refer to the relevant description of the above step S21 for details.

[0137] In the above step S32, the language big model can be any pre-trained open source big model. The electronic device extracts the label of each second sample work process record, obtains the question type annotated by each second sample work process record, and inputs the current work process record and a preset number of question types into the language big model. The language big model performs semantic analysis on the current work process record, determines a question type output from the preset number of question types, and the electronic device uses the question type output by the language big model as the first question type.

[0138] In an embodiment of the present application, the electronic device can fill the current work process record and the problem type marked in each second sample work process record into a preset prompt template to obtain prompt information, and input the prompt information into the language model.

[0139] Before evaluating the customer service work, the electronic device can also train the language model based on the sample work process records and sample question types, as well as the corresponding standard question types. For each sample work process record, the sample work process record and the sample question type are input into the initial language model to obtain the initial question type, determine the loss value between the standard question type and the initial question type, and output the language model when the loss value is less than the preset loss value threshold. The loss value threshold can be set according to the actual situation, and the loss value threshold is not limited here. The device for training the language model can also be other devices besides electronic devices, and this is not limited.

[0140] In some embodiments, see Figure 4 , which is a third detailed schematic diagram of step S12 provided in the embodiment of the present application. The above step S12 may include the following steps.

[0141] Step S41, matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record. This is the same as the above step S31.

[0142] Step S42, inputting the question types annotated in the current work process record and a preset number of second sample work process records into the language macro model to obtain the second question type and the similarity of the second question type.

[0143] Step S43, determine whether the similarity of the second question type is greater than the similarity of the third sample work process record, and the third sample work process record is the second sample work process record with the highest similarity; if so, execute step S44; if not, execute step S45.

[0144] Step S44: The second problem type is used as the first problem type requested to be solved in the current customer service work.

[0145] Step S45: Use the problem type marked in the third sample work process record as the first problem type requested to be solved in the current customer service work.

[0146] In the technical solution provided in the embodiment of the present application, the electronic device determines the second question type based on the sample work process records included in the pre-built sample library and the pre-trained language model, and selects a more accurate question type as the first question type from the second question type obtained based on the language model and the question type directly obtained based on the sample library according to the similarity of the question types, thereby further improving the accuracy of the determined first question type.

[0147] In the above step S42, the similarity of the question type is the confidence, which indicates the probability that the question type is correct. The electronic device inputs the current work process record and a preset number of question types into the language model. The language model performs semantic analysis on the current work process record, determines a question type from the preset number of question types, and determines the similarity of the question type. The language model outputs the question type and the similarity, and the electronic device uses the output question type as the second question type and the output similarity as the similarity of the second question type.

[0148] In the embodiment of the present application, the electronic device can also train the language model according to the sample work process record and the sample question type, as well as the corresponding standard question type and standard similarity. For each sample work process record, the sample work process record and the sample question type are input into the initial language model to obtain the initial question type and the initial similarity, determine the loss value between the standard question type and the initial question type, and the loss value between the standard similarity and the initial similarity, and output the language model when both loss values ​​are less than the preset corresponding loss value threshold.

[0149] In the above step S43, the third sample work process record is the sample work process record with the highest similarity in the second sample work process record, that is, the third sample work process record is the same as the first sample work process record. The similarity of the third sample work process record is the similarity of the marked problem type.

[0150] In the embodiment of the present application, after executing the above step S41, the electronic device can select the second sample work process record with the highest similarity from the preset number of second sample work process records as the third sample work process record. The electronic device can also directly use the sample work process record with the highest similarity as the third sample work process record after matching the current work process record with multiple sample work process records.

[0151] The electronic device selects a more accurate question type as the first question type from the second question type obtained based on the language large model and the question type directly obtained based on the sample library according to the similarity. The electronic device determines whether the similarity of the second question type is greater than the similarity of the third sample work process record.

[0152] When the similarity of the second question type is greater than the similarity of the third sample work process record, the electronic device determines that the second question type obtained based on the language large model is more accurate, and the electronic device executes step S44 to use the second question type as the first question type.

[0153] When the similarity of the second question type is less than or equal to the similarity of the third sample work process record, the electronic device determines that the question type directly obtained based on the sample library is more accurate, and the electronic device executes step S45 to use the question type marked in the third sample work process record as the first question type.

[0154] In the embodiment of the present application, when the similarity of the second question type is equal to the similarity of the third sample work process record, the electronic device can also use the second question type as the first question type, and this is not limited.

[0155] In some embodiments, the electronic device can match the current work process record with the sample work process records included in the sample library in the following manner: converting the current work process record into text data; vectorizing the text data to obtain a current feature vector corresponding to the text data; and matching the current feature vector with the sample feature vector corresponding to the sample work process record included in the sample library.

[0156] In the embodiment of the present application, the electronic device can convert the current conversation content included in the current work process record into text data. For example, when the current conversation content is an online conversation content, the electronic device can analyze the online conversation content (i.e., directly use the text as text data; perform voice recognition on the voice and convert the voice into text data; perform image recognition on the picture and convert the picture into text data, etc., to obtain text data; when the current conversation content is a real-time voice conversation content, the electronic device can obtain a voice stream, perform voice recognition on the voice stream, convert the voice stream into text form, and obtain text data corresponding to the real-time voice conversation content; when the current conversation content is a video conversation content, the electronic device can obtain a video stream, perform recognition on the video stream, convert the video stream into text form, and obtain text data corresponding to the conversation content.

[0157] The electronic device can also convert the current operation content included in the current work process record into text data. The electronic device can determine the operation identifier and operation description corresponding to each operation of the first customer service personnel according to the pre-set correspondence between the operation, the operation identifier and the operation description, convert the multiple operations included in the current operation content into the operation identifier and the operation description, and obtain the text data corresponding to the current operation content.

[0158] In an embodiment of the present application, after obtaining the text data corresponding to the current work process record, the electronic device performs vectorization processing on the text data to obtain a feature vector corresponding to the text data, and the feature vector is the current feature vector.

[0159] The electronic device can perform vectorization processing on the text data corresponding to the current conversation content and the text data corresponding to the current operation content respectively, to obtain a feature vector corresponding to the current conversation content and a feature vector corresponding to the current operation content.

[0160] In an embodiment of the present application, the electronic device matches the current feature vector with the sample feature vector corresponding to each sample work process record, determines the similarity between the current feature vector and each sample feature vector, and then obtains the similarity between the current work process record and the sample work process record.

[0161] The electronic device can match the feature vector corresponding to the current conversation content with the feature vector corresponding to the sample conversation content, and match the feature vector corresponding to the current operation content with the feature vector corresponding to the sample operation content, to obtain the similarity between the current conversation content and the sample conversation content, and the similarity between the current operation content and the sample operation content.

[0162] In the technical solution provided in the embodiment of the present application, the electronic device converts the work process record into text data and performs vectorization processing to obtain a feature vector, and accurately matches the current work process record based on the feature vector.

[0163] In some embodiments, the first problem type includes a first problem main type and a first problem sub-type; the electronic device can perform the above step S14 in the following manner: among all customer service jobs that have been completed within the current evaluation cycle, retrieve the first customer service job that serves the first user and requests to solve the first problem sub-type under the first problem main type.

[0164] In the embodiment of the present application, the first problem type recorded in the current customer service work can be divided into a first problem main type and a first problem subtype. The first problem main type indicates the general category of the problem that the first user requests to be solved, and the first problem subtype indicates the specific type of the problem that the first user requests to be solved under the first problem main type. The electronic device can determine the first problem main type and the first problem subtype by the above method of determining the first problem type.

[0165] For example, the electronic device can match the current work process record with the sample work process records included in the sample library to obtain a first sample work process record, extract the problem main type and problem sub-type marked in the first sample work process record, and use the determined problem main type as the first problem main type and the determined problem sub-type as the first problem sub-type.

[0166] For another example, the electronic device can match the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record, and input the main type of questions and the sub-type of questions marked with the current work process record and the preset number of second sample work process records into the language model to obtain the first main type of questions and the first sub-type of questions.

[0167] For another example, the electronic device can also match the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record, and input the main question type and sub-question type annotated by the current work process record and the preset number of second sample work process records into the language model to obtain the second question main type, the second question sub-type and the corresponding similarity; if the corresponding similarity between the second question main type and the second question sub-type is greater than the similarity between the third sample work process record, the second question main type is used as the first question main type and the second question sub-type is used as the first question sub-type; if the corresponding similarity between the second question main type and the second question sub-type is less than or equal to the similarity between the third sample work process record, the question main type annotated by the third sample work process record is used as the first question main type, and the annotated question sub-type is used as the first question sub-type. For details, please refer to the above Figure 2 to Figure 4 Part of the description.

[0168] In an embodiment of the present application, the electronic device can retrieve the first customer service work based on the first main type of question and the first sub-type of question. The electronic device can first retrieve the customer service work whose user identifier is the first user identifier, then retrieve the customer service work whose customer service personnel belong to the category of the first customer service personnel, then retrieve the customer service work whose main type of question is the first main type of question, and finally retrieve the customer service work whose sub-type of question is the first sub-type of question. The order in which the electronic device searches based on the user identifier, the category to which the customer service personnel belongs, the main type of question, and the sub-type of question is not limited. For details, please refer to the relevant description of the above step S14.

[0169] In the technical solution provided in the embodiment of the present application, the electronic device can retrieve the first customer service work based on the main type of the question and the sub-type of the question. When the main type of the question consulted by the user is the same but the sub-type of the question is different, the service result corresponding to the customer service work will not be updated to improve the accuracy of the service quality evaluation.

[0170] In some embodiments, see Figure 5 , which is a first flow chart of modifying service results provided in an embodiment of the present application, may include the following steps.

[0171] Step S51, receiving a modification operation instruction, where the modification operation instruction includes a third problem type requested to be solved in a second customer service work serving a second user.

[0172] Step S52, modifying the fourth problem type requested to be solved in the second customer service work into the third problem type.

[0173] Step S53, searching for a third customer service job that serves the second user and requests a solution to a third problem type from among all customer service jobs that have been completed within the evaluation period of the second customer service job.

[0174] Step S54: if the third customer service task is retrieved, the service result corresponding to the third customer service task is updated.

[0175] In the technical solution provided in the embodiment of the present application, when the problem type is modified, the electronic device can modify the service result corresponding to the customer service work, that is, modify the service result corresponding to the customer service work with the modified problem type to ensure that the number of the first customer service work can be accurately counted later.

[0176] In the above step S51, the second customer service work is any customer service work that has been completed and whose problem type needs to be modified, the fourth problem type is the type of problem that the second user requests to be solved during the second customer service work, and the third problem type is the modified problem type.

[0177] The electronic device receives the modification operation instruction, extracts the customer service work identifier and the problem type from the modification operation instruction, the extracted customer service work identifier is the second customer service work identifier, and the extracted problem type is the third problem type.

[0178] In the embodiment of the present application, the electronic device can send each created customer service work and the related information of each recorded customer service work to the preset terminal device, and the preset terminal device can be a terminal device that displays the customer service work and related information to the customer service personnel. The preset terminal device can be a mobile phone device, a computer device, etc., and the preset terminal device is not limited here.

[0179] The customer service personnel involved in the second customer service work can modify the problem type of the second customer service work to the third problem type on the preset terminal device, and then the preset terminal device can generate a modification operation instruction according to the operation of the customer service personnel and send the modification operation instruction to the electronic device. When the electronic device and the preset terminal device are the same device, the electronic device can generate the modification operation instruction by itself.

[0180] In the embodiment of the present application, the preset terminal device can also display customer service work and related information to the quality inspector. The quality inspector can modify the problem type of the second customer service work to the third problem type on the preset terminal device, and then the preset terminal device can generate a modification operation instruction according to the operation of the quality inspector and send the modification operation instruction to the electronic device.

[0181] In the above step S52, the electronic device determines the second customer service task according to the identifier of the second customer service task, and modifies the problem type of the second customer service task to the third problem type.

[0182] In the above step S53, the electronic device retrieves a third customer service job that serves the same user as the second customer service job (i.e., the service users are all the second user) and solves the modified third problem type from all customer service jobs that have ended within the evaluation period of the second customer service job. This is similar to the method of retrieving the first customer service job in the above step S14, and for details, please refer to the above step S14 and the related descriptions.

[0183] In the embodiment of the present application, when searching for the third customer service work, the electronic device may also search based on the category to which the customer service personnel who participated in the second work belong, or the electronic device may also search based on the third question main type and the third question subtype included in the third question type. For details, please refer to the above description of searching for the first customer service work.

[0184] In the above step S54, after the electronic device retrieves the third customer service work, it updates the service result corresponding to the third customer service work. In the embodiment of the present application, the electronic device determines that the problem requested to be solved in the third customer service work with the latest end time has been solved according to the end time of the third customer service work, sets the corresponding service result to the first preset value, and determines that the problem requested to be solved in other third customer service work is not solved, and sets the corresponding service result to the second preset value. If the electronic device does not retrieve the third customer service work, no processing is performed.

[0185] In some embodiments, see Figure 6 , which is a second flow chart for modifying service results provided in an embodiment of the present application, may include the following steps.

[0186] Step S61, receiving a modification operation instruction, the modification operation instruction including a third problem type requested to be solved in a second customer service work serving a second user, which is the same as the above step S51.

[0187] Step S62, modifying the fourth problem type requested to be solved in the second customer service work to the third problem type, which is the same as the above step S52.

[0188] Step S63, searching for a third customer service job that serves the second user and requests to solve a third problem type from among all customer service jobs that have been completed within the evaluation period of the second customer service job, which is the same as the above step S53.

[0189] Step S64: if the third customer service task is retrieved, the service result corresponding to the third customer service task is updated, which is the same as the above step S54.

[0190] Step S65 , searching for a fourth customer service job that serves the second user and requests a solution to a fourth problem type from among all customer service jobs that have been completed within the evaluation period of the second customer service job.

[0191] Step S66: if the fourth customer service task is retrieved, then the service result corresponding to the fourth customer service task is updated.

[0192] In the technical solution provided in the embodiment of the present application, the electronic device can also modify the service results corresponding to the customer service work whose problem type is the problem type before modification, further ensuring that the number of the first customer service work can be accurately counted later.

[0193] Figure 6 In the description, only the example of first executing step S63 to step S64 and then executing step S65 to step S66 is used for explanation. The execution order of step S63 to step S64 and step S65 to step S66 is not limited.

[0194] In the above step S65, the electronic device retrieves a fourth customer service job that serves the same user as the second customer service job (i.e., the service users are all the second user) and solves the problem of the fourth problem type before the modification from among all the customer service jobs that have ended within the evaluation period of the second customer service job. This is similar to the method of retrieving the first customer service job in the above step S14, and for details, please refer to the above step S14 and the related descriptions in.

[0195] In the embodiment of the present application, when searching for the fourth customer service work, the electronic device may also search based on the category to which the customer service personnel who participated in the second work belong, or the electronic device may also search based on the fourth question main type and the fourth question subtype included in the fourth question type. For details, please refer to the above description of searching for the first customer service work.

[0196] In the above step S66, after the electronic device retrieves the fourth customer service work, it updates the service result corresponding to the fourth customer service work. In the embodiment of the present application, the electronic device determines that the problem requested to be solved in the fourth customer service work with the latest end time has been solved according to the end time of the fourth customer service work, sets the corresponding service result to the first preset value, and determines that the problem requested to be solved in other fourth customer service work is not solved, and sets the corresponding service result to the second preset value. If the electronic device does not retrieve the fourth customer service work, no processing is performed.

[0197] In the embodiment of the present application, the electronic device can also analyze the content of the conversation to obtain the content label corresponding to the customer service work, and evaluate the quality of the customer service work provided by the customer service personnel based on the content label, the first quantity and the second quantity, and the comprehensive customer service work process and customer service work results.

[0198] After the current customer service work is completed, the electronic device can obtain the current conversation content and convert the current conversation content into text data. Based on NLP, the text data of the current conversation content is recognized and analyzed to obtain the content label of the current conversation content, that is, the content label corresponding to the current customer service work.

[0199] The electronic device can also obtain each conversation record from the first customer service staff and the first user in real time during the current customer service work, and convert each conversation record into text data, identify and analyze the text data of each conversation record based on NLP, and obtain the content label of each conversation record. After the conversation between the first customer service staff and the first user ends, the electronic device can count the content labels of all conversation records to obtain the content label of the current conversation content. The way in which the electronic device obtains the conversation content is not limited here. The specific way in which the electronic device converts the conversation content / conversation record into text data can be found in the above-mentioned related description.

[0200] In an embodiment of the present application, the electronic device can obtain the time information of each conversation record. For example, when the first customer service staff has a real-time voice conversation with the first user, the electronic device can obtain the voice stream, perform voice recognition and voice activity detection (VAD) on the voice stream, and while converting the voice stream into text data, obtain the start time and end time of each real-time voice conversation record from the first customer service staff and from the first user as time information. The electronic device can also connect the voice stream to an automatic speech recognition (Automatic Speech Recognition, ASR) system, and convert the voice stream into text data with VAD information through the ASR system.

[0201] After acquiring the information, the electronic device can identify and analyze the text data and time information based on NLP to obtain the content label of the current conversation content.

[0202] In the embodiment of the present application, the electronic device can analyze the current conversation content from three aspects, namely, the wording standard, the service attitude and the call sequence, based on NLP, and obtain three types of content tags, namely, the wording standard, the service attitude and the call sequence. The electronic device can analyze only the text data of the conversation record from the first customer service staff to obtain the content tags of the wording standard type and the service attitude type, and analyze the time information of the conversation record from the first customer service staff and the first user to obtain the content tag of the call sequence type.

[0203] In the embodiment of the present application, the content tag of the standardized wording type includes at least one of the following tags: a welcome tag, a closing tag, and an illegal word tag.

[0204] The welcome tag indicates that the conversation record from the first customer service staff does not contain a welcome message. Based on NLP, the electronic device identifies whether the text data of the conversation record from the first customer service staff contains a welcome message, and obtains a corresponding content tag. When the recognition result is that the welcome message is not contained, the content tag includes a welcome message tag. In the embodiment of the present application, the welcome message tag may also indicate that the welcome message is contained. For ease of description, the following description will be taken as an example in which the welcome message tag indicates that the welcome message is not contained.

[0205] The electronic device can match the text data of the first conversation record from the first customer service staff with a preset welcome word library based on NLP, and when the similarity is greater than a preset threshold, determine that the first conversation record contains a welcome word. The method for the electronic device to recognize the welcome word is not limited here.

[0206] The closing remark tag indicates that the conversation record from the first customer service staff does not contain a closing remark. Based on NLP, the electronic device identifies whether the text data of the conversation record from the first customer service staff contains a closing remark, and obtains a corresponding content tag. When the recognition result is that the closing remark is not contained, the content tag includes a closing remark tag. In the embodiment of the present application, the closing remark tag may also indicate that the closing remark is contained. For ease of description, the following description will be taken as an example in which the closing remark tag indicates that the closing remark is not contained.

[0207] The electronic device can match the text data of the last conversation record from the first customer service staff with a preset closing word library based on NLP, and when the similarity is greater than a preset threshold, determine that the last conversation record contains the closing word. The method for the electronic device to identify the closing word is not limited here.

[0208] The illegal word label indicates that the conversation record from the first customer service staff contains illegal words. The electronic device identifies whether the text data of the conversation record from the first customer service staff contains illegal words based on NLP, and obtains a corresponding content label. When the identification result is that illegal words are contained, the content label includes the illegal word label. In an embodiment of the present application, when multiple illegal words are identified, the content label may include a corresponding number of illegal word labels.

[0209] The electronic device can match the text data of the conversation record from the first customer service staff with a preset illegal word library based on NLP, and when the similarity is greater than a preset threshold, it is determined that the conversation record from the first customer service staff contains illegal words. The method for the electronic device to identify illegal words is not limited here.

[0210] In an embodiment of the present application, a content tag of the service attitude type is a service attitude tag, which indicates that the first customer service staff has an attitude problem during the conversation with the first user. The electronic device identifies the text data from the conversation record of the first customer service staff based on NLP, determines whether the first customer service staff has an attitude problem, and obtains a corresponding content tag. When the recognition result is that there is an attitude problem, the content tag includes the corresponding service attitude tag. In an embodiment of the present application, when multiple attitude problems are identified, the content tag may include a corresponding number of service attitude tags.

[0211] The electronic device can analyze the text data from the conversation record of the first customer service staff based on NLP, obtain the attitude of the first customer service staff, and determine whether the attitude of the first customer service staff is the same as the preset attitude. When the attitude of the first customer service staff is the same as the preset attitude, it is determined that the first customer service staff has an attitude problem. The preset attitude may include rhetorical questions, sarcasm, procrastination, evasion, etc., and the method of identifying attitude problems and the preset attitude of the electronic device are not limited here.

[0212] In the embodiment of the present application, the content tag of the call sequence type includes at least one of the following tags: a call interruption tag and a timeout tag.

[0213] The "interruption" tag indicates that the first customer service staff has interrupted the conversation during the conversation. Based on NLP, the electronic device determines whether there is an interruption according to the time information of the conversation record between the first customer service staff and the first user, and obtains a corresponding content tag. When the recognition result is that there is an interruption, the content tag includes the "interruption" tag. In an embodiment of the present application, when multiple interruptions are identified, the content tag may include a corresponding number of "interruption" tags.

[0214] Based on NLP, the electronic device can determine whether the start time of the conversation record from the first customer service staff is earlier than the end time of the conversation record from the first user when recognizing that the time sequence of the conversation record is the conversation record from the first user > the conversation record from the customer service staff. When the start time is earlier than the end time, it is determined that the first customer service staff has interrupted the conversation during the conversation. The way in which the electronic device determines the interruption is not limited here.

[0215] The timeout tag indicates that a timeout occurred during the conversation with the first customer service staff. Based on NLP, the electronic device determines whether a timeout occurred according to the time information of the conversation record from the first customer service staff, and obtains a corresponding content tag. When the recognition result is that a timeout occurred, the content tag includes a timeout tag. In an embodiment of the present application, when multiple timeouts are identified, a corresponding number of timeout tags may be included in the content tag.

[0216] The electronic device can determine, based on NLP, whether the duration from the start time (i.e., the start time of the first conversation record) to the end time (i.e., the start time of the last conversation record) of the conversation record from the first customer service staff is greater than a preset duration, and if it is greater than the preset duration, it is determined that the first customer service staff has a timeout during the conversation. The manner in which the electronic device determines the timeout is not limited herein.

[0217] When the first customer service staff has an online conversation with the first user, the electronic device can analyze the current conversation content based on NLP from only two aspects: terminology standards and service attitude, and obtain two types of content tags: terminology standards and service attitude.

[0218] In the embodiment of the present application, the electronic device can evaluate the quality of customer service work from two aspects: service process and service result. After the current evaluation cycle ends, the electronic device can obtain the content label corresponding to each customer service work participated in by the target customer service personnel in the current evaluation cycle, determine the service process score corresponding to each customer service work according to the preset correspondence between the content label and the score, and calculate the average of the service process scores corresponding to all customer service works to obtain the service process score corresponding to the target customer service personnel.

[0219] The electronic device can obtain a service result score based on the first quantity and the second quantity, and calculate the customer service work quality score corresponding to the target customer service personnel based on the preset weights of the service process score and the service result score.

[0220] In the embodiment of the present application, the corresponding content label can also be recorded in the customer service work and displayed to the quality inspector on the preset terminal device. The quality inspector can modify the content label of any customer service work on the preset terminal device, and then the preset terminal device can generate a modification operation instruction according to the operation of the quality inspector and send the modification operation instruction to the electronic device. The electronic device can modify the corresponding content label according to the modification operation instruction and update the service result score.

[0221] Below through Figure 7 to Figure 9 The service evaluation method provided in the embodiment of the present application is described in detail.

[0222] Figure 7 A schematic diagram of a service evaluation provided in an embodiment of the present application. Figure 7 In the example, the online conversation and real-time voice conversation between the customer service staff and the user are taken as examples. The user can access through the online customer service or voice customer service. When the user accesses through the online customer service, the electronic device can obtain the conversation content between the online customer service staff and the user in real time, and input it into the NLP program, and perform standardized terminology recognition, call sequence recognition and service attitude recognition based on NLP to obtain content labels. When the user accesses through the voice customer service, the electronic device will pass the real-time acquired voice stream through the interactive voice response (IVR) application, and perform voice recognition through the ASR system, and input the recognized conversation text record information (i.e. text data) and VAD record (i.e. time information) into the NLP program for processing to obtain content labels.

[0223] In the work order system, the electronic device can also input the conversation content and the customer service staff's operation content into the AI ​​model, so that the AI ​​model outputs the work order classification (i.e., problem type). After the conversation ends, the customer service staff can fill in the work order (i.e., service description information) of this customer service work through the work order system. After obtaining the work order of each customer service work, the electronic device determines that this customer service work is completed, and updates the one-time solution information (i.e., service result) of the relevant customer service work based on the work order classification, user identification, and customer service staff's skill group information.

[0224] The electronic device can send content tags and work orders to the terminal device of the quality inspector (i.e., the preset terminal device), so that the quality inspector can conduct quality inspection and review, and feedback the electronic device to modify the information. After the evaluation cycle is over, the electronic device calculates the one-time resolution rate (i.e., the service result score), and calculates the service quality score based on the content tag. The total score is calculated based on the service quality score and the one-time resolution rate, and recorded in the scoring system. Service

[0225] Figure 8 A schematic diagram of setting a work order provided in an embodiment of the present application. Take the case where the service description information is a work order and the service result is a one-time solution information as an example. The electronic device obtains the work order, sets the one-time solution information of the work order to yes (i.e., the first preset value), and searches for work orders created within the evaluation period that have the same service user and the same skill group of the customer service personnel (i.e., the category to which the customer service personnel belong). When found, the electronic device determines whether the problems are the same, that is, whether the customer service work classifications (i.e., problem types) included in the work orders are the same. If they are the same, the one-time solution information of the work orders for the same problem (i.e., the same problem type) is set to no (i.e., the second preset value); if they are not the same, no processing is performed.

[0226] Fig. 9 A schematic diagram of modifying one-time resolution information provided in an embodiment of the present application. Take the service description information as a work order and the service result as a one-time resolution information as an example. The electronic device receives the modification information including the modification of the work order classification (i.e., the problem type) included in the work order, modifies the work order classification, and searches for work orders created within the evaluation period with the same service user, the same customer service staff skill group (i.e., the category to which the customer service staff belongs), and the same work order classification, and sets the corresponding one-time resolution information. For details, please refer to the above Figure 1 to Figure 6 Related description.

[0227] In the technical solution provided in the embodiment of the present application, the electronic device is evaluated from two aspects. One aspect is to evaluate from the service process, that is, to judge whether the service process complies with the service specifications, operating specifications, and whether it violates the service taboos. On the other hand, it is evaluated from the service results to judge whether the customer service work of the customer service staff has successfully solved the user's problem. Combined with the above two aspects, the scores are calculated separately, and the two scores are weighted statistics to calculate a total score as the evaluation result of the customer service quality and the customer service staff's ability.

[0228] Corresponding to the above service evaluation method, the present application embodiment also provides a service evaluation device, such as Fig.10 As shown, the above device comprises:

[0229] The first acquisition module 101 is used to acquire the current work process record of the current customer service work serving the first user;

[0230] A first obtaining module 102 is used to determine the first problem type requested to be solved in the current customer service work according to the current work process record;

[0231] A first setting module 103 is used to set the service result corresponding to the current customer service work to a first preset value after the current customer service work is completed, and the first preset value indicates that the problem requested to be solved in the customer service work has been solved;

[0232] A search module 104 is used to search for a first customer service job that serves a first user and requests to solve a first problem type from among all customer service jobs that have been completed in a current evaluation cycle;

[0233] A second setting module 105 is used to set the service result corresponding to the first customer service work to a second preset value if the first customer service work is retrieved, and the second preset value indicates that the problem requested to be solved in the customer service work is not solved;

[0234] The second acquisition module 106 is used to acquire, after the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value in the current evaluation cycle, and to acquire a second number of all customer service tasks in which the target customer service personnel participated in the current evaluation cycle;

[0235] The second obtaining module 107 is used to determine the customer service work quality score corresponding to the target customer service personnel according to the first quantity and the second quantity.

[0236] In some embodiments, the first obtaining module 102 is specifically used to:

[0237] Matching the current work process record with the sample work process records included in the sample library to obtain a first sample work process record having the highest similarity to the current work process record;

[0238] The problem type marked in the first sample work process record is extracted as the first problem type requested to be solved in the current customer service work.

[0239] In some embodiments, the first obtaining module 102 is specifically used to:

[0240] Matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0241] The problem types marked in the current work process record and a preset number of second sample work process records are input into the language model to obtain the first problem type requested to be solved in the current customer service work.

[0242] In some embodiments, the first obtaining module 102 is specifically used to:

[0243] Matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record;

[0244] Inputting the question types marked by the current work process record and a preset number of second sample work process records into the language macro model to obtain the second question type and the similarity between the second question type;

[0245] If the similarity of the second question type is greater than the similarity of the third sample work process record, the second question type is used as the first question type requested to be solved in the current customer service work, and the third sample work process record is used as the second sample work process record with the highest similarity;

[0246] If the similarity of the second problem type is less than or equal to the similarity of the third sample work process record, the problem type marked in the third sample work process record is used as the first problem type requested to be solved in the current customer service work.

[0247] In some embodiments, the first obtaining module 102 is specifically used to:

[0248] Convert the current work process records into text data;

[0249] Vectorize the text data to obtain the current feature vector corresponding to the text data;

[0250] The current feature vector is matched with the sample feature vector corresponding to the sample work process record included in the sample library.

[0251] In some embodiments, the first question type includes a first question main type and a first question subtype; the above-mentioned retrieval module 104 is specifically used to:

[0252] A first customer service job that serves a first user and requests a solution to a first problem subtype under a first problem main type is retrieved.

[0253] In some embodiments, the apparatus further comprises: a modification module; the modification module is configured to:

[0254] receiving a modification operation instruction, wherein the modification operation instruction includes a third problem type requested to be solved in a second customer service work serving a second user;

[0255] Change the fourth problem type requested to be solved in the second customer service work to the third problem type;

[0256] Retrieving a third customer service job that serves the second user and requests resolution of a third problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0257] If the third customer service task is retrieved, the service result corresponding to the third customer service task is updated.

[0258] In some embodiments, the modification module is further configured to:

[0259] Retrieving a fourth customer service job that serves the second user and requests a solution to a fourth problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located;

[0260] If the fourth customer service task is retrieved, the service result corresponding to the fourth customer service task is updated.

[0261] In the technical solution provided in the embodiment of the present application, the type of problem that the current customer service work requests to solve is determined based on the current work process record of the current customer service work. After the current customer service work is completed, the service result corresponding to the current customer service work indicating whether the user problem is solved is set, and the service result corresponding to the first customer service work that serves the same user and solves the same problem is updated. By counting the first number of customer service work that solves user problems and the second number of all customer service work that the target customer service personnel participate in during the evaluation period, the quality of customer service work provided by the customer service personnel is evaluated based on the first number and the second number. Through the technical solution provided in the embodiment of the present application, the service quality of the customer service work of the customer service personnel is evaluated based on the service results, and then the quality of the customer service work of the customer service personnel is evaluated, which reflects the working ability of the customer service personnel.

[0262] The present application also provides an electronic device, such as Fig.11 As shown, it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0263] Memory 113, used for storing computer programs;

[0264] The processor 111 is used to execute the program stored in the memory 113 to implement the following steps:

[0265] Obtain the current work process record of the current customer service work serving the first user;

[0266] Based on the current work process records, determine the first type of problem that is being requested to be solved in the current customer service work;

[0267] After the current customer service work is completed, the service result corresponding to the current customer service work is set to a first preset value, and the first preset value indicates that the problem requested to be solved in the customer service work has been solved;

[0268] Retrieve the first customer service work that serves the first user and requests to solve the first problem type from all customer service work that has been completed in the current evaluation cycle;

[0269] If the first customer service work is retrieved, the service result corresponding to the first customer service work is set to a second preset value, the second preset value indicating that the problem requested to be solved in the customer service work is not solved;

[0270] After the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value is obtained during the current evaluation cycle, and a second number of all customer service tasks in which the target customer service personnel participated during the current evaluation cycle is obtained;

[0271] A customer service work quality score corresponding to the target customer service personnel is determined based on the first quantity and the second quantity.

[0272] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0273] The communication interface is used for communication between the above terminal and other devices.

[0274] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0275] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0276] In another embodiment provided in the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the service evaluation method described in any one of the above embodiments is implemented.

[0277] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the service evaluation method described in any one of the above embodiments.

[0278] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0279] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0280] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0281] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A service evaluation method, characterized in that: The method comprises: Obtain the current work process record of the current customer service work serving the first user; Determine, according to the current work process record, the first problem type requested to be solved in the current customer service work; After the current customer service work is completed, setting the service result corresponding to the current customer service work to a first preset value, wherein the first preset value indicates that the problem requested to be solved in the customer service work has been solved; Retrieving, from all customer service jobs that have been completed in the current evaluation cycle, a first customer service job that serves the first user and requests a solution to the first problem type; If the first customer service work is retrieved, setting the service result corresponding to the first customer service work to a second preset value, wherein the second preset value indicates that the problem requested to be solved in the customer service work is not solved; After the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value is obtained during the current evaluation cycle, and a second number of all customer service tasks in which the target customer service personnel participated during the current evaluation cycle is obtained; A customer service work quality score corresponding to the target customer service personnel is determined based on the first number and the second number.

2. The method according to claim 1, characterized in that The step of determining the first problem type requested to be solved in the current customer service work according to the current work process record comprises: Matching the current work process record with the sample work process records included in the sample library to obtain a first sample work process record with the highest similarity to the current work process record; The problem type marked in the first sample work process record is extracted as the first problem type requested to be solved in the current customer service work.

3. The method according to claim 1, characterized in that The step of determining the first problem type requested to be solved in the current customer service work according to the current work process record comprises: Matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record; The problem types marked in the current work process record and the preset number of second sample work process records are input into the language model to obtain the first problem type requested to be solved in the current customer service work.

4. The method according to claim 1, characterized in that The step of determining the first problem type requested to be solved in the current customer service work according to the current work process record comprises: Matching the current work process record with the sample work process records included in the sample library to obtain a preset number of second sample work process records with the highest similarity to the current work process record; Inputting the question types marked by the current work process record and the preset number of second sample work process records into the language macro model to obtain the similarity between the second question type and the second question type; If the similarity of the second question type is greater than the similarity of the third sample work process record, the second question type is used as the first question type requested to be solved in the current customer service work, and the third sample work process record is used as the second sample work process record with the highest similarity; If the similarity of the second problem type is less than or equal to the similarity of the third sample work process record, the problem type marked in the third sample work process record is used as the first problem type requested to be solved in the current customer service work.

5. The method according to any one of claims 2 to 4, characterized in that: The step of matching the current work process record with the sample work process record included in the sample library includes: Converting the current work process record into text data; Performing vectorization processing on the text data to obtain a current feature vector corresponding to the text data; The current feature vector is matched with a sample feature vector corresponding to a sample work process record included in the sample library.

6. The method according to claim 1, characterized in that The first question type includes a first question main type and a first question subtype; The step of retrieving a first customer service job that serves the first user and requests to solve the first problem type includes: A first customer service job that serves the first user and requests a solution to the first problem subtype under the first problem main type is retrieved.

7. The method according to claim 1, characterized in that The method further comprises: receiving a modification operation instruction, wherein the modification operation instruction includes a third problem type requested to be solved in a second customer service work serving a second user; Modify the fourth problem type requested to be solved in the second customer service work to the third problem type; Retrieving a third customer service job that serves the second user and requests to solve the third problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located; If the third customer service task is retrieved, the service result corresponding to the third customer service task is updated.

8. The method according to claim 7, characterized in that The method further comprises: Retrieving a fourth customer service job that serves the second user and requests resolution of a fourth problem type from all customer service jobs that have been completed within the evaluation period in which the second customer service job is located; If the fourth customer service task is retrieved, the service result corresponding to the fourth customer service task is updated.

9. A service evaluation device, characterized in that: The device comprises: A first acquisition module is used to acquire a current work process record of a current customer service work serving a first user; A first obtaining module is used to determine the first problem type requested to be solved in the current customer service work according to the current work process record; A first setting module is used to set the service result corresponding to the current customer service work to a first preset value after the current customer service work is completed, and the first preset value indicates that the problem requested to be solved in the customer service work has been solved; A retrieval module, configured to retrieve, from among all customer service jobs that have been completed in a current evaluation cycle, a first customer service job that serves the first user and requests a solution to the first problem type; A second setting module is used to set the service result corresponding to the first customer service work to a second preset value if the first customer service work is retrieved, and the second preset value indicates that the problem requested to be solved in the customer service work is not solved; A second acquisition module is used to acquire, after the current evaluation cycle ends, a first number of customer service tasks in which the target customer service personnel participated and whose service results are a first preset value in the current evaluation cycle, and to acquire a second number of all customer service tasks in which the target customer service personnel participated in the current evaluation cycle; The second obtaining module is used to determine the customer service work quality score corresponding to the target customer service personnel according to the first number and the second number.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Images