A platform training method based on AI customer service

Abstract

The present invention relates to the field of AI training, and in particular to a platform training method based on AI customer service, which reads a number of customer service data as training samples and divides them into a number of sub-samples according to a preset sampling rate, uses the sub-samples to train a number of AI customer service models respectively, verifies the training results according to the sub-samples, determines the case closing rate of each AI customer service model according to the verification results, sets a case closing rate threshold, compares the case closing rate of each AI customer service model with the case closing rate threshold, and obtains the corresponding case closing comparison result, determines the average pass rate of the AI ​​customer service platform according to the case closing comparison result, compares the average pass rate with the pass rate threshold, and when the average pass rate is less than the pass rate threshold, adjusts the sampling rate of the sub-samples. By adjusting the sampling rate in real time, the problem of low recognition accuracy caused by the inability to accurately match the AI ​​customer service model with the question data is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of AI training, and in particular to a platform training method based on AI customer service. Background Art

[0002] At present, the intelligent customer service systems on the market only support keyword retrieval, which makes it difficult to truly understand the user's query intent. If the chat content exceeds the corpus, no effective answer can be given, and the corpus needs to be revised and expanded frequently. Because the corpus is not powerful enough, users will only get programmed and simplified replies after consulting questions, and "turnovers" often occur, and the problems cannot be solved. This seriously affects the user experience, and users will have a sense of distrust in this, which will damage the company's good image and be detrimental to the company's long-term development. Our customer service platform based on AI training can understand the user's intentions more intelligently, provide more professional replies, and achieve free questions and answers, which directly improves the user's consulting experience.

[0003] Chinese Patent Authorization Announcement No.: CN112365894B discloses a composite voice interaction method, device, computer equipment and storage medium based on AI, involving artificial intelligence technology, which can be applied to smart government affairs scenarios, including receiving the data to be replied sent by the user end, parsing it to obtain the text data to be replied; if the number of consecutive times that the answer data corresponding to the text data to be replied is not obtained in the local knowledge base exceeds the number threshold, the first manual service prompt information of transferring to the manual seat customer service is sent to the user end; if the number of consecutive times that the answer data corresponding to the text data to be replied is not obtained in the local knowledge base does not exceed the number threshold, the answer data corresponding to the text data to be replied is obtained in the local knowledge base. This method realizes the priority use of intelligent customer service to interact with users, and only prompts to switch to manual customer service when the current cumulative number of times that the intelligent customer service has not accurately replied to the user exceeds the number threshold, and the collected real dialogue data can be used as a training set to further train the AI ​​model.

[0004] However, the above method has the following problems: the recognition accuracy is not high. Summary of the invention

[0005] To this end, the present invention provides a platform training method based on AI customer service to overcome the problem in the prior art that the matching degree between the AI ​​customer service model and the question data is low, resulting in low recognition accuracy.

[0006] To achieve the above objectives, the present invention provides a platform training method based on AI customer service, comprising:

[0007] Read some customer service data as training samples and divide them into several sub-samples according to the preset sampling rate;

[0008] The sub-samples will be used to train several AI customer service models respectively, and the sub-samples will be used to verify the training results;

[0009] Determine the case closing rate of each AI customer service model based on the verification results;

[0010] Set a case closing rate threshold, compare the case closing rate of each AI customer service model with the case closing rate threshold, and obtain the corresponding case closing comparison results;

[0011] Determine the average pass rate of the AI ​​customer service platform based on the case closing comparison results;

[0012] comparing the average pass rate with a pass rate threshold;

[0013] When the average pass rate is less than the pass rate threshold, adjusting the sampling rate of the sub-sample;

[0014] The case closing rate is the percentage of cases in which the training results are the same as the results of each case in the subsample after the AI ​​customer service model is used for training and the training results are obtained;

[0015] The sampling rate is related to the recognition capability of the AI ​​customer service model.

[0016] Furthermore, when dividing the plurality of sub-samples, the plurality of customer service data are used as the training samples and are sorted according to a plurality of indicators.

[0017] The indicators include the type, specialty and / or processing time of the customer service data.

[0018] Furthermore, the step of collecting the case closing rate of a single AI customer service model includes:

[0019] Corresponding to each sub-sample received by the single AI customer service model;

[0020] Poll the number of cases answered by the customer service in the corresponding AI customer service model, and calculate the case closing rate of the AI ​​customer service model in the corresponding sub-sample;

[0021] When the case closing rate of a single AI customer service model is lower than the case closing rate threshold, adjusting the sampling rate of the corresponding AI customer service model;

[0022] The case closing rates of several AI customer service models are collected again and used as new case closing rates;

[0023] Among them, a timer is provided in the AI ​​customer service model for polling the case closing status of the AI ​​customer service model at fixed intervals.

[0024] Further, the case closing rate is compared with the case closing rate threshold to obtain the corresponding case closing comparison result.

[0025] Among them, the case closing rate threshold is the minimum standard for the case closing rate of a single AI customer service model.

[0026] Furthermore, when the case closing rate of a single AI customer service model is lower than the case closing rate threshold, the case data of the case is collected.

[0027] The case data includes pictures, texts and / or sounds.

[0028] Furthermore, the AI ​​customer service model is used to automatically annotate the case data, the annotated results are compared with the answer results of the AI ​​customer service model, the sampling rate of the AI ​​customer service model is adjusted and the adjusted sampling rate is used as the new sampling rate.

[0029] Further, the case closing rates of several AI customer service models are calculated again and used as new case closing rates to calculate the average pass rate of the AI ​​customer service platform.

[0030] Among them, the average pass rate is the average of the case closing rates of several AI customer service models.

[0031] Further, the average pass rate is compared with the pass rate threshold,

[0032] Among them, the pass rate threshold is the minimum standard of the average case closing rate of several AI customer service models.

[0033] Furthermore, when the average pass rate is less than the pass rate threshold, the sampling rate of the AI ​​customer service model is adjusted secondary.

[0034] Furthermore, the several AI customer service models after the second adjustment are sorted from high to low according to the pass rate, and the AI ​​customer service models are matched with the sub-samples according to several indicators.

[0035] Compared with the prior art, the present invention reads a number of customer service data as training samples, and divides them into a number of sub-samples according to a preset sampling rate, uses the sub-samples to train a number of AI customer service models respectively, verifies the training results according to the sub-samples, determines the case closing rate of each AI customer service model according to the verification results, sets a case closing rate threshold, compares the case closing rate of each AI customer service model with the case closing rate threshold, and obtains the corresponding case closing comparison result, determines the average pass rate of the AI ​​customer service platform according to the case closing comparison result, compares the average pass rate with the pass rate threshold, and adjusts the sampling rate of the sub-samples when the average pass rate is less than the pass rate threshold. By adjusting the sampling rate in real time, the labor cost is reduced, the efficiency of customer service is improved, and the accuracy of answering customer needs is increased, which effectively solves the problem of low recognition accuracy caused by the inability to accurately match the AI ​​customer service model with the question data.

[0036] Furthermore, by using several customer service data as training samples and sorting them according to several indicators, corresponding the sub-samples to the AI ​​customer service models, and obtaining the case closing rate of the AI ​​customer service model for each sub-sample, after completing the verification, the AI ​​customer service model with a higher case closing rate is selected, which can improve the targeting of the sub-samples, thereby effectively saving the training time of the AI ​​customer service model.

[0037] Furthermore, by determining the case closing rate of the AI ​​customer service model, the AI ​​customer service model's ability to answer questions is judged. After the judgment is completed, the AI ​​customer service model's ability to answer various types of questions can be quickly determined, thereby effectively improving the accuracy of answering customer needs.

[0038] Furthermore, by setting a timer in the AI ​​customer service model and polling the case closure status of the AI ​​customer service model at fixed intervals, real-time monitoring of the case closure rate is achieved, thereby improving the efficiency of customer service.

[0039] Furthermore, by setting case closure rate thresholds and pass rate thresholds, the AI ​​customer service model was adjusted to reduce labor costs and increase the accuracy of answering customer needs.

[0040] Furthermore, by adjusting the sampling rate of the AI ​​customer service model, the AI ​​customer service model is optimized, allowing users to get more accurate answers.

[0041] Furthermore, by sorting the AI ​​customer service models after the second adjustment from high to low according to the pass rate, and matching the AI ​​customer service models with the labels of various types of questions, real-time adjustment of the AI ​​customer service models is achieved, enabling the AI ​​customer service models to quickly adapt to questions with corresponding labels, thereby reducing errors in model answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1This is a flow chart of a platform training method based on AI customer service of the present invention;

[0043] Figure 2 This is a flow chart of collecting the case closing rate of a single AI customer service model in the present invention;

[0044] Figure 3 The present invention is a flow chart for adjusting the sampling rate. DETAILED DESCRIPTION

[0045] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0047] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is only for description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0048] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0049] See also Figure 1 As shown, it is a flow chart of a platform training method based on AI customer service of the present invention, including:

[0050] Step S1, reading a number of customer service data as training samples, and dividing them into a number of sub-samples according to a preset sampling rate;

[0051] Step S2: using the sub-samples to train several AI customer service models respectively, and using the sub-samples to verify the training results;

[0052] Step S3, determining the case closing rate of each AI customer service model according to the verification result;

[0053] Step S4, setting a case closing rate threshold, and comparing the case closing rate of each AI customer service model with the case closing rate threshold, and obtaining the corresponding case closing comparison result;

[0054] Step S5, determining the average pass rate of the AI ​​customer service platform according to the case comparison results;

[0055] Step S6, comparing the average pass rate with the pass rate threshold;

[0056] Step S7, when the average qualified rate is less than the qualified rate threshold, adjusting the sampling rate of the sub-samples;

[0057] Among them, the case closing rate is the percentage of cases in the sub-sample whose training results are the same as those of the cases in the sub-sample after the AI ​​customer service model is trained and the training results are obtained;

[0058] The sampling rate is related to the recognition capability of the AI ​​customer service model.

[0059] The present invention reads a number of customer service data as training samples, and divides them into a number of sub-samples according to a preset sampling rate, uses the sub-samples to train a number of AI customer service models respectively, verifies the training results according to the sub-samples, determines the case closing rate of each AI customer service model according to the verification results, sets a case closing rate threshold, compares the case closing rate of each AI customer service model with the case closing rate threshold, and obtains the corresponding case closing comparison result, determines the average pass rate of the AI ​​customer service platform according to the case closing comparison result, compares the average pass rate with the pass rate threshold, and adjusts the sampling rate of the sub-samples when the average pass rate is less than the pass rate threshold. By adjusting the sampling rate in real time, the labor cost is reduced, the efficiency of customer service is improved, the accuracy of answering customer needs is increased, and the problem of low matching between the AI ​​customer service model and the question data, which leads to low recognition accuracy, is effectively solved.

[0060] Specifically, when dividing several sub-samples, several customer service data are used as training samples and sorted according to several indicators.

[0061] Among them, the indicators include the type, specialty and / or processing time of customer service data.

[0062] By using several customer service data as training samples and sorting them according to several indicators, matching the sub-samples with the AI ​​customer service models, and obtaining the case closing rate of the AI ​​customer service model for each sub-sample, after completing the verification, the AI ​​customer service model with a higher case closing rate is selected, which can improve the targeting of the sub-samples and effectively save the training time of the AI ​​customer service model.

[0063] Embodiment 1:

[0064] The processing time of sample A is 10 minutes, the processing time of sample B is 8 minutes, and the processing time of sample C is 11 minutes. The samples are ranked as sample B, sample A, and sample C.

[0065] Embodiment 2:

[0066] If sample type A is technology, sample type B is finance, and sample type C is technology, then when classifying samples, A and C are in one group, and B is in another group.

[0067] By determining the case closing rate of the AI ​​customer service model, the AI ​​customer service model's ability to answer questions is judged. After the judgment is completed, the AI ​​customer service model's ability to answer various types of questions can be quickly determined, thereby effectively improving the accuracy of answering customer needs.

[0068] See also Figure 2 As shown, the steps of collecting the case closing rate of a single AI customer service model in the present invention include:

[0069] Step St1, corresponding to each sub-sample received by a single AI customer service model;

[0070] Step St2, polling the number of cases answered by the customer service in the corresponding AI customer service model, and calculating the case closing rate of the AI ​​customer service model in the corresponding sub-sample;

[0071] Step St3: when the case closing rate of a single AI customer service model is lower than the case closing rate threshold, the sampling rate of the corresponding AI customer service model is adjusted;

[0072] Step St4, collecting the case closing rates of several AI customer service models again and using them as new case closing rates;

[0073] Among them, a timer is set in the AI ​​customer service model to poll the case closing status of the AI ​​customer service model at fixed intervals.

[0074] In the specific implementation, setting the fixed time to 0.1s has the best optimization effect on the AI ​​customer service model.

[0075] Embodiment 3:

[0076] The fixed time is 0.1s. At 0.1s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 10. At 0.2s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 15. At 0.3s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 18. At 0.4s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 20.

[0077] Embodiment 4:

[0078] The fixed time is 0.5s. At 0.5s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 11. At 1.0s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 15. At 1.5s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 18. At 2.0s, the number of cases answered by the customer service in the AI ​​customer service model is polled, and the number of cases answered by the customer service is monitored to be 22.

[0079] By setting a timer in the AI ​​customer service model and polling the case closure status of the AI ​​customer service model at fixed intervals, real-time monitoring of the case closure rate is achieved, thereby improving the efficiency of customer service.

[0080] Specifically, the case closing rate is compared with the case closing rate threshold to obtain the corresponding case closing comparison result.

[0081] Among them, the case closure rate threshold is the minimum standard for the case closure rate of a single AI customer service model.

[0082] Specifically, when the case closing rate of a single AI customer service model is lower than the case closing rate threshold, the case data of the case is collected.

[0083] The case data includes data that can be annotated with images, texts and / or sounds.

[0084] Embodiment 5:

[0085] If the case closure rate threshold is 80%, the monitored case closure rate is 75%, and the case closure rate is less than the case closure rate threshold, the image data of the case is collected. If the question raised by the user is the number of an object in picture A, the object in picture A needs to be labeled.

[0086] Embodiment 6:

[0087] If the case closure rate threshold is 80%, the monitored case closure rate is 75%, and the case closure rate is less than the case closure rate threshold, the text data of the case is collected. If the question raised by the user is to inquire about the meaning of text B, all the text in text B needs to be annotated.

[0088] Embodiment 7:

[0089] If the case closure rate threshold is 80%, the monitored case closure rate is 75%, and the case closure rate is less than the case closure rate threshold, the sound data of the case is collected. If the question raised by the user is the meaning of sound clip C, the content in clip C needs to be translated and extracted.

[0090] See also Figure 3As shown, the steps of adjusting the sampling rate of the present invention include:

[0091] Step Sp1, using the AI ​​customer service model to automatically label case data;

[0092] Step Sp2, comparing the obtained annotation results with the answer results of the AI ​​customer service model;

[0093] Step Sp3, adjust the sampling rate of the AI ​​customer service model and use the adjusted sampling rate as the new sampling rate.

[0094] Specifically, the case closing rates of several AI customer service models are calculated again and used as the new case closing rates. The calculator calculates the average pass rate of the AI ​​customer service platform.

[0095] Among them, the average pass rate is the average of the case closing rates of several AI customer service models.

[0096] Embodiment 8:

[0097] If platform G has four AI customer service models A, B, C, and D, the case closing rate of customer service model A is 80%, the case closing rate of customer service model B is 70%, the case closing rate of customer service model C is 90%, and the case closing rate of customer service model D is 80%, then the average pass rate of platform G is 80%.

[0098] Embodiment 9:

[0099] If platform Z has four AI customer service models E, F, K, and Y, the case closure rate of the E customer service model is 80%, the case closure rate of the F customer service model is 70%, the case closure rate of the K customer service model is 60%, and the case closure rate of the Y customer service model is 70%, then the average pass rate of platform Z is 70%.

[0100] Embodiment 10:

[0101] If the J platform has three AI customer service models Q, I, and R, the case closing rate of the Q customer service model is 50%, the case closing rate of the I customer service model is 60%, and the case closing rate of the R customer service model is 65%, then the average pass rate of the J platform is 58.33%.

[0102] Embodiment 11:

[0103] If the case closure rate threshold is 80%, the sampling rate of the AI ​​customer service model is 1, and the monitored case closure rate is 75%, the case data will be collected. If the accuracy of the annotation result is lower than the accuracy of the answer result, the sampling rate of the AI ​​customer service model will be adjusted to 1.5.

[0104] Specifically, the average pass rate is compared with the pass rate threshold.

[0105] Among them, the pass rate threshold is the minimum standard of the average case closing rate of several AI customer service models.

[0106] By setting case closure rate thresholds and pass rate thresholds, the AI ​​customer service model is adjusted to reduce labor costs and increase the accuracy of answering customer needs.

[0107] Specifically, when the average pass rate is less than the pass rate threshold, the sampling rate of the AI ​​customer service model is adjusted secondary.

[0108] Embodiment 12:

[0109] If the pass rate threshold is 80%, the sampling rate of the AI ​​customer service model is 1.5, and the average pass rate is monitored to be 75%, then the sampling rate of the AI ​​customer service model will be adjusted to 2.

[0110] By adjusting the sampling rate of the AI ​​customer service model, the AI ​​customer service model is optimized, allowing users to get more accurate answers.

[0111] Specifically, several AI customer service models that have undergone secondary adjustments are ranked from high to low according to their pass rates, and the AI ​​customer service models are matched with sub-samples according to indicators.

[0112] By sorting the AI ​​customer service models after the second adjustment from high to low according to the pass rate, and matching the AI ​​customer service models with the labels of various types of questions, real-time adjustment of the AI ​​customer service models is achieved, enabling the AI ​​customer service models to quickly adapt to questions with corresponding labels, thereby reducing errors in the model's answers.

[0113] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A platform training method based on AI customer service, characterized in that: include: Read some customer service data as training samples and divide them into several sub-samples according to the preset sampling rate; The sub-samples will be used to train several AI customer service models respectively, and the sub-samples will be used to verify the training results; Determine the case closing rate of each AI customer service model based on the verification results; Set a case closing rate threshold, compare the case closing rate of each AI customer service model with the case closing rate threshold, and obtain the corresponding case closing comparison results; Determine the average pass rate of the AI ​​customer service platform based on the case closing comparison results; comparing the average pass rate with a pass rate threshold; When the average pass rate is less than the pass rate threshold, adjusting the sampling rate of the sub-sample; Calculate the case closing rates of several AI customer service models again and use them as new case closing rates to calculate the average pass rate of the AI ​​customer service platform. The average pass rate is the average of the case closing rates of several AI customer service models; comparing the average pass rate with the pass rate threshold, The pass rate threshold is the lowest standard of the average case closing rates of several AI customer service models; When the average pass rate is less than the pass rate threshold, a second adjustment is made to the sampling rate of the AI ​​customer service model; Sort the qualified rates of the several AI customer service models after the secondary adjustment from high to low, and correspond the AI ​​customer service models to the sub-samples according to several indicators; Matching the sub-samples with the AI ​​customer service models, and obtaining the case closing rate of the AI ​​customer service model for each sub-sample, and after completing the verification, selecting the AI ​​customer service model with a higher case closing rate; The case closing rate is the percentage of cases in which the training results are the same as the results of each case in the subsample after the AI ​​customer service model is used for training and the training results are obtained; The sampling rate is related to the recognition capability of the AI ​​customer service model.

2. The platform training method based on AI customer service according to claim 1, characterized in that: When dividing the sub-samples, the customer service data are sorted as the training samples according to the indicators. The indicators include the type, specialty and / or processing time of the customer service data.

3. The platform training method based on AI customer service according to claim 1, characterized in that: The steps of collecting the case closing rate of a single AI customer service model include: Corresponding to each sub-sample received by the single AI customer service model; Poll the number of cases answered by the customer service in the corresponding AI customer service model, and calculate the case closing rate of the AI ​​customer service model in the corresponding sub-sample; When the case closing rate of a single AI customer service model is lower than the case closing rate threshold, adjusting the sampling rate of the corresponding AI customer service model; The case closing rates of several AI customer service models are collected again and used as new case closing rates; Among them, a timer is provided in the AI ​​customer service model for polling the case closing status of the AI ​​customer service model at fixed intervals.

4. The platform training method based on AI customer service according to claim 3 is characterized in that: Compare the case closing rate with the case closing rate threshold to obtain the corresponding case closing comparison result, Among them, the case closing rate threshold is the minimum standard for the case closing rate of a single AI customer service model.

5. The platform training method based on AI customer service according to claim 4 is characterized in that: When the case closing rate of a single AI customer service model is lower than the case closing rate threshold, case data of the case is collected. The case data includes pictures, texts and / or sounds.

6. The platform training method based on AI customer service according to claim 5, characterized in that: The case data is automatically labeled using the AI ​​customer service model, the labeled results are compared with the answer results of the AI ​​customer service model, the sampling rate of the AI ​​customer service model is adjusted and the adjusted sampling rate is used as the new sampling rate.

Images