Online learning system by automatic extraction of ai failure span and operator correction differential

Abstract

To automatically extract a failure part in an AI voice response and to immediately reflect a differential from a correction response of an operator on learning.SOLUTION: The present invention includes a failure span detection unit configured to evaluate a reliability, a BLEU score, and an emotion score of an AI response by using a speech recognition result and extract a section less than a threshold value as a failure span, a difference learning unit configured to generate a difference from an operator response in a JSON format, and an update unit configured to update the model online by a LoRA adapter for a certain number of cases or a certain time. Thus, the AI response is immediately improved and the learning effect is improved. Selection Figure-1 SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001]

[0001] The present invention relates to a technology for linking an AI voice response system with a human operator, and in particular to a system that automatically extracts "failure spans" where an AI fails to generate a response, and updates an AI model in real time using the difference between the failed span and the operator's corrected response as learning data. [Background technology]

[0002]

[0002] Conventional AI voice response systems (e.g., Amazon Connect + Lex, Google Dialogflow) have the function of transferring calls to an operator when it is difficult to respond, but they have the following issues: [0002A] JP 2020-123456 (NTT) proposes post-transfer log learning, but does not disclose automatic tagging of failure locations or differential learning.

[0003] 1. Don't extract "where the AI ​​failed" as structured data 2. Operator correction responses are not used as "differences" for learning. 3. Learning is a batch process (after a few hours to a few days), so immediate improvement is not possible.

[0003] Patent Publication No. 2020-123456 (NTT) proposes log learning after transfer, but does not disclose automatic tagging of failure points or differential learning. [0003A] The disclosed technologies include individual elements such as (i) handoff from a bot to an operator, (ii) detection of response failures based on emotion scores and confidence, and (iii) a mechanism for using the operator's responses for subsequent learning. However, there is no disclosure of a process flow that integrates these elements, including (1) extracting failure spans as time intervals based on voice, (2) associating the failure spans with the immediately following operator responses and storing the differences in machine-readable JSON format, and (3) updating the model online using a lightweight adapter such as LoRA when triggered by a certain number of occurrences or a certain period of time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] (Patent Document 1) JP 2020-123456 A (Nippon Telegraph and Telephone Corporation) [Non-patent literature]

[0005] [Non-Patent Document 1] (Non-patent document 1) Amazon Web Services, "Amazon Connect Integration with Lex", technical document, 2020. (Non-patent document 2) Google Developers, "Dialogflow CX - Fallback Handling", 2021. (Non-patent document 3) Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models", arXiv:2106.09685, 2021. Summary of the Invention [Problem to be solved by the invention]

[0006]

[0004] The object of the present invention is to Automatically extracting AI failure points (failure spans) on a per-speech basis The "difference" between the operator's corrective response and the real-time response is used as learning data The goal is to instantly improve AI through online learning in under 5 minutes.

[0007] [0004A] In particular, the present invention is characterized in that it does not simply target failed AI response points for relearning, but extracts the failed intervals using time information, structures the association with the operator's response, and immediately improves the model using lightweight learning (LoRA). This makes it possible to precisely identify the erroneous response points of the AI ​​and relearn them using minimal difference information, simultaneously realizing faster learning processes and immediate improvements in response accuracy. [Means for solving the problem]

[0008] Means to solve the problem The system of the present invention comprises the following modules: 1. Speech recognition module: Real-time transcription using Whisper (OpenAI) 2. AI response module: Llama-3-8B-Instruct (fine-tuned) 3. Failed Span Detection Module (the core of the present invention) 4. Transfer Control Module 5. Differential Learning Module (the core of the present invention) 6. Online Learning Modules

[0009] Failed Span Detection Module

[0006] After generating the AI ​​response, the failure span (start and end times) is automatically extracted based on the following three conditions: 1. Intent classification confidence < 0.7 2. Response generation BLEU score (self-reference) < 0.3 3. User utterance emotion score (anger) > 0.6

[0007] Example: User: "The charge is wrong!!" (Anger score 0.8) AI: "Sorry. Can you repeat that?" (BLEU=0.2) → Failure Span = [00:15.2 ~ 00:18.7]

[0010] Differential Learning Module

[0008] After the operator responds, the AI ​​response and the operator response are paired and the differences are extracted. As shown in Figure 5, this module extracts the content from the item, the failure span, the audio file + transcription, the AI ​​response (failure), "Can you repeat that?", Operator Response (correct), "I'd like to review your billing details from the previous month. Would you like to share your screen?",Diff, and Add: Associate and process data in the order of "Screen Share" suggestions.

[0009] The differential data format is shown in Fig. 6. Fig. 6 is an example of differential data structured in JSON format, and shows the correspondence between each item (input, ai_failed, operator_correct, failure_span, label).

[0011] Online Learning Modules

[0010] After 5 minutes have passed since 10 differential data items were accumulated or transferred, update Llama-3 online using the LoRA adapter. After the update, the update will be applied immediately from the next call. [Effects of the Invention]

[0012]

[0011] According to the present invention, the following effects are expected. 1. By structuring failure points, learning efficiency is expected to improve by more than three times compared to conventional methods. 2. Differential learning is predicted to reduce the recurrence rate of similar errors by more than 90%. 3. Updates within 5 minutes are expected to improve service on the same day.

[0013] [0011A] Furthermore, the present invention is characterized in that it not only targets failed parts of AI responses for relearning, but also extracts failure spans as time information, structures the correspondence with operator responses, and then instantly updates the AI ​​model using lightweight learning (LoRA). This configuration clarifies the causes of failures in AI voice responses and enables real-time model improvement with minimal correction data. [0011B] With these configurations, the present invention realizes an "instant improvement cycle of AI responses" that was not possible with conventional technologies, and enables continuous optimization of AI response quality in actual operational environments, particularly in call centers and interactive business support. [Brief explanation of the drawings]

[0014] [Figure 1] Overall system configuration block diagram [Figure 2] Failed span detection sequence diagram [Figure 3] Flowchart of differential learning and online update [Figure 4] LoRA adapter update image [Figure 5] Example of differential learning data [Figure 6] JSON structure of differential data DETAILED DESCRIPTION OF THE INVENTION

[0015]

[0013] Below, an embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to these embodiments, and various modifications and improvements can be made by those skilled in the art.

[0016] (1) Overall system configuration As shown in FIG. 1, the AI ​​voice response system 100 of the present invention includes: A voice recognition module 101, an AI response module 102, a failure span detection module 103, It is composed of a transfer control module 104, a differential learning module 105, and an online learning module . Each of these modules is implemented in software on a cloud server, a local server, or a VPS environment.

[0017] (2) Voice Recognition Module

[0015] The voice recognition module 101 receives user voice data during a call in real time; Convert the speech into text data using a speech recognition model (e.g., Whisper). The results are recorded as text containing time-series information (timestamps), This is used to extract failed spans in subsequent processing.

[0018] (3) AI response module

[0016] The AI ​​response module 102 takes the above character string data as input and generates a natural language response. For generation, we use large-scale language models such as Llama-3-8B-Instruct, Fine-tuned parameters ensure consistency in dialogue responses. When generating a response, the intention classification confidence score and response self-concordance score (BLEU value) are calculated.

[0019] (4) Failed Span Detection Module After generating the AI ​​response, the failed span detection module 103: Speech sections that meet any of the following conditions are automatically extracted as "failed spans." Condition Threshold Intent classification confidence < 0.7 Response BLEU score (self-reference) < 0.3 User utterance emotion score (anger) > 0.6 The extracted failure spans are stored as a log along with the audio time information (start time, end time). It is linked to the content of the corresponding user utterance and AI response.

[0020] (5) Transfer control module

[0018] When a failed span is detected, the transport control module 104: Transfer calls to a human operator in real time. When forwarding, the audio text and sentiment analysis results for the span are added. This allows operators to quickly understand the content.

[0021]

[0019] The differential learning module 105 compares the operator responses recorded after the transfer with It is paired with the corresponding AI failure response and the difference between the two is extracted. This difference is saved in JSON format with the following structure: { "input": "Incorrect billing", "ai_failed": "Can you try again?", "operator_correct": "I'd like to review your statement from the previous month. Would you like to share your screen?" "failure_span": [15.2, 18.7], "label": "add_screen_share" }

[0022] (7) Online learning modules The online learning module 106 determines whether the number of differential data items reaches a predetermined number (for example, 10 items) or Or after a certain time (e.g. 5 minutes) has passed since the transfer, Lightweight fine tuning (LoRA adapter method) is applied. Partially update AI model parameters. Updates are reflected in real time, and improvements are immediately reflected from the next call.

[0023] (8) System operation example

[0021] An example of actual operation is shown below. User: "The charge is wrong!" AI: "Can you repeat that?" → Failure span detection (BLEU=0.2, sentiment score 0.8) The call is transferred to an operator Operator: "I'd like to review your bill from last month. Would you like to share your screen?" Extract differences → Register JSON → Update LoRA The next time you make a similar call, the AI ​​will correctly answer "Do you want to share your screen?" (Improvement reflected)

[0024] (9) Application form

[0022] In addition to call center operations, the present invention can also be used for Chat response, medical consultation support, financial inquiries, and local government support, etc. It is applicable to various fields requiring voice or text response. In addition, the definition of failure span is not limited to the above threshold, It can also be extended to dynamic evaluation based on topic similarity and dialogue history scores.

[0025] (10) Summary

[0023] With the above configuration, the present invention identifies the location of the AI ​​response failure along with time information, This makes it possible to instantly reflect the differences between the corrective responses of human operators in learning. As a result, the self-improvement cycle of the AI ​​voice response system has been shortened to just a few minutes. This has the remarkable effect of improving response accuracy on the day of operation. [Example]

[0026]

[0012] 1. User: "The billing is wrong!" 2. AI: "Can you repeat that?" → Failed span detection 3. Transfer → Operator : As in, “Shall I check with screen sharing?” 4. Generate differential data → Update LoRA 5. Next time the same user says, "The billing is wrong!" → AI says, "Would you like to check by sharing your screen?" (Immediate improvement) The present invention is not limited to screen sharing, but can also be applied to various corrective responses by operators, such as sending short messages or sharing PDF documents. [Industrial Applicability]

[0027]

[0024] The AI ​​voice response system of the present invention is suitable for use in call centers, contact centers, Customer support, medical consultation service, government service information, financial transaction inquiries, educational support, and other interactive business support, and can be applied to a wide range of industrial fields.

[0025] In particular, in industrial fields where dialogue processing using voice recognition and natural language response is performed, When AI gives an incorrect response, the work of correcting it is a huge burden in terms of human costs. According to the present invention, the failure points of AI are automatically extracted, The difference between the correct answer and that of a human operator can be reflected in immediate learning as structured data. Response accuracy can be continuously improved while reducing human resources costs.

[0026] In addition, by using lightweight learning techniques such as LoRA (Low-Rank Adaptation), The learning process can be performed quickly in a GPU cloud or local environment. There is no need to run a high-performance server all the time. For this reason, even small and medium-sized businesses and local government agencies The present invention can be introduced and operated at low cost.

[0027] Furthermore, the present invention is not limited to audio input and output, It can also be applied to text dialogue systems such as chatbots, automatic email responses, and automatic FAQ generation. It is effective in any industrial field that has dialogue data.

[0028] Therefore, the present invention provides As an "autonomous learning dialogue system that continues to improve AI response quality in real time," Information and communications industry, customer support industry, education industry, medical and nursing care field, etc. It has wide industrial applicability. [Explanation of symbols]

[0028] Figure 1. Overall system configuration block diagram Figure 2. Failed span detection sequence diagram Figure 3: Flowchart of differential learning and online update Figure 4 LoRA adapter update image Figure 5. Example of differential learning data Figure 6 JSON structure of differential data

Claims

[Claim 1] Claim 1 An AI voice response system comprising: a failure span detection means for automatically extracting, when an AI response module processes a voice-recognized user utterance, a time interval in which the response generation reliability is less than a predetermined value as a "failure span"; a difference learning means for generating learning data that is the difference between the failure span and the operator's corrected response corresponding to the failure span; and an update means for updating an AI model through online learning using the learning data. Claim 2 2. The system of claim 1, wherein the failed span detection means uses at least one of an intent classification confidence, a response BLEU score, and an emotion score. Claim 3 The system according to claim 1 or 2, wherein the update means performs fine tuning using the LoRA adapter within 5 minutes or when 10 cases have been accumulated.

Images