Intelligent voice customer service system based on AI

Abstract

The invention provides an AI-based intelligent voice customer service system, which comprises a real-time voice input and real-time interruption module used for real-time voice input and real-time voice interruption, and the real-time voice input comprises voice stream receiving and voice detection instruction execution; wherein the voice detection instruction comprises an activeness detection instruction and a non-activeness detection instruction, and comprises the following steps: when a VAD module detects that more than 70% of audio blocks in N continuous audio blocks are active voice blocks, judging that an active voice segment starts; comprising the steps that a VAD module detects that more than 70% of audio blocks in N continuous audio blocks are inactive voice blocks, and it is judged that an inactive voice segment starts; the method further comprises the steps of executing a network sending instruction and sending the input voice segment to a TTS module to execute a voice-to-text instruction; wherein the real-time interruption module comprises a voice active section start notification instruction; the problems that the answering efficiency of manual customer service staff is low, and switching of AI voice output and active voice detection of AI customer service staff is not smooth are solved.

Description

Technical Field

[0001] The present invention belongs to the fields of AI and IVR, and particularly relates to an AI-based intelligent voice customer service system. Background Art

[0002] In a telephone voice customer service system, labor cost has always been one of the main expenses. How to reduce the labor cost in the customer service system is a problem worthy of research. In recent years, with the development of artificial intelligence technology, intelligent telephone customer service has gradually been applied to the customer service systems of various enterprises to replace some manual customer service and improve the answering efficiency of manual customer service. On the other hand, existing AI voice customer service cannot respond to the user's voice input in a timely manner during AI voice output, and the user cannot interrupt the AI voice output, resulting in a poor user experience. Summary of the Invention

[0003] The purpose of this embodiment is to provide an AI-based intelligent voice customer service system, which is used to improve the answering efficiency of manual customer service and solve the problem that the user cannot interrupt the AI voice output in a timely manner when the AI voice output cannot respond to the user's voice input in a timely manner.

[0004] An AI-based intelligent voice customer service system includes:

[0005] A real-time voice input and real-time interruption module: used for real-time voice input and real-time voice interruption. The real-time voice input includes receiving a voice stream and executing a voice detection instruction. The voice detection instruction includes an activity detection instruction and an inactivity detection instruction. It includes that when the VAD module detects that more than 70% of the N consecutive audio blocks are active voice blocks, it is determined that the active voice segment starts; when the VAD module detects that more than 70% of the N consecutive audio blocks are inactive voice blocks, it is determined that the inactive voice segment starts. Further, it also includes executing a voice segment extension instruction, and the voice segment extension instruction includes connecting the T voice blocks before the active voice segment and the T voice blocks after the active voice segment to the front and back of the active voice segment to form an input voice segment. Further, it also includes executing a network sending instruction to send the input voice segment to the TTS module to execute a voice-to-text instruction. The real-time interruption module includes a voice active segment start notification instruction. When the real-time voice input module detects the start of an active voice segment, it sends an instruction to notify the large model output module to execute a large model output stop instruction, and sends an instruction to notify the AI voice playback module to stop the AI voice playback;

[0006] An IVR module: used for receiving user telephone key operations and parsing key values, including: a DTMF parsing module for parsing DTMF signals as user key inputs and a voice receiving module for receiving user voice data through the voice channel of the voice board card;

[0007] System Status Switching Module: Used for VAD detection of voice and DTMF status switching;

[0008] Large Language Model Output and Real - time Speech Synthesis Module: Used to convert the output text of the large language model (LLM) into text sentences and execute speech synthesis output sentence by sentence by the speech synthesis module.

[0009] Furthermore, executing the voice detection instruction also includes that if an active voice segment starts to be recognized first and then an inactive voice segment starts to be recognized, it is determined that the active voice segment ends.

[0010] Furthermore, T includes 5 and N includes 10.

[0011] Furthermore, the large model output stop instruction includes stopping receiving the large model output statement to convert to TTS audio instruction and the audio volume intensity decreasing instruction.

[0012] Furthermore, the audio volume intensity decreasing instruction includes decreasing the volume by 50% of the previous volume intensity each time.

[0013] Furthermore, the VAD module includes the webrtcvad module in the python scenario.

[0014] Furthermore, the IVR module also includes starting the VAD detection thread when it detects the end key while responding to the user's key input DTMF string.

[0015] Furthermore, the system status switching module also includes switching to receive a DTMF delay decoder for DTMF decoding when detecting DTMF input.

[0016] Furthermore, the large language model output also includes executing a natural language sentence - breaking instruction to output independent broken - sentence sentences, where the natural language sentence - breaking instruction includes completing the instruction execution by the HanLP library in the python scenario.

[0017] Furthermore, the real - time speech synthesis module also includes generating speech audio according to the independent broken - sentence sentences by executing the speech synthesis instruction.

[0018] An AI - based intelligent voice customer service system provided by the present invention is used to solve the problems of low efficiency of manual customer service answering, inability to respond to the user's voice input in time during AI voice output, and the user's inability to interrupt the AI voice output.

[0019] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes detailed descriptions as follows. Brief Description of the Drawings

[0020] Figure 1: System framework diagram of the AI-based intelligent voice customer service system provided by the embodiments of the present invention. Detailed implementation manners

[0021] Next, the technical solutions in the embodiments of the present invention will be described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0022] The present invention provides an AI-based intelligent voice customer service system, aiming to improve the timeliness of user voice input feedback, increase the diversity and accuracy of user input forms.

[0023] All functions of this embodiment need to be cooperated with the IVR module of the voice board card to be realized.

[0024] An AI-based intelligent voice customer service system (see Figure 1 ), includes:

[0025] P1000: Real-time voice input and interruption module: For real-time voice input and real-time voice interruption. The real-time voice input includes receiving a voice stream and voice activity detection. Among them, the voice activity detection is used to detect the start and end of an active voice segment. According to the network connection with the IVR module of the voice board card, continuously receive the uplink audio data of the call user sent by the IVR module. When acquiring audio data, use the VAD (voice activity detection) function module to detect each received audio block, and judge whether the audio in this segment is active voice. If so, it is called an active voice block, otherwise it is called an inactive voice block. Specifically, when implementing, 10 consecutive audio blocks are divided into a group. When the VAD module determines that more than 70% of the audio blocks in the latest group of audio blocks are active voice blocks, it is judged that the active voice segment (an active voice segment contains several consecutive audio blocks) starts; when the VAD module determines that more than 70% of the voice blocks in the latest group of audio blocks are inactive voice blocks, it is determined that the inactive voice segment starts. If the start of the active voice segment is recognized first and then the start of the inactive voice segment is further recognized, it is considered that the active voice segment ends. All the voice blocks from the start to the end of the active voice segment are connected to form a complete active voice segment. Specifically, when implementing, in order to make the active voice segment more complete, appropriately extend the beginning and end of the active voice segment, that is, extend the beginning of the recognized active voice segment forward by 5 voice blocks (if any), and extend the end backward by 5 voice blocks (if any) to ensure the integrity of the obtained active voice blocks. When a complete active voice segment is obtained, execute the network instruction to send it to the speech recognition module for speech recognition.

[0026] During the process of the system executing the answer output of the large language model (including the text output of the large language model, sentence segmentation of the text cache, conversion of sentences to TTS audio, and playback of TTS audio), if the voice activity detection module detects the start of an active voice segment, that is, the voice input starts, the system immediately interrupts the current answer output receiving process of the large language model, stops converting the sentences of the large language model answer to TTS audio, clears the TTS synthesized voice prepared for playback, immediately stops the synthesized voice being played or gradually reduces the volume of the synthesized voice being played. Specifically, in implementation, it is reduced once every 0.5 seconds, and each time the volume is reduced to 50% of the previous volume until it is muted and finally stopped. After the voice activity detection module subsequently detects the end of the active voice segment, the newly recognized active voice segment is provided as new input to the speech recognition module, and the speech recognition module converts it into text and then executes the network submission instruction to send it to the background large language model. It should be noted in the embodiment that when the user generates a new active voice segment, the previous text request sent to the large language model is not interrupted. This is to enable the large language model to fully understand each input of the user, obtain complete context information, and facilitate better understanding of the user's request context.

[0027] In the voice input processing process of this embodiment, the VAD (Voice Activity Detection) technology is used. The VAD module of the open source WebRTC project is selected for implementation (the webrtcvad library for the Python platform, and the installation instruction is to install with "pip install webrtcvad"). Using VAD can identify the active segments and non-active segments of the user's voice in real time from the user input voice stream, so as to cut the audio stream into several segments according to the voice activity. After each active voice segment is segmented, the system can perform speech recognition processing without additional operations by the user, which greatly improves the convenience and immediacy of the user's voice input.

[0028] P1001: IVR Module: It is used to receive users' telephone key operations and users' upstream voice. By default, the module receives the upstream voice data of telephone users as input. When it is necessary for users to input some precise information, DTMF signals are used as input. When it is necessary for users to input precise information such as ID numbers, mobile phone numbers, and SMS verification codes, the IVR module of the voice board is notified to play a voice prompt tone and wait for the user to input a call key signal, that is, a DTMF signal. Further, when the IVR module of the voice board receives a DTMF signal, it executes a network request to send the DTMF key digits (in specific implementation, the IVR module of the voice board and this embodiment are connected through websocket, the voice data is transmitted through binary streams, while the DTMF signal is transmitted in the form of a string). When the precise information input is completed (usually, it is required that the user input a specific key, such as the "#" key, to indicate the end of the input), this customer service system automatically restarts the reception of voice data. In specific implementation, the reception of voice data and the detection of voice activity are enabled.

[0029] P1002: System State Switching Module, which is used for switching between the voice input state and the DTMF input state. In specific implementation, the user input type is automatically switched according to the state of the conversation. When it is necessary for the user to input key information, it automatically switches to the DTMF signal channel and starts the DTMF delay decoder to perform DTMF decoding. When the user finishes inputting the DTMF signal, it automatically switches back to the voice channel. The end of DTMF is identified by receiving a specific DTMF character, realizing the automatic and seamless switching between the voice channel and the DTMF channel, ensuring the convenience and precision of the user input.

[0030] P1003: Large Language Model Output and Real - time Speech Synthesis Module: It is used to convert the output text of the large language model (LLM) into independently segmented sentences and perform speech synthesis sentence by sentence. The segmentation method in specific implementation is as follows: The text output by the large model is sent into a text buffer, and all the text in the buffer is sent to the "Natural Language Sentence Segmentation Module" (in this embodiment, the HanLP library of Python is used, installed through "pip install hanlp") for sentence segmentation. When the output of the large model is not completed, several sentences except the last one (the last sentence may be incomplete, so it is not taken for speech synthesis temporarily) of the segmentation result are used as the sentences to be synthesized into speech, and are sent to the speech synthesis module sentence by sentence for speech synthesis. Further, when the large model finishes executing the text output, all the sentences of the segmentation result are directly used as the sentences to be synthesized into speech, and are sent to the speech synthesis module sentence by sentence for speech synthesis. This ensures that when generating speech, the speech synthesis module can have a certain context environment, thus generating relatively high - quality speech. At the same time, it can output speech relatively real - time during the text output process of the large model, reducing the delay of audio output and improving the user's auditory experience.

[0031] An AI-based intelligent voice customer service system provided by the present invention solves the problems of low answering efficiency of manual customer service and the inability to respond to the user's voice input in a timely manner during AI voice output, and the user cannot interrupt the AI voice output.

[0032] The above are only embodiments of the present invention and are not intended to limit the protection scope of the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An AI-based intelligent voice customer service system, characterized in that: include: Real-time voice input and real-time interruption module: used for real-time voice input and real-time voice interruption, wherein the real-time voice input includes receiving a voice stream and executing a voice detection instruction, wherein the voice detection instruction includes an activity detection instruction and an inactivity detection instruction, including a VAD module detecting that more than 70% of the audio blocks in N consecutive audio blocks are active voice blocks, and determining that an active voice segment begins; It includes a VAD module detecting that more than 70% of the audio blocks in N consecutive audio blocks are inactive voice blocks, and determining that an inactive voice segment has started; it further includes executing a voice segment extension instruction, and the voice segment extension instruction includes connecting T voice blocks before the active voice segment and T voice blocks after the active voice block with the active voice segment to form an input voice segment; it further includes executing a network sending instruction to send the input voice segment to a TTS module to execute a voice-to-text instruction; wherein the real-time interruption module includes a voice active segment start notification instruction, and when the real-time voice input module detects the start of an active voice segment, it sends an instruction to notify the large model output module, executes the large model output stop instruction, and sends an instruction to notify the AI ​​voice playback module to stop the AI ​​voice playback; IVR module: used to receive user telephone key operations and analyze key values, including: DTMF analysis module, used to analyze DTMF signals as user key input and voice receiving module, used to receive user voice data through the voice channel of the voice board; System state switching module: used for voice VAD detection and DTMF state switching; Large language model output and real-time speech synthesis module: The output text of the large language model (LLM) is converted into text sentences and the speech synthesis module performs speech synthesis output sentence by sentence.

2. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The executing voice detection instruction also includes determining that the active voice segment ends if the start of an active voice segment is first recognized and the start of an inactive voice segment is further recognized.

3. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The T includes 5, and the N includes 10.

4. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The large model output stop instruction includes a stop receiving large model output sentence conversion TTS audio instruction and an audio volume intensity decreasing instruction.

5. The intelligent voice customer service system in the IVR scenario according to claim 4, characterized in that: The audio volume intensity decreasing instruction includes decreasing the volume by 50% of the previous volume intensity each time.

6. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The VAD module includes a webrtcvad module in a python scenario.

7. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The IVR module also includes a method for starting a VAD detection thread in response to a user inputting a DTMF string and detecting an end key.

8. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The system state switching module also includes switching the receiving DTMF delay decoder to perform DTMF decoding when detecting DTMF input.

9. The intelligent voice customer service system in the IVR scenario according to claim 1, characterized in that: The large language model output also includes executing natural language segmentation instructions to output independent segmented sentences, wherein the natural language segmentation instructions include the HanLP library in the python scenario to complete the instruction execution.

10. The intelligent voice customer service system in the IVR scenario according to claims 1 and 9, characterized in that: The real-time speech synthesis module also includes executing speech synthesis instructions according to independent segmented sentences to generate speech audio.

Images