Intelligent speech recognition enhancement system based on plug-in

Abstract

The invention discloses an intelligent voice recognition enhancement system based on a plug-in. The intelligent voice recognition enhancement system comprises a call center platform, a plug-in component and an Asr service module, the call center platform, the Asr service module and the plug-in component are communicated, and the plug-in component comprises a Cc service module, an intelligent processing module and an audio data stream module; the intelligent processing module is communicated with the Cc service module and the audio data stream module, and the intelligent processing module comprises a Cc client module, an audio module and an Asr client module; the Cc client module is communicated with the Cc service module, and the Asr client module is communicated with the Asr service module; according to the intelligent voice recognition enhancement system based on the plug-in, the lightweight plug-in is installed on a computer of a customer service representative to achieve a real-time voice transcription function so as to efficiently assist the customer service representative.

Description

Technical Field

[0001] The present invention relates to the technical field of web information enhancement, and more specifically, to an intelligent speech recognition enhancement system based on a plug-in. Background Art

[0002] With the rapid development of information technology, call centers, as an important bridge for communication between enterprises and customers, are also constantly evolving and developing. Although traditional call center systems can meet basic needs for answering and making calls, they often seem powerless in the face of growing customer needs and business challenges. Especially for those customers who are still using old third-party call systems, it is difficult for them to directly upgrade or replace existing call center solutions due to various reasons (such as cost, compatibility, or protection of existing investments). Therefore, it is particularly necessary to develop a plug-in intelligent agent assistance tool that can be quickly integrated into the existing system. Summary of the invention

[0003] The present invention overcomes the shortcomings of the prior art and provides a plug-in-based intelligent speech recognition enhancement system that realizes a real-time speech transcription function by installing a lightweight plug-in on the computer of a customer service representative to efficiently assist the customer service representative.

[0004] The technical solution of the present invention is as follows:

[0005] A plug-in-based intelligent speech recognition enhancement system includes a call center platform, a plug-in component and an Asr service module; the call center platform, the Asr service module and the plug-in component are connected, and the plug-in component includes a Cc service module, an intelligent processing module and an audio data stream module; the intelligent processing module is connected to the Cc service module and the audio data stream module, wherein the intelligent processing module includes a Cc client module, an audio module and an Asr client module; the Cc client module is connected to the Cc service module, and the Asr client module is connected to the Asr service module;

[0006] The Asr service module is a third-party automatic speech recognition service system;

[0007] The Asr client module connects to the Asr service module to realize the voice-to-text function;

[0008] The Cc service module serves as a bridge between the Cc customer module and the call center platform;

[0009] The Cc client module is responsible for receiving control instructions from the Cc service module and passing them to other component modules;

[0010] The audio data stream module is used to monitor the sound input / output of the microphone and speaker and collect call data;

[0011] The intelligent processing module integrates the Asr client module, Cc client module, and audio data stream module to provide intelligent auxiliary functions;

[0012] The audio module is used to monitor the computer's microphone and speakers and capture the voice stream generated during a call.

[0013] Further, the specific steps for using the system are as follows:

[0014] 1) Initialization step: Initialize the Cc client module, create an instance and establish a connection with the call center platform through the Cc service module;

[0015] 2) Monitoring step: monitoring the voice stream data of microphone input and speaker output required by the audio module through the audio data stream module;

[0016] 3) Audio processing step: The Asr client module pre-processes the voice stream data, and then sends the processed audio stream data to the Asr service module for voice-to-text conversion to form text data;

[0017] 4) Text processing step: The intelligent processing module analyzes and identifies text data through NLP technology, extracting keywords and customer intentions from the text data;

[0018] 5) Auxiliary suggestion step: Based on keywords and customer intentions, the intelligent processing module analyzes and generates a call summary, determines the customer's potential needs, and provides corresponding suggestions or solutions; the results are displayed to the customer service representative through the Cc service module to assist the customer service representative in completing the task.

[0019] Furthermore, when the Cc client module is initialized, it will load the global configuration file and establish a secure WebSocket connection with the business-side service of the call center based on the information in the configuration file, using the wss protocol.

[0020] Furthermore, it also includes a heartbeat mechanism, which sends heartbeat judgment packets regularly through the member object CcHeartBeatTimer, that is, the timer sends judgment information to the service side of the call center at every time interval specified in the global configuration to maintain the validity of the connection.

[0021] Furthermore, the text data is pushed to the call center in real time.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] The present invention installs a lightweight plug-in on the computer of the customer service representative. When there is an incoming call, the plug-in can monitor the microphone and speaker of the computer, capture the voice stream data generated during the call, and send it to a special ASR service module for voice recognition. The recognized text content can be pushed to the customer service representative end of the call center platform through the interface provided by this plug-in, and finally the real-time text data, corresponding suggestions or solutions will be presented on the web browser interface used by the customer service representative, so as to achieve efficient assistance to the customer service representative in responding to customer needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall framework of the present invention;

[0025] Figure 2 It is a schematic diagram of the overall process framework of the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and cannot be used as a limitation of the present invention.

[0027] It will be appreciated by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and unless defined as herein, will not be interpreted in an idealized or overly formal sense. Where not described in detail in this embodiment, conventional technical means can be used for implementation, so no further description will be given.

[0028] The numbers of the steps mentioned in each embodiment are only for the convenience of description and have no substantial sequence relationship. Different steps in each specific embodiment can be combined in different sequences to achieve the purpose of the invention.

[0029] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0030] like Figure 1As shown, a plug-in-based intelligent speech recognition enhancement system includes a call center platform (i.e., a traditional old call center), a plug-in component, and an Asr service module. The call center platform, the Asr service module, and the plug-in component are connected, and the plug-in component includes a Cc service module, an intelligent processing module agentAssistant, and an audio data stream module. The intelligent processing module is connected to the Cc service module and the audio data stream module, wherein the intelligent processing module includes a Cc client module CcClient, an audio module audioDevice, and an Asr client module AsrClient. The Cc client module is connected to the Cc service module, and the Asr client module is connected to the Asr service module.

[0031] The ASR service module is a third-party automatic speech recognition service system.

[0032] The Asr client module connects to the Asr service module to realize the speech-to-text function.

[0033] The Cc service module serves as a bridge between the Cc customer module and the call center platform.

[0034] The Cc client module is responsible for receiving control instructions from the Cc service module and passing them to other component modules.

[0035] The audio data stream module is used to monitor the sound input / output of the microphone and speaker and collect call data.

[0036] The intelligent processing module integrates the Asr client module, Cc client module, and audio data stream module to provide intelligent auxiliary functions such as automatic summarization and customer intent recognition.

[0037] The audio module is used to monitor the computer's microphone and speakers and capture the voice stream generated during a call.

[0038] like Figure 2 The specific steps for using the system are as follows:

[0039] 1) Initialization step: Initialize the Cc client module, create an instance and establish a connection with the call center platform through the Cc service module. That is, create a CcClient instance and establish a connection with the business layer of the call center platform, so that the call center platform can call this enhanced system.

[0040] In specific implementation, when the Cc client module is initialized, it will load the global configuration file and establish a secure WebSocket connection with the business side service of the call center based on the information in the configuration file. The protocol uses wss. At the same time, in order to ensure the stability of the connection, a heartbeat mechanism is implemented to periodically send heartbeat judgment packets through the member object ccHeartBeatTimer. The timer sends judgment information to the business side of the call center platform at the time interval specified in the global configuration to maintain the validity of the connection. The specific definition of the heartbeat mechanism is as follows:

[0041] class HeartBeatTimer{

[0042] public:

[0043] HeartBeatTimer() = delete;

[0044] HeartBeatTimer(CcClient&ccClient,int64_t interval_sec);

[0045] void on_timer(const boost::system::error_code&ec);

[0046] void start();

[0047] private:

[0048] CcClient&m_client;

[0049] boost::asio::steady_timer m_timer;

[0050] int64_t m_interval = 5;

[0051] void timer_reset();

[0052] };

[0053] This mechanism uses the steady_timer of the boost::asio library to time, with a default interval of 5s. The on_timer method is bound to the async_wait method of m_timer as the callback function for the timer expiration. In on_timer, a ping message is sent to the business side. After the message is sent, the timer_reset() method is called internally to reset the timer. This cycle repeats to ensure that the connection with the business side is normal. The entire mechanism needs to call the start() method to start timing.

[0054] The message_handler member function on_message is bound to the CcClient object to receive messages from the business side of the call center platform.

[0055] The initialization also includes the audio module audioDevice, which is responsible for detecting the audio device (microphone and speaker) of the agent computer and initializing the device management object according to the global configuration. When the Asr client module AsrClient is initialized, the relevant global parameters are set to interact with the third-party Asr service module.

[0056] 2) Monitoring step: Start the audio module and monitor the voice stream data of the microphone input and the speaker output that the audio module needs to obtain through the audio data stream module.

[0057] 3) Audio processing step: When the service side of the call platform center detects that the agent has an incoming call and the call is successfully answered, the on_message function will receive a message with msg_type "in_call". The message contains other important information required for this session, such as temporary verification parameters of the third-party ASR service.

[0058] After receiving the "in_call" message, the audio module audioDevice is activated and starts to monitor the voice stream data in the microphone and speaker. At this time, the input stream of the microphone is the voice of the agent, and the output stream of the speaker is the voice of the other party (customer). The Asr client module will preprocess the voice stream data to improve the accuracy of speech recognition. That is, these voice streams are passed to the member function audio_pre_process of the audio module audioDevice for preprocessing to meet the format requirements of the third-party ASR server for media streams. Among them, the audio_pre_process function performs necessary audio preprocessing operations, such as gain adjustment, noise suppression, resampling, etc., to ensure that the audio quality meets the requirements of the third-party ASR server. The preprocessed voice stream is sent to the buffer of the Asr service module asrClient for further processing. The processed audio stream data is sent to the Asr service module for voice-to-text conversion to form text data.

[0059] The third-party ASR server in the Asr service module converts the speech into text and returns the text result to the Asr client module asrClient.

[0060] The Asr client module asrClient sends the converted text results to the intelligent processing module agentAssistant module for further processing.

[0061] 4) Text processing step: The intelligent processing module analyzes and identifies the text data through NLP technology, and extracts keywords and customer intentions from the text data. That is, the intelligent processing module agentAssistant receives the real-time call text from the Asr client module asrClient. After receiving the real-time text, the intelligent processing module agentAssistant also immediately performs preliminary text processing, including but not limited to spelling check, grammar correction and format standardization, etc., to improve the accuracy of subsequent processing steps.

[0062] The processed real-time text data is pushed to the business side system of the call center platform so that business personnel can view the call content in real time.

[0063] 5) Assisted suggestion step: After the call is over, the intelligent processing module agentAssistant automatically collects all text records during the entire call and conducts a comprehensive analysis of them. This includes using natural language processing (NLP) technology to extract key information of the call, such as key events, emotional tendencies, product mentions, etc.

[0064] Using advanced natural language understanding algorithms, such as deep learning-based methods, call summaries are generated to quickly understand the main content of the call. Key information points in the call are also extracted, such as product models, service requests, order numbers, etc. Named entity recognition (NER) technology is used to mark specific entities, such as names of people, company names, dates, etc. This information is also integrated into easy-to-understand content through context understanding and association analysis.

[0065] The intelligent processing module uses semantic analysis and sentiment analysis technology to analyze the customer's intention and emotional state. It analyzes the customer's questioning patterns, keywords and phrases to identify potential needs or problems. It also applies machine learning models to identify customer intentions, such as consultations, complaints, suggestions, etc., and assess their urgency.

[0066] Sentiment analysis technology mainly analyzes the emotions expressed during calls, such as satisfaction, dissatisfaction, anxiety, etc. It evaluates the customer satisfaction level and takes appropriate follow-up measures accordingly. If a call with negative emotions occurs, an automatic alarm mechanism can be triggered to notify relevant personnel to intervene in time.

[0067] The intelligent processing module also generates corresponding suggestions or solutions. For example, if it detects that the customer has made a specific service request, the system can recommend a corresponding solution or service process. If potential product improvement suggestions are mentioned during the call, the system can mark them as to-do items for follow-up.

[0068] The intelligent processing module also includes compliance checks on call content to ensure that all communications comply with industry standards and regulatory requirements. Potential risk points, such as sensitive information leakage and inappropriate remarks, are identified and appropriate measures are taken to avoid risks.

[0069] The intelligent processing module will automatically generate a detailed call analysis report based on the above analysis results. The results will be presented to the customer service representative through the Cc service module to assist the customer service representative in completing the task. The report includes call summary, customer intention summary, key information extraction, intelligent suggestions, sentiment analysis results, etc. The analysis report and other processing results will be pushed to the business side of the call center platform for further analysis or decision-making.

[0070] Preferably, key information can also be synchronized to the enterprise's CRM system or other management system through an integrated interface to facilitate subsequent tracking and management.

[0071] In summary, this solution installs a lightweight plug-in on the customer service representative's computer. When there is an incoming call, the plug-in can monitor the computer's microphone and speaker, capture the voice stream data generated during the call, and send it to the dedicated ASR service module for voice recognition. The recognized text content can be pushed to the customer service representative end of the call center platform through the interface provided by this plug-in, and finally the real-time text data will be presented on the web browser interface used by the customer service representative, and corresponding suggestions or solutions will be given later, so as to efficiently assist the customer service representative to respond to customer needs.

[0072] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the concept of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A plug-in-based intelligent speech recognition enhancement system, characterized in that: It includes a call center platform, a plug-in component and an Asr service module; the call center platform, the Asr service module and the plug-in component are connected, and the plug-in component includes a Cc service module, an intelligent processing module and an audio data stream module; the intelligent processing module is connected with the Cc service module and the audio data stream module, wherein the intelligent processing module includes a Cc client module, an audio module and an Asr client module; the Cc client module is connected with the Cc service module, and the Asr client module is connected with the Asr service module; The Asr service module is a third-party automatic speech recognition service system; The Asr client module connects to the Asr service module to realize the voice-to-text function; The Cc service module serves as a bridge between the Cc customer module and the call center platform; The Cc client module is responsible for receiving control instructions from the Cc service module and passing them to other component modules; The audio data stream module is used to monitor the sound input / output of the microphone and speaker and collect call data; The intelligent processing module integrates the Asr client module, Cc client module, and audio data stream module to provide intelligent auxiliary functions; The audio module is used to monitor the computer's microphone and speakers and capture the voice stream generated during a call.

2. The plug-in-based intelligent speech recognition enhancement system according to claim 1, characterized in that: The specific steps for using the system are as follows: 1) Initialization step: Initialize the Cc client module, create an instance and establish a connection with the call center platform through the Cc service module; 2) Monitoring step: monitoring the voice stream data of microphone input and speaker output required by the audio module through the audio data stream module; 3) Audio processing step: The Asr client module pre-processes the voice stream data, and then sends the processed audio stream data to the Asr service module for voice-to-text conversion to form text data; 4) Text processing step: The intelligent processing module analyzes and identifies text data through NLP technology, extracting keywords and customer intentions from the text data; 5) Auxiliary suggestion step: Based on keywords and customer intentions, the intelligent processing module analyzes and generates a call summary, determines the customer's potential needs, and provides corresponding suggestions or solutions; the results are displayed to the customer service representative through the Cc service module to assist the customer service representative in completing the task.

3. The plug-in-based intelligent speech recognition enhancement system according to claim 2, characterized in that: When the Cc client module is initialized, it will load the global configuration file and establish a secure WebSocket connection with the business-side service of the call center based on the information in the configuration file. The protocol uses wss.

4. The plug-in-based intelligent speech recognition enhancement system according to claim 3, characterized in that: It also includes a heartbeat mechanism, which sends heartbeat judgment packets regularly through the member object CcHeartBeatTimer, that is, the timer sends judgment information to the service side of the call center at every time interval specified in the global configuration to maintain the validity of the connection.

5. The plug-in-based intelligent speech recognition enhancement system according to claim 2, characterized in that: Text data is pushed to the call center in real time.

Images