System and method for awakening intelligent assistant by voice in real-time call

Abstract

The invention discloses a system and a method for awakening an intelligent assistant by voice in real-time conversation, and the system is characterized in that a switching center server is used for binding channel IDs for a calling channel and a called channel respectively, initiating two port distribution requests to a voice awakening server, and after the two ports are obtained, sending the port distribution requests to the voice awakening server; data packets of a calling channel and a called channel are mapped to a port of the voice wake-up server, the real-time call voice stream monitoring and analyzing module is used for monitoring the port on the voice wake-up server, and the received data packets are sent to the user-defined voice wake-up training and detecting module; the self-defined voice wake-up training and detection module judges and wakes up the intelligent assistant three-party call module, and the intelligent assistant three-party call module is used for reconstructing a three-party call channel on the basis of the original two-party call and enabling both the calling party and the called party to interact with the intelligent assistant. According to the method and the device, the problems of realizing customized word voice wake-up in a call process, constructing a three-party session channel and accurately arousing the auxiliary service of the intelligent assistant in real time are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of voice wake-up during real-time calls, and in particular to a system and method for voice wake-up of an intelligent assistant during real-time calls. Background Art

[0002] With the continuous development of big model technology, smart assistants and applications have begun to appear in various scenarios and services. Due to the improvement of the accuracy of big model responses, users' acceptance and dependence on smart assistants have greatly increased, and they tend to seek help from smart assistants in various scenarios. For graphic and text interaction scenarios such as web pages and hardware integrated devices, the access to smart assistants is relatively simple. Whether it is clicking on a page button or switching on a device, the purpose of accessing a smart assistant for interaction can be easily achieved.

[0003] However, in the field of telephone communications, how to seamlessly access intelligent assistants during a real-time call between two parties, and provide real-time consultation, language translation, call summary shorthand and other functions for both parties to communicate on issues, is still in the product form exploration stage. In this context, to achieve seamless access to intelligent assistants during real-time telephone communications, it is necessary to find a way to directly call up intelligent customer service through the call voice, and at the same time establish a three-party call so that the intelligent assistant can play an auxiliary role in the call.

[0004] Therefore, there is an urgent need for a system and method for voice awakening an intelligent assistant during a real-time call to solve the problem of how to implement customized word voice awakening during a call, build a three-party conversation channel, and awaken the intelligent assistant auxiliary service in real time and accurately. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a system and method for voice awakening an intelligent assistant during a real-time call, so as to solve the problem of how to implement customized word voice awakening during a call, and build a three-party conversation channel to accurately awaken the auxiliary service of the intelligent assistant in real time.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A system for voice waking up an intelligent assistant in a real-time call, characterized in that it includes: a switching center server, a real-time call voice stream monitoring and analysis module, a custom voice wake-up training and detection module, an intelligent assistant three-party call module and a voice wake-up server, wherein the switching center server is used to bind a unique channel ID to a calling channel and a called channel respectively, and initiate two port allocation requests to the voice wake-up server, and after obtaining two ports, map the data packets of the calling channel and the called channel to the port corresponding to the voice wake-up server, the real-time call voice stream monitoring and analysis module is used to monitor the port on the voice wake-up server, and parse and transcode the received data packets, splice them into voice stream segments, and then send them to the custom voice wake-up training and detection module, the custom voice wake-up training and detection module is used to judge and wake up the intelligent assistant three-party call module, and the intelligent assistant three-party call module is used to reconstruct the three-party call channel on the basis of the original two-party call after the voice wake-up is successful, and enable both the calling and the called parties to interact with the intelligent assistant through the switching center server.

[0007] To optimize the above technical solutions, the specific measures taken also include: Furthermore, a method for waking up an intelligent assistant by voice in a real-time call is characterized by comprising the following steps: A mapping port allocation service is built on the voice wake-up server. After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests to the voice wake-up server. After obtaining two ports, the data packets of the calling channel and the called channel are mapped to the corresponding ports of the voice wake-up server. The real-time call voice stream monitoring and parsing module is used to monitor the port on the voice wake-up server, and the received data packets are parsed and transcoded, and after being spliced ​​into voice stream segments, they are sent to the custom voice wake-up training and detection module; The customized voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by voice, it reconstructs the three-party call channel based on the original two-way call, and enables the caller and the called party to interact with the intelligent assistant separately through the exchange center server.

[0008] Furthermore, after the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, initiates two port allocation requests to the voice wake-up server, and after obtaining two ports, maps the data packets of the calling channel and the called channel to the port corresponding to the voice wake-up server, including the following steps: After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests; first, with the calling channel ID as the main ID and the called channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain a port, and the voice data packet sent from the caller to the called party is mapped to the port, and the port is recorded as the calling channel port; then, with the called channel ID as the main ID and the calling channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain another port, and the voice data packet sent from the called party to the calling party is mapped to the port, and the port is recorded as the called channel port; When the port allocation service receives a port allocation request, it takes out an unoccupied port number from the free port pool to be allocated on the voice wake-up server and returns it, records the primary ID and secondary ID provided in this request, and creates a temporary occupation record of the port; if there is no port number in the free port pool, it means that the concurrency limit has been reached. At this time, the user will be prompted that the smart assistant is busy and cannot provide services temporarily.

[0009] Furthermore, the real-time call voice stream monitoring and parsing module is used to monitor the port on the voice wake-up server, and the received data packets are parsed and transcoded, and after being spliced ​​into voice stream segments, they are sent to the custom voice wake-up training and detection module, including the following steps: After the voice wake-up server allocates the port number, it immediately starts the port monitoring service of the real-time call voice stream monitoring and parsing module to start monitoring the port. After the monitoring starts, it will continuously receive the RTP data packets sent by the switching center server mapping; When the current thread monitors the calling channel port, before processing each RTP data packet, it first checks whether the bound port of the port is still in the occupied state. If the bound port has been released, it means that the called party has hung up, and the call is ended, and no RTP data packet needs to be parsed. Similarly, if the current thread monitors the called channel port, the bound port is released, which means that the calling party has hung up, and the call is also ended, and no RTP data packet needs to be parsed. When it is found that the bound port is still occupied, the call is proceeding smoothly. The real-time call voice stream monitoring and analysis module parses each RTP data packet received in real time, splices the voice data obtained by the analysis in the order of packet reception, intercepts the newly received voice stream in the current time period every second, splices it with the voice stream data of the previous second to form a two-second voice stream and sends it to the custom voice wake-up training and detection module for voice wake-up status inspection after transcoding; if the voice wake-up status of this port or its bound port changes to awakened, the two-second splicing and inspection process is stopped.

[0010] Furthermore, the real-time call voice stream monitoring and parsing module performs real-time parsing on each received RTP data packet, including the following steps: Each RTP data packet consists of an RTP header and a payload. The RTP header consists of 12 bytes, a total of 96 bits of code, including various control information. After removing the RTP header information, the remaining bytes are the actual voice payload. Each voice payload is 160 bytes. The voice payload is stored in a temporary voice file. The channel sends 50 RTP data packets per second. If the port is always occupied during this second, the voice payload is extracted from the RTP packet and appended to the temporary voice file in sequence.

[0011] Furthermore, the two-second voice stream is formed and sent to the custom voice wake-up training and detection module for voice wake-up status inspection after transcoding, including the following steps: In the header information of the RTP packet, the 10th to 16th bits represent the encoding information of the currently transmitted voice payload. When transcoding the voice stream, the remaining payload after removing the RTP header is transcoded according to the encoding information. In the call center traffic, the payload is encoded in PCMA format. Audioop is used to transcode the PCMA format voice stream into the PCM standard encoding format, and then transcoded into a wav format voice file to be sent to the custom voice wake-up training and detection module for voice wake-up status verification. If other encoding formats appear, they also need to be uniformly transcoded into PCM encoding and then converted into wav files.

[0012] Furthermore, the custom voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by the voice, it reconstructs the three-party call channel based on the original two-party call, and enables both the caller and the called party to interact with the intelligent assistant through the exchange center server, including the following steps: When the custom voice wake-up training and detection module detects the preset wake-up word, the voice wake-up detection operation of the calling channel and the called channel in the two port monitoring services is stopped at the same time according to the binding relationship between the primary ID and the secondary ID, and the monitored voice stream is sent to the smart assistant three-party call module; After detecting the voice wake-up word, the port monitoring service of the real-time call voice stream monitoring and parsing module continues to monitor and receive the data packets sent by the corresponding voice channel and parses the real-time voice stream to the intelligent assistant three-party call module. The intelligent assistant three-party call module has no feedback on silent segments. After receiving valid non-silent segments, the voice recognition engine of the intelligent assistant three-party call module is used to translate the text. After the translation is completed, the intelligent assistant service of the intelligent assistant three-party call module is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the intelligent assistant's reply to complete the interaction with the intelligent assistant.

[0013] Furthermore, the preset of the wake-up word includes the following steps: The user logs in to the configuration platform and customizes the voice wake-up word of the smart assistant three-party call module. After entering the wake-up word, training is carried out. The background automatically creates a model training task to train the basic model of the custom wake-up word. First, the speech synthesis interface is automatically called to synthesize 100 positive sample corpora with 20 pre-selected timbres, fast speech speed, slow speech speed, high volume, low volume, and standard speech speed and volume for men and women. Then, a pre-recorded channel noise background is added to the positive sample corpus to generate simulated training sample data. Finally, the simulated positive samples are mixed with the 100 pre-prepared negative samples and put into the training data folder, and the initial training task is started. Based on the speech detection baseline model provided by modelscope, the basic detection model of the custom voice wake-up word is trained. The detection score of the initial basic detection model is generally low. After the user tests and adjusts the threshold on the configuration platform, clicks on the line to apply the model to the voice wake-up of the custom-configured smart assistant three-party call module.

[0014] Furthermore, the intelligent assistant three-party call module has no feedback on the silent segment. After receiving the valid non-silent segment, the voice recognition engine of the intelligent assistant three-party call module is used to translate the text. After the translation is completed, the intelligent assistant service is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the intelligent assistant reply, including the following steps: After the voice wake-up is successful, based on the object-oriented nature of the intelligent assistant service of the current intelligent assistant three-party call module, it is determined whether the subsequent semantic analysis of the intelligent assistant is for the calling channel voice, the called channel voice, or the two-way channel voice; When targeting the voice of the calling channel, the real-time voice recognition engine is only enabled in the calling port monitoring process. After the RTP packet mapped by the calling channel is parsed into a voice stream, it is directly sent to the voice recognition engine for recognition. After the voice recognition engine detects a non-silent segment, voice recognition starts and the caller's inquiry ends. After receiving the silent segment again, the voice recognition engine returns the translation text, which is sent to the intelligent assistant service for semantic analysis. If the intelligent assistant service targets the called channel voice or the two-way channel voice, the voice recognition service is enabled in the corresponding monitoring port process. The answer obtained after analysis by the intelligent assistant service is sent to voice synthesis. After being synthesized into a voice stream, the calling IP port service of the switching center server is found through the calling channel ID, and then the voice data is broadcast to the calling IP port service through the switching center server; similarly, if feedback is required to the called party, the voice data will be broadcast to the called party IP port service, or feedback will be given to the channels of both parties accordingly.

[0015] Furthermore, the following steps are included: the calling and called parties continue to interact with the intelligent assistant until either party hangs up, and then the port allocation service of the real-time call voice stream monitoring and parsing module is re-called, and the port monitoring service stops monitoring the port corresponding to the channel, and at the same time stops monitoring the corresponding port of the other party's channel corresponding to the channel, thereby releasing the line and port occupation of the three-party call module of the intelligent assistant.

[0016] The beneficial effects of the present invention are: The present invention realizes fast customization of wake-up words, and the call voice directly wakes up the intelligent customer service, and provides auxiliary services for three-party calls; by building a real-time dual-channel call data forwarding and monitoring service, the dual-channel voice data packets are forwarded and monitored in real time, thereby realizing the monitoring of the original call process and providing a voice channel for voice wake-up detection and intelligent assistant interaction; by providing a custom wake-up word configuration channel, the basic model of the custom wake-up word is quickly launched through automatic synthesis of training samples; by saving the corpus and detection results generated by the actual call, a text translation auxiliary function and a manual annotation channel are provided, and semi-automatic cyclic optimization training is realized, which provides an effective means for improving the accuracy of voice wake-up on the basis of rapid launch; by building a three-party call environment for the intelligent assistant, in different intelligent assistant services, according to the service settings, the dual-channel or designated channel voice stream is read, and the corresponding channel is played back according to the needs to make feedback, thereby realizing the needs of the intelligent assistant to assist in answering questions during the call; corresponding call channel construction methods are provided for different intelligent assistant services, which solves the problem of assistant service access to two-way calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for waking up an intelligent assistant by voice in a real-time call proposed by the present invention; Figure 2 A schematic diagram of a real-time call voice stream monitoring and parsing process of a method for waking up an intelligent assistant by voice during a real-time call proposed by the present invention; Figure 3 A schematic diagram of a custom voice wake-up training and detection process of a method for voice wake-up of an intelligent assistant in a real-time call proposed by the present invention; Figure 4 A schematic diagram of a three-party call process for an intelligent assistant according to a method for waking up an intelligent assistant by voice in a real-time call proposed by the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] A system for voice waking up an intelligent assistant in a real-time call according to an embodiment of the present invention comprises: a switching center server, a real-time call voice stream monitoring and analysis module, a custom voice wake-up training and detection module, an intelligent assistant three-party call module and a voice wake-up server, wherein the switching center server is used to bind a unique channel ID to a calling channel and a called channel respectively, and initiate two port allocation requests to the voice wake-up server, and after obtaining two ports, map the data packets of the calling channel and the called channel to the port corresponding to the voice wake-up server, the real-time call voice stream monitoring and analysis module is used to monitor the port on the voice wake-up server, and parse and transcode the received data packets, splice them into voice stream segments, and then send them to the custom voice wake-up training and detection module, the custom voice wake-up training and detection module is used to judge and wake up the intelligent assistant three-party call module, and the intelligent assistant three-party call module is used to reconstruct the three-party call channel on the basis of the original two-party call after the voice wake-up is successful, and enable both the calling and the called parties to interact with the intelligent assistant through the switching center server.

[0020] As attached Figure 1 As shown, in another embodiment, a method for voice waking up an intelligent assistant in a real-time call includes the following steps: A mapping port allocation service is built on the voice wake-up server. After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests to the voice wake-up server. After obtaining two ports, the data packets of the calling channel and the called channel are mapped to the corresponding ports of the voice wake-up server. The real-time call voice stream monitoring and parsing module is used to monitor the port on the voice wake-up server, and the received data packets are parsed and transcoded, and then spliced ​​into voice stream segments, which are then sent to the custom voice wake-up training and detection module; The customized voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by voice, it reconstructs the three-party call channel based on the original two-way call, and enables the caller and the called party to interact with the intelligent assistant separately through the exchange center server.

[0021] As attached Figure 2As shown, after the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests to the voice wake-up server. After obtaining two ports, the data packets of the calling channel and the called channel are mapped to the port corresponding to the voice wake-up server, including the following steps: After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests; first, with the calling channel ID as the main ID and the called channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain a port, and the voice data packet sent from the caller to the called party is mapped to the port, and the port is recorded as the calling channel port; then, with the called channel ID as the main ID and the calling channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain another port, and the voice data packet sent from the called party to the calling party is mapped to the port, and the port is recorded as the called channel port; When the port allocation service receives a port allocation request, it takes out an unoccupied port number from the free port pool to be allocated on the voice wake-up server and returns it, records the primary ID and secondary ID provided in this request, and creates a temporary occupation record of the port; if there is no port number in the free port pool, it means that the concurrency limit has been reached. At this time, the user will be prompted that the smart assistant is busy and cannot provide services temporarily.

[0022] The above-mentioned use of the real-time call voice stream monitoring and parsing module to monitor the port on the voice wake-up server, parse and transcode the received data packets, splice them into voice stream segments, and then send them to the custom voice wake-up training and detection module includes the following steps: After the voice wake-up server allocates the port number, it immediately starts the socket of the port monitoring service of the real-time call voice stream monitoring and parsing module to start monitoring the port. After the monitoring starts, it will continuously receive the RTP data packets sent by the switching center server mapping; When the current thread monitors the calling channel port, before processing each RTP data packet, it first checks whether the bound port of the port is still in the occupied state. If the bound port has been released, it means that the called party has hung up, and the call is ended, and no RTP data packet needs to be parsed. Similarly, if the current thread monitors the called channel port, the bound port is released, which means that the calling party has hung up, and the call is also ended, and no RTP data packet needs to be parsed. When it is found that the bound port is still occupied, the call is proceeding smoothly. The real-time call voice stream monitoring and analysis module parses each RTP data packet received in real time, splices the voice data obtained by the analysis in the order of packet reception, intercepts the newly received voice stream in the current time period every second, splices it with the voice stream data of the previous second to form a two-second voice stream and sends it to the custom voice wake-up training and detection module for voice wake-up status inspection after transcoding; if the voice wake-up status of this port or its bound port changes to awakened, the two-second splicing and inspection process is stopped.

[0023] The above-mentioned real-time call voice stream monitoring and parsing module performs real-time parsing on each received RTP data packet, including the following steps: Each RTP data packet consists of an RTP header and a payload. The RTP header consists of 12 bytes, a total of 96 bits of code, including various control information. After removing the RTP header information, the remaining bytes are the actual voice payload. Each voice payload is 160 bytes. The voice payload is stored in a temporary voice file. The channel sends 50 RTP data packets per second. If the port is always occupied during this second, the voice payload is extracted from the RTP packet and appended to the temporary voice file in sequence.

[0024] The above-mentioned two-second voice stream is formed and sent to the custom voice wake-up training and detection module for voice wake-up status inspection after transcoding, including the following steps: In the header information of the RTP packet, the 10th to 16th bits represent the encoding information of the currently transmitted voice payload. When transcoding the voice stream, the remaining payload after removing the RTP header is transcoded according to the encoding information. In the call center traffic, the payload is encoded in PCMA format. Audioop is used to transcode the PCMA format voice stream into the PCM standard encoding format, and then transcoded into a wav format voice file to be sent to the custom voice wake-up training and detection module for voice wake-up status verification. If other encoding formats appear, they also need to be uniformly transcoded into PCM encoding and then converted into wav files.

[0025] As attached Figure 3 As shown, the above-mentioned custom voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by the voice, it reconstructs the three-party call channel based on the original two-party call, and enables both the caller and the called party to interact with the intelligent assistant through the exchange center server, including the following steps: When the custom voice wake-up training and detection module detects the preset wake-up word, the voice wake-up detection operation of the calling channel and the called channel in the two port monitoring services is stopped at the same time according to the binding relationship between the primary ID and the secondary ID, and the monitored voice stream is sent to the smart assistant three-party call module; After detecting the voice wake-up word, the port monitoring service of the real-time call voice stream monitoring and parsing module continues to monitor and receive the data packets sent by the corresponding voice channel and parses the real-time voice stream to the intelligent assistant three-party call module. The intelligent assistant three-party call module has no feedback on silent segments. After receiving valid non-silent segments, the voice recognition engine of the intelligent assistant three-party call module is used to translate the text. After the translation is completed, the intelligent assistant service of the intelligent assistant three-party call module is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the intelligent assistant's reply to complete the interaction with the intelligent assistant.

[0026] The above-mentioned preset of the wake-up word includes the following steps: The user logs in to the configuration platform and customizes the voice wake-up word of the smart assistant three-party call module. After entering the wake-up word, training is carried out. The background automatically creates a model training task to train the basic model of the custom wake-up word. First, the speech synthesis interface is automatically called to synthesize 100 positive sample corpora with 20 pre-selected timbres, fast speech speed, slow speech speed, high volume, low volume, and standard speech speed and volume for men and women. Then, a pre-recorded channel noise background is added to the positive sample corpus to generate simulated training sample data. Finally, the simulated positive samples are mixed with the 100 pre-prepared negative samples and put into the training data folder, and the initial training task is started. Based on the speech detection baseline model provided by modelscope, the basic detection model of the custom voice wake-up word is trained. The detection score of the initial basic detection model is generally low. After the user tests and adjusts the threshold on the configuration platform, clicks on the line to apply the model to the voice wake-up of the custom-configured smart assistant three-party call module.

[0027] As attached Figure 4 As shown, the above-mentioned smart assistant three-party call module has no feedback on the silent segment. After receiving the valid non-silent segment, the voice recognition engine of the smart assistant three-party call module is used to translate the text. After the translation is completed, the smart assistant service is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the smart assistant reply, including the following steps: After the voice wake-up is successful, based on the object-oriented nature of the intelligent assistant service of the current intelligent assistant three-party call module, it is determined whether the subsequent semantic analysis of the intelligent assistant is for the calling channel voice, the called channel voice, or the two-way channel voice; When targeting the voice of the calling channel, the real-time voice recognition engine is only enabled in the calling port monitoring process. After the RTP packet mapped by the calling channel is parsed into a voice stream, it is directly sent to the voice recognition engine for recognition. After the voice recognition engine detects a non-silent segment, voice recognition starts and the caller's inquiry ends. After receiving the silent segment again, the voice recognition engine returns the translation text, which is sent to the intelligent assistant service for semantic analysis. If the intelligent assistant service targets the called channel voice or the two-way channel voice, the voice recognition service is enabled in the corresponding monitoring port process. The answer obtained after analysis by the intelligent assistant service is sent to voice synthesis. After being synthesized into a voice stream, the calling IP port service of the switching center server is found through the calling channel ID, and then the voice data is broadcast to the calling IP port service through the switching center server; similarly, if feedback is required to the called party, the voice data will be broadcast to the called party IP port service, or feedback will be given to the channels of both parties accordingly.

[0028] Among them, the above also includes the following steps: the calling and called parties continue to interact with the intelligent assistant until either party hangs up, and then re-call the port allocation service of the real-time call voice stream monitoring and parsing module, and make the port monitoring service stop monitoring the port corresponding to the channel, and at the same time stop monitoring the corresponding port of the other party's channel corresponding to the channel, so as to release the line and port occupation of the three-party call module of the intelligent assistant.

[0029] In the field of technology for accessing intelligent assistance systems in real-time calls, there are three traditional approaches: One is to develop intelligent agents, develop operation pages for agents, and cooperate with page assistants to provide intelligent responses. However, this method cannot be adapted to ordinary landlines, and requires operators to type in real time to transfer user questions to intelligent assistants. It is even more impossible to achieve real-time two-way translation and cross-language communication during calls required in special scenarios.

[0030] The second is a technical solution that triggers a designated button during a call between two parties to transfer the call to an intelligent assistant for direct voice Q&A service. However, under this solution, the button operation is rather abrupt, the original human customer service can no longer participate in the conversation after the call is transferred, the assistant's auxiliary function cannot be reflected, and it is difficult to achieve ideal human-computer coupling interaction. Similarly, this solution is also difficult to solve the need for cross-language translation in two-way calls.

[0031] The third is to use keyword detection for voice wake-up. This method has the same defects as the second method of button detection. After the call is transferred, three-party communication is also impossible. In addition, in order to identify specific voice assistant trigger words, it is necessary to do voice-to-text conversion and keyword detection throughout the process, which consumes too much energy in the voice-to-text engine and is not suitable for concurrent promotion.

[0032] The present invention builds a real-time dual-channel call data forwarding and monitoring service, including voice packet mapping port allocation service, dual-channel voice data packet forwarding and real-time monitoring, data packet parsing and voice stream transcoding, voice stream segment splicing, and dual-channel recording. The voice wake-up service is sent for detection once per second until the call ends or any channel wakes up successfully. Through automatic training and real-time detection of custom wake-up words, a custom wake-up word configuration channel is provided, and the initial model training of the wake-up word is performed based on a small amount of corpus generated by a baseline model and a speech synthesis engine. During voice wake-up detection, the corresponding model is called according to the wake-up word, the sent corpus is saved, and the model is trained cyclically after manual annotation for further operation and maintenance optimization. A three-party call environment is constructed by accessing an intelligent assistant. In different intelligent assistant services, according to the service settings, the dual-channel or designated channel voice stream is read, and after being recognized as a command text, it is sent to the intelligent assistant for recognition. After obtaining the assistant feedback, the corresponding channel is played to make feedback, so as to meet the needs of the intelligent assistant to assist in answering questions during the call.

[0033] The present invention realizes fast customization of wake-up words, and the call voice directly wakes up the intelligent customer service, and provides auxiliary services for three-party calls; by building a real-time dual-channel call data forwarding and monitoring service, the dual-channel voice data packets are forwarded and monitored in real time, thereby realizing the monitoring of the original call process and providing a voice channel for voice wake-up detection and intelligent assistant interaction; by providing a custom wake-up word configuration channel, the basic model of the custom wake-up word is quickly launched through automatic synthesis of training samples; by saving the corpus and detection results generated by the actual call, a text translation auxiliary function and a manual annotation channel are provided, and semi-automatic cyclic optimization training is realized, which provides an effective means for improving the accuracy of voice wake-up on the basis of rapid launch; by building a three-party call environment for the intelligent assistant, in different intelligent assistant services, according to the service settings, the dual-channel or designated channel voice stream is read, and the corresponding channel is played back according to the needs to make feedback, thereby realizing the needs of the intelligent assistant to assist in answering questions during the call; corresponding call channel construction methods are provided for different intelligent assistant services, which solves the problem of assistant service access to two-way calls.

[0034] In the present invention, the real-time call voice stream monitoring and analysis module, the custom voice wake-up training and detection module and the intelligent assistant three-party call module are described in detail as follows: (1) Real-time call voice stream monitoring and analysis module The real-time call voice stream monitoring and parsing module mainly captures, parses and transcodes the voice data packets in the call, splices them into voice stream segments, and sends them to the voice wake-up detection module. Furthermore, the steps of the real-time call voice stream monitoring and parsing module are as follows: Step 1: Set up a mapping port allocation service. On the voice wake-up server, set up a port dynamic allocation service. After each call starts, the switching center uses the caller and called party's call IDs as the primary IDs and the other party as the secondary ID to request the port dynamic allocation service, and obtains two temporary mapping ports bound to the caller and called party's call IDs. After the bidirectional voice data packets appear, the switching center obtains the voice data packets and maps the data packets to the temporary ports bound to the channel.

[0035] Step 2: Data packet monitoring and analysis. On the voice wake-up server, after the port dynamic allocation service allocates a port, it immediately monitors the port for data. For each RTP data packet received, the voice wake-up server parses it in real time, strips the RTP header file, obtains valid PCMA format voice data, and splices the parsed voice data in the order of receiving packets. Every second, it intercepts the new voice stream received in the current time period and splices it with the voice stream data of the previous second to form a two-second voice stream for subsequent transcoding.

[0036] Step 3: PCMA voice stream transcoding. The voice wake-up service only supports voice wake-up keyword detection for voice files in wav format. After splicing the two-second voice stream, the PCMA compressed format voice stream is first transcoded into a regular PCM-encoded voice stream, and then transcoded into a temporary wav file for wake-up word detection. If a wake-up word is detected, the voice wake-up detection operation of the master and called dual channels in the two port monitoring services is stopped at the same time according to the binding relationship between the master and slave IDs, and the monitored voice stream is sent to the subsequent intelligent assistant engine service.

[0037] (2) Customized voice wake-up training and detection module The custom voice wake-up training and detection module is mainly used to customize and self-service the voice wake-up word detection. Further, the steps of the custom voice wake-up training and detection module are as follows: Step 1: Basic detection model training. After the user sets a custom voice wake-up word, the speech synthesis interface is automatically called to synthesize 100 training corpora of the wake-up word with different timbres, speaking speeds, and volumes. A pre-recorded channel noise background is added to the synthesized voice data to generate simulated training data. Based on the speech detection baseline model, the basic detection model of the custom voice wake-up word is trained.

[0038] Step 2: Online detection and corpus collection. The monitored voice stream will be transcoded and sent to the inspection in real time. This step reads the two-second voice file, detects in real time whether there is wake-up word information in the current voice file, and feeds the wake-up result back to the voice monitoring module. The two-second voice file sent for inspection is automatically marked as positive samples and negative samples according to the wake-up result, and stored in the temporary corpus. At the same time, the current detected voice wake-up matching score is recorded.

[0039] Step 3: Self-training optimization. The speech recognition engine is called daily to translate the data in the temporary corpus. The translation results and matching scores are manually assisted to calibrate the positive and negative sample label data in the temporary corpus. After the calibration, based on the basic detection model of the custom voice wake-up word, the newly added positive and negative samples in the temporary corpus are called for optimization iteration. The accuracy of the models before and after the iteration is tested, and the model with improved accuracy covers the original model and is provided to the online service.

[0040] (3) Smart assistant three-party call module The smart assistant three-party call module mainly reconstructs the three-party call channel based on the original two-party call after the voice wake-up is successful, so that both the caller and the called party can interact with the smart assistant. Furthermore, the steps of the smart assistant three-party call module are as follows: Step 1: Dual-channel monitoring for real-time voice translation. After the voice wake-up word is detected, semantic analysis is performed based on whether the currently loaded intelligent assistant is for the caller, the called party, or both the caller and the called party. The voice wake-up service continues to monitor and receive data packets sent by the corresponding voice channel to parse the real-time PCMA voice stream, and also transcodes it into PCM standard voice stream encoding and sends it to the voice engine for translation. The voice engine has no feedback on silent segments. It starts translating after receiving valid non-silent segments. After the translation is completed, the text is sent to the natural language processing engine.

[0041] Step 2: The smart assistant plays the answer to the designated channel. While keeping the original conversation between the two parties unchanged, the natural language processing engine processes the answer, synthesizes the voice text, and sends the voice stream to the designated call IP port according to the smart assistant function design, so that the designated channel can obtain the content of the smart assistant's reply.

[0042] Step 3: End of monitoring and release of port. After either the caller or the called party hangs up, the monitoring port allocation service is called again. The allocation service stops monitoring the port corresponding to the channel and the corresponding port of the other party's channel corresponding to the channel, releasing the smart assistant line and port occupation.

[0043] When a certain intelligent assistant application needs to be awakened by voice during a real-time call, it is necessary to monitor the call voice, recognize the voice awakening word, and establish a voice channel between the intelligent assistant and the caller and the called party. A specific implementation of the present invention is as follows: The real-time call voice stream monitoring and parsing module mainly captures, parses and transcodes the voice data packets in the call, splices them into voice stream segments, and sends them to the voice wake-up detection module. The specific steps are as follows: Step 1: On the voice wake-up server, set up a port allocation service. Take 10,000 unoccupied ports between 50,000 and 60,000 on the server and put them into the idle port pool to be allocated. One smart assistant auxiliary call occupies two ports to ensure that the smart assistant can handle 5,000 concurrent calls.

[0044] Step 1.1: When the port allocation service receives a port allocation request, it takes an unoccupied port number from the free port pool to be allocated and returns it, records the primary channel ID and secondary channel ID provided in this request, and creates a temporary occupation record for the port. If there is no port number in the free port pool, it means that the concurrency limit has been reached, and the user will be prompted that the smart assistant is busy and cannot provide services temporarily.

[0045] Step 1.2: After the incoming call is connected, the exchange center server binds a unique channel ID to the calling and called channels respectively, and initiates two port allocation requests. First, with the calling channel ID as the primary ID and the called channel ID as the secondary ID, the voice wake-up server port dynamic allocation service request obtains a port, maps the voice data packet sent by the caller to the called party to the port, and records the port as the calling channel port. Next, with the called channel ID as the primary ID and the calling channel ID as the secondary ID, the voice wake-up server port dynamic allocation service request obtains another port, maps the voice data packet sent by the called party to the calling party to the port, and records the port as the called channel port. That is, after a call starts, there will be two different ports on the voice wake-up server to receive the calling voice data packet and the called voice data packet of the call respectively. At the same time, the two ports are bound by the primary and secondary channel IDs, and the calling channel port and the called channel port are each other's bound ports.

[0046] Step 2: After the voice wake-up server allocates the port number, it immediately starts the socket to start monitoring the port. After the monitoring starts, it will continuously receive the RTP data packets sent by the switching center server mapping.

[0047] Step 2.1, taking the current thread monitoring the calling channel port as an example, before processing each RTP data packet, first check whether the bound port of the port is still in the occupied state. If the bound port has been released, it means that the called party has hung up, and the call is ended, and no RTP data packet needs to be parsed. On the contrary, if the current thread monitors the called channel port, the bound port is released, which means that the caller has hung up, and the call is also ended, and no RTP data packet is parsed.

[0048] Step 2.2: If the bound port is still occupied, the call is going smoothly. The RTP data packet needs to be parsed this time. Each RTP data packet consists of an RTP header and a payload. The RTP header consists of 12 bytes, a total of 96 bits of code, including various control information. After removing the RTP header information, the remaining bytes are the actual voice payload. Each voice payload is 160 bytes, and the voice payload needs to be stored in a temporary voice file. The channel sends 50 RTP data packets per second. If the port is always occupied during this second, the voice payload is extracted from the RTP packet and appended to the temporary voice file in sequence.

[0049] Step 2.3, query the port voice wake-up status record. If the voice wake-up status of this port and its bound port are both not awakened, then after one second, read the voice stream stored in the current second and splice it with the voice stream stored in the previous second to generate a two-second detection voice stream, which is sent to step 3 for transcoding. After inspection, overwrite the voice stream file of the previous second with the voice stream information stored in the current second, start a new round of voice stream splicing, and perform voice wake-up status inspection before the next second inspection. If the voice wake-up status of this port or its bound port changes to awakened, stop the two-second splicing inspection process, start the voice recognition engine service, and directly transcode the voice stream read from each RTP packet into PCM format, send it to the voice recognition engine, and enter the intelligent assistant pre-recognition process.

[0050] Step 3: In the header information of the RTP packet, the 10th to 16th bits indicate the encoding information of the currently transmitted voice payload. When transcoding the voice stream, the remaining payload after removing the RTP header needs to be transcoded based on the encoding information. In this call center traffic, the payload is encoded in PCMA format. Audioop is used to transcode the PCMA format voice stream into the PCM standard encoding format, and then transcoded into a wav format voice file for the voice wake-up module to detect. If other encoding formats appear, they also need to be uniformly transcoded into PCM encoding, and then converted into wav files.

[0051] The custom voice wake-up training and detection module is mainly used to customize and self-service the voice wake-up word detection. The specific steps are as follows: Step 1. The user logs in to the smart assistant configuration platform and creates a new custom voice wake-up word for the assistant. After entering the wake-up word, click on the training page, and the background automatically creates a model training task to train the basic model of the custom wake-up word. First, the speech synthesis interface is automatically called to synthesize 100 positive sample corpora with 20 pre-selected timbres, fast speech speed, slow speech speed, high volume, low volume, and standard speech speed volume for men and women. Next, add a pre-recorded channel noise background to the positive sample corpus to generate simulated training sample data. Finally, mix the simulated positive samples with the 100 pre-prepared negative samples and put them into the training data folder, start the initial training task, and train the basic detection model of the custom voice wake-up word based on the speech detection baseline model provided by modelscope. The detection score of the initial basic detection model is generally low. After the user tests and adjusts the threshold on the configuration platform, click on the line to apply the model to the voice wake-up of the smart assistant.

[0052] Step 2. After the model is launched, in actual calls, the monitored voice stream information will be spliced ​​into two-second voice and sent to this model for detection based on the activated smart assistant service. This step provides a model calling interface bound to the wake-up word. By inputting the wake-up word model ID and the two-second voice file, the corresponding wake-up word model is called to detect in real time whether the current voice file contains the wake-up word information customized by the smart assistant service, and obtain the wake-up result feedback. After the detection is completed, the two-second voice file sent for inspection is automatically labeled as positive samples and negative samples according to the wake-up detection results, and stored in a temporary corpus. The file name, sample positive and negative recommended values, and preliminary detection matching scores are recorded in the temporary sample database for subsequent optimization training.

[0053] Step 3. Every day, the system calls the speech recognition engine regularly to perform text translation on the untrained data in the temporary corpus, and the translation results are bound to the voice files and their records. The manual trainer logs in to the system regularly, clicks on the temporary corpus on the page, sorts in descending order according to the matching score, and calibrates the positive and negative sample labels in the temporary corpus with reference to the translated text and the actual voice listening results. After the calibration is completed, click Optimize Model to create a new model training task based on the custom voice wake-up word, select the calibrated corpus, click Train, and the system will add the selected training corpus to the basic detection model based on the custom voice wake-up word for re-iterative training. After the training is completed, the corpus participating in this training is automatically removed from the temporary corpus and entered into the previous training database. The manual trainer tests the new model, readjusts the threshold and goes online again. The accuracy will continue to improve in the process.

[0054] The smart assistant three-party call module mainly reconstructs the three-party call channel based on the original two-party call after the voice wake-up is successful, so that both the caller and the called party can interact with the smart assistant. The specific steps are as follows: Step 1. In view of the differences in the categories of auxiliary services provided by smart assistants, the audio source of the assistant's semantic analysis and the feedback object of the answer are different. Each smart assistant needs to set the object-oriented and feedback object of the current smart assistant service when it goes online. Object-oriented represents the audio source channel that the smart assistant listens to and analyzes, which can be set to caller, called, or dual-channel. Feedback object represents the party to whom the smart assistant's answer is played, and can also be set to caller, called, or dual-channel. The free combination of object-oriented and feedback object settings can meet the various types of smart assistant services required in current calls.

[0055] Step 1.1. After the voice wake-up is successful, based on the object-oriented nature of the current intelligent assistant service, determine whether the subsequent semantic analysis of the intelligent assistant is for the calling channel voice, the called channel voice, or the two-way channel voice. Taking the auxiliary interpretation of the calling channel voice as an example, the real-time voice recognition service is enabled only in the calling port monitoring process. After the RTP packet mapped by the calling channel is parsed into a PCM voice stream, it is directly sent to the voice recognition service for recognition. After a non-silent segment is detected, voice recognition begins; after the caller's inquiry ends and the silent segment is received again, the voice recognition service returns the translated text and sends the text to the intelligent assistant for semantic analysis. If the intelligent assistant targets the called channel or both the calling and called parties, the voice recognition service is enabled in the corresponding monitoring port process.

[0056] Step 2: Based on the feedback object of the current intelligent assistant service, determine whether the intelligent assistant needs to give feedback to the calling channel, the called channel, or both channels. Taking the intelligent assistant giving feedback to the calling channel as an example, the answer obtained after the natural language processing engine is analyzed is sent to the voice synthesis. After being synthesized into a voice stream, the calling IP port service of the exchange center is searched through the calling channel ID, and then the voice data is broadcast to the calling IP port service through the exchange center. Similarly, if feedback needs to be given to the called party, the voice data is broadcast to the called party IP port service.

[0057] Step 3. The calling and called parties continue to interact with the assistance of the intelligent assistant until either party hangs up. During this process, the port monitoring service is continuously performed on the two ports bound to the calling channel and the called channel. At the moment of hanging up, the exchange center will call the port allocation service again with the primary ID and secondary ID used when obtaining the port, notify the port service occupation has ended, and release the port back to the port pool. If the port monitoring service no longer receives monitoring data from the port mapped by the channel, or finds that the port has been released, or finds that the port bound to the port has been released, it means that the call is over, and the monitoring service for the port will end and the port will be released back to the port pool.

[0058] In another embodiment, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned method for voice-awakening an intelligent assistant in a real-time call.

[0059] In another embodiment, the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for voice waking up an intelligent assistant in a real-time call as described above is implemented.

[0060] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0061] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0062] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A system for waking up an intelligent assistant by voice during a real-time call, characterized in that: include: An exchange center server, a real-time call voice stream monitoring and analysis module, a custom voice wake-up training and detection module, an intelligent assistant three-party call module and a voice wake-up server. The exchange center server is used to bind a unique channel ID to the calling channel and the called channel respectively, and initiate two port allocation requests to the voice wake-up server. After obtaining two ports, the data packets of the calling channel and the called channel are mapped to the corresponding ports of the voice wake-up server. The real-time call voice stream monitoring and analysis module is used to monitor the port on the voice wake-up server, and parse and transcode the received data packets, splice them into voice stream segments, and then send them to the custom voice wake-up training and detection module. The custom voice wake-up training and detection module is used to judge and wake up the intelligent assistant three-party call module. The intelligent assistant three-party call module is used to reconstruct the three-party call channel based on the original two-way call after the voice wake-up is successful, and enable both the calling and the called parties to interact with the intelligent assistant through the exchange center server.

2. A method for waking up an intelligent assistant by voice during a real-time call, characterized in that: The steps include: A mapping port allocation service is built on the voice wake-up server. After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests to the voice wake-up server. After obtaining two ports, the data packets of the calling channel and the called channel are mapped to the corresponding ports of the voice wake-up server. The real-time call voice stream monitoring and parsing module is used to monitor the port on the voice wake-up server, and the received data packets are parsed and transcoded, and after being spliced ​​into voice stream segments, they are sent to the custom voice wake-up training and detection module; The customized voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by voice, it reconstructs the three-party call channel based on the original two-way call, and enables the caller and the called party to interact with the intelligent assistant separately through the exchange center server.

3. The method for waking up an intelligent assistant by voice in a real-time call according to claim 2, characterized in that: After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, initiates two port allocation requests to the voice wake-up server, and after obtaining two ports, maps the data packets of the calling channel and the called channel to the port corresponding to the voice wake-up server, including the following steps: After the incoming call is connected, the exchange center server binds a unique channel ID to the calling channel and the called channel respectively, and initiates two port allocation requests; first, with the calling channel ID as the main ID and the called channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain a port, and the voice data packet sent from the caller to the called party is mapped to the port, and the port is recorded as the calling channel port; then, with the called channel ID as the main ID and the calling channel ID as the secondary ID, the port allocation service request of the real-time call voice stream monitoring and parsing module is obtained to obtain another port, and the voice data packet sent from the called party to the calling party is mapped to the port, and the port is recorded as the called channel port; When the port allocation service receives a port allocation request, it takes out an unoccupied port number from the free port pool to be allocated on the voice wake-up server and returns it, records the primary ID and secondary ID provided in this request, and creates a temporary occupation record of the port; if there is no port number in the free port pool, it means that the concurrency limit has been reached. At this time, the user will be prompted that the smart assistant is busy and cannot provide services temporarily.

4. The method for waking up an intelligent assistant by voice in a real-time call according to claim 2, characterized in that: The real-time call voice stream monitoring and parsing module is used to monitor the port on the voice wake-up server, and the received data packets are parsed and transcoded, and after being spliced ​​into voice stream segments, they are sent to the custom voice wake-up training and detection module, including the following steps: After the voice wake-up server allocates the port number, it immediately starts the port monitoring service of the real-time call voice stream monitoring and parsing module to start monitoring the port. After the monitoring starts, it will continuously receive the RTP data packets sent by the switching center server mapping; When the current thread monitors the calling channel port, before processing each RTP data packet, it first checks whether the bound port of the port is still in the occupied state. If the bound port has been released, it means that the called party has hung up, and the call is ended, and no RTP data packet needs to be parsed. Similarly, if the current thread monitors the called channel port, the bound port is released, which means that the calling party has hung up, and the call is also ended, and no RTP data packet needs to be parsed. When it is found that the bound port is still occupied, the call is proceeding smoothly. The real-time call voice stream monitoring and analysis module parses each RTP data packet received in real time, splices the voice data obtained by the analysis in the order of packet reception, intercepts the newly received voice stream in the current time period every second, splices it with the voice stream data of the previous second to form a two-second voice stream and sends it to the custom voice wake-up training and detection module for voice wake-up status inspection after transcoding; if the voice wake-up status of this port or its bound port changes to awakened, the two-second splicing and inspection process is stopped.

5. The method for waking up an intelligent assistant by voice in a real-time call according to claim 4, characterized in that: The real-time call voice stream monitoring and parsing module performs real-time parsing on each received RTP data packet, including the following steps: Each RTP data packet consists of an RTP header and a payload. The RTP header consists of 12 bytes, a total of 96 bits of code, including various control information. After removing the RTP header information, the remaining bytes are the actual voice payload. Each voice payload is 160 bytes. The voice payload is stored in a temporary voice file. The channel sends 50 RTP data packets per second. If the port is always occupied during this second, the voice payload is extracted from the RTP packet and appended to the temporary voice file in sequence.

6. The method for waking up an intelligent assistant by voice in a real-time call according to claim 4, characterized in that: The two-second voice stream is formed and, after transcoding, sent to the custom voice wake-up training and detection module for voice wake-up status inspection, including the following steps: In the header information of the RTP packet, bits 10 to 16 indicate the encoding information of the currently transmitted voice payload. When transcoding the voice stream, the remaining payload after removing the RTP header is transcoded based on the encoding information. In this call center traffic, the payload is encoded in PCMA format. Audioop is used to transcode the PCMA voice stream into PCM standard encoding format, and then transcoded into wav format voice file to send to the custom voice wake-up training and detection module for voice wake-up status verification. If other encoding formats appear, they also need to be uniformly transcoded into PCM encoding and then converted into wav files.

7. The method for waking up an intelligent assistant by voice in a real-time call according to claim 3, characterized in that: The custom voice wake-up training and detection module determines and wakes up the intelligent assistant three-party call module. After the intelligent assistant three-party call module is successfully awakened by the voice, it reconstructs the three-party call channel based on the original two-party call, and enables both the caller and the called party to interact with the intelligent assistant through the exchange center server, including the following steps: When the custom voice wake-up training and detection module detects the preset wake-up word, the voice wake-up detection operation of the calling channel and the called channel in the two port monitoring services is stopped at the same time according to the binding relationship between the primary ID and the secondary ID, and the monitored voice stream is sent to the smart assistant three-party call module; After detecting the voice wake-up word, the port monitoring service of the real-time call voice stream monitoring and parsing module continues to monitor and receive the data packets sent by the corresponding voice channel and parses the real-time voice stream to the intelligent assistant three-party call module. The intelligent assistant three-party call module has no feedback on silent segments. After receiving valid non-silent segments, the voice recognition engine of the intelligent assistant three-party call module is used to translate the text. After the translation is completed, the intelligent assistant service of the intelligent assistant three-party call module is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the intelligent assistant's reply to complete the interaction with the intelligent assistant.

8. The method for waking up an intelligent assistant by voice in a real-time call according to claim 7, characterized in that: The preset of the wake-up word includes the following steps: The user logs in to the configuration platform and customizes the voice wake-up word of the smart assistant three-party call module. After entering the wake-up word, training is performed. The background automatically creates a model training task to train the basic model of the custom wake-up word. First, the speech synthesis interface is automatically called to synthesize 100 positive sample corpora using 20 pre-selected timbres, fast speech speed, slow speech speed, high volume, low volume, and standard speech speed and volume for men and women. Then, a pre-recorded channel noise background is added to the positive sample corpus to generate simulated training sample data. Finally, the simulated positive samples are mixed with 100 pre-prepared negative samples and put into the training data folder, and the initial training task is started. Based on the speech detection baseline model provided by modelscope, the basic detection model of the custom voice wake-up word is trained. The detection score of the initial basic detection model is generally low. After the user tests and adjusts the threshold on the configuration platform, click Go Online to apply the model to the voice wake-up of the custom-configured smart assistant three-party call module.

9. The method for waking up an intelligent assistant by voice in a real-time call according to claim 7, characterized in that: The smart assistant three-party call module has no feedback on the silent segment. After receiving the valid non-silent segment, the voice recognition engine of the smart assistant three-party call module is used to translate the text. After the translation is completed, the smart assistant service is used to obtain the answer, and the voice stream is sent to the corresponding designated call IP port, so that the designated channel obtains the content of the smart assistant reply, including the following steps: After the voice wake-up is successful, based on the object-oriented nature of the intelligent assistant service of the current intelligent assistant three-party call module, it is determined whether the subsequent semantic analysis of the intelligent assistant is for the calling channel voice, the called channel voice, or the two-way channel voice; When targeting the voice of the calling channel, the real-time voice recognition engine is only enabled in the calling port monitoring process. After the RTP packet mapped by the calling channel is parsed into a voice stream, it is directly sent to the voice recognition engine for recognition. After the voice recognition engine detects a non-silent segment, voice recognition starts and the caller's inquiry ends. After receiving the silent segment again, the voice recognition engine returns the translation text, which is sent to the intelligent assistant service for semantic analysis. If the intelligent assistant service targets the called channel voice or the two-way channel voice, the voice recognition service is enabled in the corresponding monitoring port process. The answer obtained after analysis by the intelligent assistant service is sent to voice synthesis. After being synthesized into a voice stream, the calling IP port service of the switching center server is found through the calling channel ID, and then the voice data is broadcast to the calling IP port service through the switching center server; similarly, if feedback is required to the called party, the voice data will be broadcast to the called party IP port service, or feedback will be given to the channels of both parties accordingly.

10. The method for waking up an intelligent assistant by voice in a real-time call according to claim 2, characterized in that: It also includes the following steps: the calling and called parties continue to interact with the intelligent assistant until either party hangs up, then the port allocation service of the real-time call voice stream monitoring and parsing module is re-called, and the port monitoring service stops monitoring the port corresponding to the channel, and at the same time stops monitoring the corresponding port of the other party's channel corresponding to the channel, thereby releasing the line and port occupation of the three-party call module of the intelligent assistant.

Images