Call processing system and method

Abstract

The present disclosure relates to a system (100) and method (200) for processing a mobile call between a user (110) and a mobile network operator (120). The method (200) comprises: receiving (202) a call audio from the mobile network operator (120), the call audio sent by the user (110) during playing of a first reply voice; detecting (204) if the call audio contains a human voice; determining (208) if a text content corresponding to the human voice matches a set of interruption words; sending (210), if the text content does not match, a stop playback instruction to the mobile network operator (120) to stop playing the first reply voice; generating (212), if the text content matches, a second reply voice based on the matched interruption words; and sending (214) the second reply voice and the stop playback instruction to the mobile network operator (120) to stop playing the first reply voice and to play the second reply voice to the user (110).

Description

[0001] CALL PROCESSING SYSTEM AND METHOD

[0002] Technical Field

[0003] The present disclosure relates generally to call processing, and in particular to a system and method for processing a mobile call between a user and a mobile network operator.

[0004] In the telecommunications industry, soft switch systems are used to establish, maintain, route, and terminate sessions in telecommunication networks. With the continuous development of artificial intelligence technology, intelligent voice technology has also developed rapidly and is being applied to soft switch systems. A soft switch system applying intelligent voice technology can provide customer service to customers through telecommunication networks including conducting multiple rounds of dialogue to interact with customers.

[0005] For example, a customer may seek customer service by making a mobile call to the mobile network operator. The customer may ask some questions or make some requests during the call. The mobile network operator receives a customer voice from the customer and sends the customer voice to the soft switch system. The soft switch system includes a soft switch subsystem and an intelligent voice subsystem. The soft switch subsystem receives the customer voice from the mobile network operator and sends the customer voice to the intelligent voice subsystem. Using artificial intelligence modules, the intelligent voice subsystem generates a reply voice corresponding to the customer voice and sends the reply voice to the soft switch subsystem. The soft switch subsystem then sends the reply voice to the mobile network operator which plays the reply voice to the customer.

[0006] In some situations, the customer may want to interrupt the playing of the reply voice. During playback of the reply voice, the customer may send an interruption voice to the mobile network operator. The mobile network operator sends the interruption voice to the soft switch subsystem, and the soft switch subsystem sends the interruption voice to the intelligent voice subsystem. Using artificial intelligence modules, the intelligent voice subsystem detects whether the interruption voice contains a human voice and sends the detection result to the soft switch subsystem. If the result is positive, the soft switch subsystem sends a stop playback instruction to the mobile network operator to stop playing the reply voice.

[0007] Although this voice interruption function of the soft switch system may help to improve the user experience of customers, there is some network latency because the soft switch subsystem and the intelligent voice subsystem are deployed on different servers. The soft switch subsystem and the intelligent voice subsystem need to communicate through a communication network which introduces the network latency. This network latency creates a communication delay such that the soft switch system is unable to promptly control the mobile network operator to stop playing the reply voice, resulting in poor user experience for customers.

[0008] Therefore, in order to address or alleviate at least one of the aforementioned problems and / or disadvantages, there is a need to provide an improved call processing system and method, particularly with an improved voice interruption function.

[0009] Summary

[0010] In accordance with a first aspect of the present disclosure, call processing system for processing a mobile call between a user and a mobile network operator is provided. The call processing system comprises a soft switch subsystem. The soft switch subsystem comprises: a communication module configured for receiving a call audio of the mobile call from the mobile network operator, the call audio sent by the user to the mobile network operator during playing of a first reply voice by the mobile network operator to the user in the mobile call; a voice detection module configured for detecting if the call audio contains a human voice; a speech recognition module configured for, if the call audio contains the human voice, determining if a text content corresponding to the human voice matches any one of a number of predefined sets of interruption words; and an interruption voice generation module configured for, if the text content matches one of the predefined sets of interruption words, generating a second reply voice based on the matched set of interruption words. The communication module is further configured for: sending, if the text content does not match any one of the predefined sets of interruption words, a stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice; and sending, in response to said generation of the second reply voice, the stop playback instruction and the second reply voice to the mobile network operator for the mobile network operator to stop playing the first reply voice and to play the second reply voice to the user.

[0011] In an embodiment, the speech recognition module comprises a set of data models configured for converting, using a set of data models of the speech recognition module, the human voice into the text content, and wherein the text content matches one of the predefined sets of interruption words if said conversion of the human voice into the text content is successful. In an embodiment, said conversion of the human voice into the text content is successful if the text content is identical to one of the predefined sets of interruption words.

[0012] In an embodiment, said conversion of the human voice into the text content is unsuccessful if the text content does not contain any one of the predefined sets of interruption words, or if the text content contains any one of the predefined sets of interruption words and one or more additional words.

[0013] In an embodiment, the data models comprise a pre-trained acoustic model, a pre-trained language model, and a pre-established recognition word model.

[0014] In an embodiment, the acoustic model is trained using a first training sample dataset comprising multiple first training samples and multiple second training samples; each first training sample comprises a respective one of the predefined sets of interruption words and comprising a respective first audio corresponding to the respective predefined set of interruption words; and each second training sample comprises a training text without any of the predefined sets of interruption words and comprising a respective second audio corresponding to the respective training text.

[0015] In an embodiment, the language model is trained using a second training sample dataset comprising one or more of the predefined sets of interruption words and comprising multiple other words related to the one or more of the predefined sets of interruption words.

[0016] In an embodiment, the acoustic model is a GMM-HMM (Gaussian Mixture Model-Hidden Markov Model) model and / or the language model comprises an N-gram language model.

[0017] In an embodiment, the interruption voice generation module comprises a text-to-speech module configured for generating the second reply voice based on an interruption reply text generated based on the matched set of interruption words.

[0018] In an embodiment, the interruption voice generation module comprises a natural language understanding module configured for generating the interruption reply text based on the matched set of interruption words.

[0019] According to a second aspect of the present disclosure, a call processing method for processing a mobile call between a user and a mobile network operator is provided. The call processing method comprises: receiving, by a communication module of a soft switch subsystem, a call audio of the mobile call from the mobile network operator, the call audio sent by the user to the mobile network operator during playing of a first reply voice by the mobile network operator to the user in the mobile call; detecting, by a voice detection module of the soft switch subsystem, if the call audio contains a human voice; determining, by a speech recognition module of the soft switch subsystem and if the call audio contains the human voice, if a text content corresponding to the human voice matches any one of a number of predefined sets of interruption words; sending, by the communication module and if the text content does not match any one of the predefined sets of interruption words, a stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice; generating, by an interruption voice generation module of the soft switch subsystem and if the text content matches one of the predefined sets of interruption words, a second reply voice based on the matched set of interruption words; and sending, by the communication module in response to said generation of the second reply voice, the second reply voice and the stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice and to play the second reply voice to the user.

[0020] In an embodiment, the call processing method comprises: converting, using a set of data models of the speech recognition module, the human voice into the text content; and determining, by the speech recognition module, that the text content matches one of the predefined sets of interruption words if said conversion of the human voice into the text content is successful.

[0021] In an embodiment, said conversion of the human voice into the text content is successful if the text content is identical to one of the predefined sets of interruption words.

[0022] In an embodiment, said conversion of the human voice into the text content is unsuccessful if the text content does not contain any one of the predefined sets of interruption words, or if the text content contains any one of the predefined sets of interruption words and one or more additional words.

[0023] In an embodiment, the data models comprise a pre-trained acoustic model, a pre-trained language model, and a pre-established recognition word model.

[0024] In an embodiment, the call processing method comprises training the acoustic model using a first training sample dataset comprising multiple first training samples and multiple second training samples, wherein each first training sample comprises a respective one of the predefined sets of interruption words and comprising a respective first audio corresponding to the respective predefined set of interruption words; and wherein each second training sample comprises a training text without any of the predefined sets of interruption words and comprising a respective second audio corresponding to the respective training text. In an embodiment, the call processing method comprises training the language model using a second training sample dataset comprising one or more of the predefined sets of interruption words and comprising multiple other words related to the one or more of the predefined sets of interruption words.

[0025] In an embodiment, the call processing method comprises establishing the recognition word model based on inter-word relationships corresponding to each of the predefined sets of interruption words.

[0026] In an embodiment, the call processing method comprises generating, by a text-to-speech module of the interruption voice generation module, the second reply voice based on an interruption reply text generated based on the matched set of interruption words.

[0027] In an embodiment, the call processing method comprises language understanding module of the interruption voice generation module, the interruption reply text based on the matched set of interruption words.

[0028] A call processing system and method with voice interruption function according to the present disclosure are thus disclosed herein. Various features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description of the embodiments of the present disclosure, by way of non-limiting examples only, along with the accompanying drawings.

[0029] Brief Description of the Drawings

[0030] FIG. 1 is a block diagram illustrating a system architecture of a call processing system, according to embodiments of the present disclosure.

[0031] FIG. 2 is a flowchart illustrating a call processing method implementable by the system architecture of FIG. 1 , according to embodiments of the present disclosure.

[0032] FIG. 3 is a swimlane diagram illustrating a communication sequence implementable by the system architecture of FIG. 1 , according to embodiments of the present disclosure.

[0033] FIG. 4 is a block diagram illustrating a technical architecture of a computing server, according to embodiments of the present disclosure. Detailed Description

[0034] For purposes of brevity and clarity, descriptions of embodiments of the present disclosure are directed to a call processing system and method with voice interruption function in accordance with the drawings. While aspects of the present disclosure will be described in conjunction with the embodiments provided herein, it will be understood that they are not intended to limit the present disclosure to these embodiments. On the contrary, the present disclosure is intended to cover alternatives, modifications and equivalents to the embodiments described herein, which are included within the scope of the present disclosure as defined by the appended claims. Furthermore, in the following detailed description, specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be recognized by an individual having ordinary skill in the art, i.e. a skilled person, that the present disclosure may be practiced without specific details, and / or with multiple details arising from combinations of aspects of particular embodiments. In a number of instances, well-known systems, methods, procedures, and components have not been described in detail so as to not unnecessarily obscure aspects of the embodiments of the present disclosure.

[0035] In embodiments of the present disclosure, depiction of a given element or consideration or use of a particular element number in a particular figure or a reference thereto in corresponding descriptive material can encompass the same, an equivalent, or an analogous element or element number identified in another figure or descriptive material associated therewith.

[0036] References to “an embodiment / example”, “another embodiment / example”, “some embodiments / examples”, “some other embodiments / examples”, and so on, indicate that the embodiment(s) / example(s) so described may include a particular feature, structure, characteristic, property, element, or limitation, but that not every embodiment / example necessarily includes that particular feature, structure, characteristic, property, element or limitation. Furthermore, repeated use of the phrase “in an embodiment I example” or “in another embodiment / example” does not necessarily refer to the same embodiment / example.

[0037] The terms “comprising”, “including”, “having”, and the like do not exclude the presence of other features / elements I steps than those listed in an embodiment. Recitation of certain features / elements / steps in mutually different embodiments does not indicate that a combination of these features / elements / steps cannot be used in an embodiment.

[0038] As used herein, the terms “a” and “an” are defined as one or more than one. The use of “ / ” in a figure or associated text is understood to mean “and / or” unless otherwise indicated. The term “set” is defined as a non-empty finite organisation of elements that mathematically exhibits a cardinality of at least one (e.g. a set as defined herein can correspond to a unit, singlet, or single-element set, or a multiple-element set), in accordance with known mathematical definitions. The terms “first”, “second”, etc. are used merely as labels or identifiers and are not intended to impose numerical requirements on their associated terms.

[0039] As used in the present disclosure, the terms “component”, “module”, “system”, “interface” and the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a module may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more modules may reside within a process and / or thread of execution and a module may be localized on one computer and / or distributed between two or more computers. As another example, an interface can include I / O components as well as associated processor, application, and / or API components. In the context of the present disclosure, the information processing and response generation system and its constituent parts may be implemented as hardware, software, or a combination thereof.

[0040] Furthermore, various embodiments of the present disclosure may be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. For instance, the claimed subject matter may be implemented as a computer-readable medium embedded with a computer executable program, which encompasses a computer program accessible from any computer-readable storage device or storage media. For example, computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, etc.), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), etc.), smart cards, and flash memory devices (e.g., card, stick, key drive, etc.).

[0041] FIG. 1 is a block diagram illustrating a system architecture of a call processing system 100 for processing a mobile call between a user 1 10 and a mobile network operator 120, according to various embodiments of the present disclosure. For example, the user 110 uses a user device 1 12, such as a mobile phone, to make the mobile call with the mobile network operator 120. For example, the user 1 10 is one who is engaging in multi-round dialogues with the mobile network operator. The call processing system 100 may also be referred to as a soft switch system 100 and includes a soft switch subsystem 130 and a voice subsystem 140 communicative with each other. Specifically, the soft switch subsystem 130, voice subsystem 140, mobile network operator 120, and user device 1 12 are communicative with each other through a communication network 150.

[0042] As used herein, the communication network 150 is a network layer, medium, or environment through which content, notifications, and / or messages are communicated among various components. Suitable security protocols, such as encryption protocols, may be implemented in the communication network 150 for secure communications among the components. Some non-limiting examples of the communication network 150 include a virtual private network (VPN), wireless fidelity (Wi-Fi) network, light fidelity (Li-Fi) network, local area network (LAN), wide area network (WAN), LoRaWAN, metropolitan area network (MAN), satellite network, Internet, fibre optic network, coaxial cable network, infrared (IR) network, radio frequency (RF) network, and any combination thereof. Various components in the communication network 150 may connect to the communication network 150 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), 2nd to 5th Generation (2G to 5G) communication protocols, Long Term Evolution (LTE) communication protocols, and any combination thereof. Each component to the communication network 150 includes a data communication or transceiver module to communicate and transmit / receive data over the communication network 150. Some non-limiting examples of a transceiver module include an antenna module, a radio frequency transceiver module, a wireless transceiver module, a Bluetooth transceiver module, an Ethernet port, a Universal Serial Bus (USB) port, or any other module I component / device configured for transmitting and receiving data.

[0043] The soft switch subsystem 130 is configured to establish, maintain, route, and terminate telecommunication sessions including mobile calls, and the voice subsystem 140 is configured to apply intelligent voice technology to the soft switch subsystem 130. The soft switch subsystem 130 and voice subsystem 140 are cooperative to provide various functionalities to users 1 10, including providing customer service to users 1 10 through the mobile calls, such as conducting dialogue interactions with users 1 10.

[0044] The soft switch subsystem 130 includes a number of artificial intelligence modules communicative with each other and configured to process mobile calls. These modules include, but are not limited to, one or more of a communication module 131 , a voice detection module 132, a speech recognition module 133, and a voice generation module 134. For example, the voice generation module 134 includes a natural language understanding module 135 and a text-to-speech module 136.

[0045] The communication module 131 provides a communication node for enabling data communication through the communication network 150. The voice detection module 132 detects whether there is human voice activity in an audio signal. For example, the voice detection module 132 is a Voice Activity Detection (VAD) module. For example, algorithms such as energy threshold algorithm, time-domain parameter algorithm, or voice domain parameter algorithm may be implemented in the voice detection module 132. The speech recognition module 133 is configured to convert the detected human voice activity into text content. For example, speech recognition algorithms based on a GMM-HMM (Gaussian Mixture Model-Hidden Markov Model) may be implemented in the speech recognition module 133. The voice generation module 134 is configured to generate a voice signal, such as a voice reply to the user 1 10. For example, the natural language understanding module 135 generates a reply text and the text-to-speech module 136 converts the reply text into speech to be played back to the user 1 10 by the mobile network operator 120.

[0046] The voice subsystem 140 includes a number of artificial intelligence modules communicative with each other and configured to process mobile calls. These modules include, but are not limited to, one or more of a communication module 141 , a voice detection module 142, a speech recognition module 143, a natural language generation module 144, a natural language understanding module 145, a text-to-speech module 146, and a dialogue management module 147.

[0047] The communication module 141 provides a communication node for enabling data communication through the communication network 150. The voice detection module 142 detects whether there is human voice activity in an audio signal. For example, the voice detection module 142 is a VAD module. The speech recognition module 143 is configured to convert the detected human voice activity into text content. For example, speech recognition algorithms based on HMM (Hidden Markov Model) or DNN-HMM (Deep Neural Network- Hidden Markov Model) may be implemented in the speech recognition module 143. The natural language generation module 144 generates natural language responses based on decisions made by the dialogue management module 147. For example, template generation or methods based on Sequence-to-Sequence (Seq2Seq) models may be implemented in the natural language generation module 144. The natural language understanding module 145 parses text, such as text generated by the speech recognition module 143, to understand its semantic content. Techniques such as Semantic Role Labelling or Dependency Parsing may be implemented in the natural language understanding module 145. The text-to-speech module 146 converts text, such as text generated by the natural language generation module 144, into speech. The dialogue management module 147 controls the flow of the conversation with the user 1 10 and decides the next action to take. For example, it determines whether to continue the conversation or end it based on the context and user input. The modules 131-136 of the soft switch subsystem 130 are integrated into the soft switch subsystem 130 and are distinct from the modules 141 -147 of the voice subsystem 140. Specifically, the modules 131 -136 of the soft switch subsystem 130 are designed for implementing a voice interruption function. For example, unlike the speech recognition module 143 of the voice subsystem 140, the speech recognition module 133 of the soft switch subsystem 130 may use a simpler acoustic model, such as GMM-HMM, as it is configured to recognize only predefined sets of interruption words. As these modules 131 -136 are integrated in the soft switch subsystem 130, no communication across the communication network 150 is required in performing the voice interruption function, allowing for quicker detection of human voice and quicker response to user interruptions.

[0048] FIG. 2 is a flowchart illustrating a call processing method 200 for processing a mobile call between the user 1 10 and the mobile network operator 120, according to various embodiments of the present disclosure. Particularly, the call processing method 200 is implemented in the call processing system 100 comprising the soft switch subsystem 130 and voice subsystem 140. The method 200 outlines the key steps involved in performing the voice interruption function.

[0049] At step 202, the communication module 131 of the soft switch subsystem 130 receives a call audio of the mobile call from the mobile network operator 120. The call audio is sent by the user 110, via the user mobile device 112, to the mobile network operator 120. More specifically, the call audio is sent to the mobile network operator 120 during playing of a first reply voice by the mobile network operator 120 to the user 110 in the mobile call. The first reply voice may originate from the voice subsystem 140 and sent to the mobile network operator 120 via the soft switch subsystem 130, such as in response to a user query in the mobile call. During the process of playing the first reply voice to the user 110, the mobile network operator 120 collects the call audio in real-time and sends the call audio to the soft switch subsystem 130.

[0050] At step 204, the voice detection module 132 of the soft switch subsystem 130 detects if the call audio contains a human voice. For example, the voice detection module 132 detects whether there are specific features of human speech in the call audio, such as the frequency, amplitude, and rhythm of the sound. The voice detection module 132 may detect that a human voice is present in the call audio if the user 1 10 makes an oral sound in the call audio. Any oral sound may be detected and the oral sound does not need to contain specific words, such as the predefined sets of interruption words. If at step 204 the voice detection module 132 detects that the call audio does not contain a human voice, the method 200 ends in step 206. If at step 204 the voice detection module 132 detects that the call audio contains a human voice, step 204 proceeds to step 208.

[0051] At step 208, the speech recognition module 133 of the soft switch subsystem 130 determines if a text content corresponding to the human voice matches any one of a number of predefined sets of interruption words. For example, a predefined set of interruption words may be a single word or may be a phrase.

[0052] If at step 208 the speech recognition module 133 determines that the text content corresponding to the human voice does not match any one of the predefined sets of interruption words, step 208 proceeds to step 210. At step 210, the communication module 131 sends a stop playback instruction to the mobile network operator 120 for the mobile network operator 120 to stop playing the first reply voice.

[0053] If at step 208 the speech recognition module 133 determines that the text content corresponding to the human voice matches one of the predefined sets of interruption words, step 208 proceeds to step 212. At step 212, the interruption voice generation module 134 generates a second reply voice based on the matched set of interruption words. For example, the natural language understanding module 135 generates a reply text based on the matched set of interruption words and the text-to-speech module 136 converts the reply text into the second reply voice.

[0054] At step 214 subsequent to step 212, the communication module 131 sends the second reply voice and the stop playback instruction to the mobile network operator 120 for the mobile network operator 120 to stop playing the first reply voice and to play the second reply voice to the user 1 10.

[0055] Accordingly, in situations when a human voice is detected in the call audio and the text content corresponding to the human voice matches one of the predefined sets of interruption words, the second reply voice is generated in response to the user’s voice input. The first reply voice stops playing and the second reply voice is played to the user 110 instead of the first reply voice. The voice input from the user 110 may be referred to as an interruption voice input and the second reply voice may be referred to as an interruption reply voice as the second reply voice interrupts the first reply voice.

[0056] In other situations when a human voice is detected in the call audio but the text content corresponding to the human voice does not match any one of the predefined sets of interruption words, the second reply voice is not generated but the stop playback instruction is still sent to the mobile network operator 120 so that the first reply voice stops playing. The first reply voice is still interrupted in response to the user’s voice input, but the user does not receive back any second reply voice to replace the first reply voice. This may happen in situations when the user 110 issues a voice input to try and interrupt the first reply voice, but may not have said words that match one of the predefined sets of interruption words. Further, the human voice in the call audio does not need to have any specific speech or voice activity to trigger the stop playback instruction.

[0057] In these situations when the human voice is detected in the call audio, the first reply voice is interrupted and stops playing to the user 1 10. As long as the call audio contains the human voice, regardless of the speech or voice activity in the human voice, the soft switch subsystem 130 will recognize the human voice and send the stop playback instruction to stop playback of the first reply voice. In yet other situations when a human voice is not detected in the call audio, i.e. the user 1 10 did not issue the interruption voice input, the soft switch subsystem 130 does not perform any further operation. The mobile network operator 120 continues playing the first reply voice to the user 1 10 without interruption.

[0058] As the modules 131 -136 are integrated in the soft switch subsystem 130, after receiving the call audio sent by the mobile network operator 120, the soft switch subsystem 130 can detect whether the call audio contains a human voice based on the voice detection module 132 contained in itself, without network transmission delay. Therefore, when the user 110 issues the interruption voice input, the soft switch subsystem 130 can promptly instruct the mobile network operator 120 to stop playing the first reply voice, thereby improving the user experience of the user 110. Moreover, when the interruption voice input issued by the user 110 matches one of the predefined sets of interruption words, the soft switch subsystem 130 can generate the second reply voice in response to the interruption voice input and instruct the mobile network operator 120 to play the second reply voice on the basis of stopping playback of the first reply voice, thereby further improving the user experience of the user 1 10.

[0059] In some embodiments as described above, the voice detection module 132 may detect that a human voice is present if the call audio contains any oral sound uttered by the user 110. In some situations, the user 1 10 may unintentionally utter an oral sound but does not want to interrupt the playback of the first reply voice, i.e. the uttered oral sound is not supposed to be the interruption voice input. The speech recognition module 133 may be adjusted and optimized to reduce the detection of such false positives, thereby improving the accuracy of detecting true interruption voice inputs intended by the user 1 10.

[0060] FIG. 3 is a swimlane diagram illustrating a communication sequence 300 for processing a mobile call between the user 1 10 and the mobile network operator 120. The sequence 300 may be implemented by the call processing system 100 described in FIG. 1 and depicts the interactions between the user device 112 of the user 1 10, the mobile network operator 1 0, the soft switch subsystem 130, and the voice subsystem 140.

[0061] At step 302, the user 1 10 uses the user device 1 12 to initiate the mobile call with the mobile network operator 120. In the mobile call, the user 110 may want to ask questions or make requests. At step 304, the user 1 10 sends a user query in the form of a first user voice via the user device 112 to the mobile network operator 120. At step 306, the mobile network operator 120 sends the first user voice to the soft switch subsystem 130. At step 308, the soft switch subsystem 130 sends the first user voice to the voice subsystem 140. At step 310, the voice subsystem 140 generates the first reply voice based on the first user voice. Specifically, the speech recognition module 143, natural language generation module 144, natural language understanding module 145, dialogue management module 147, and text-to-speech module 146 of the voice subsystem 140 are cooperative to generate the first reply voice corresponding to the first user voice. At step 312, the voice subsystem 140 sends the first reply voice to the soft switch subsystem 130.

[0062] At step 314, the soft switch subsystem 130 sends the first reply voice to the mobile network operator 120. At step 316, the mobile network operator 120 plays the first reply voice to the user 1 10 through the user device 1 12. At step 318, during playback of the first reply voice to the user 110 in the mobile call, the mobile network operator 120 receives a call audio of the mobile call in real-time from the user 1 10 via the user device 112. At step 320, the mobile network operator 120 sends the call audio to the soft switch subsystem 130.

[0063] At step 322, the voice detection module 132 integrated in the soft switch subsystem 130 detects if the received call audio contains a human voice. Specifically, after inputting the call audio into the voice detection module 132, the voice detection module 132 can perform human voice detection on the call audio based on a predefined algorithm to determine whether the call audio contains a human voice. The predefined algorithm may include, but is not limited to, energy threshold algorithm, time-domain parameter algorithm, voice domain parameter algorithm, etc.

[0064] If at step 322 the voice detection module 132 detects that the call audio does not contain a human voice, this would mean that the user 1 10 has not issued the interruption voice input during the playback of the first reply voice by the mobile communication operator 120. The soft switch subsystem 130 does not perform any further operation and the mobile network operator 120 continues at step 316 of playing the first reply voice to the user 110 without interruption. If at step 322 the voice detection module 132 detects that the call audio contains a human voice, this would mean that the user 1 10 has issued the interruption voice input during the playback of the first reply voice by the mobile communication operator 120. The voice detection module then extracts the human voice from the call audio for the speech recognition module 133 to process at step 324.

[0065] At step 324, the speech recognition module 133 integrated in the soft switch subsystem 130 determines if a text content corresponding to the detected human voice matches one of the predefined sets of interruption words. Specifically, there are one or more predefined sets of interruption words, and each predefined set of interruption words has one or more words. For example, a predefined set of interruption words may be a single word “stop”, “pause”, or “end”, but are not limited to these. For example, a predefined set of interruption words may be a phrase such as “don't want to listen”, “don’t continue”, or “don't say it”, but are not limited to these.

[0066] The speech recognition module 133 of the soft switch subsystem 130 is different from the speech recognition module 143 of the voice subsystem 140. In order to ensure the accuracy of recognizing complex speech in calls, the speech recognition module 143 of the voice subsystem 140 is configured to perform speech recognition on call speech based on the DNN- HMM model with a complex model structure. On the other hand, the speech recognition module 133 of the soft switch subsystem 130 is used to determine whether the text content corresponding to the human voice contained in the call audio is one of the predefined sets of interruption words. Therefore, the speech recognition module 133 only needs to be able to perform speech recognition on human voice that only contains any one of the predefined sets of interruption words, so that the speech recognition module 133 can perform speech recognition on human voice in the call audio based on a simple acoustic model.

[0067] In some embodiments, the speech recognition module 133 includes a set of data models configured to determine whether the text content corresponding to the human voice matches any one of the predefined sets of interruption words. Specifically, the data models are configured for converting the human voice into the text content, wherein the speech recognition module 133 determines that the text content matches one of the predefined sets of interruption words if said conversion of the human voice into the text content is successful.

[0068] Said conversion of the human voice into the text content is successful if the text content is identical to one of the predefined sets of interruption words. Specifically, when the human voice contains only one of the predefined sets of interruption words, i.e. exactly identical, the speech recognition module 133 converts the human voice into the text content based on the data models. When the speech recognition module 133 converts the human voice and outputs the text content, this would mean that the text content corresponding to the human voice is one of the predefined sets of interruption words, i.e. the matched set.

[0069] Said conversion of the human voice into the text content is unsuccessful if the text content is not any one of the predefined sets of interruption words. For example, the text content does not contain any one of the predefined sets of interruption words. For example, the text content contains any one of the predefined sets of interruption words and one or more additional words. Particularly, these additional words refer to words with actual meaning, as opposed to meaningless interjections which do not affect the recognition of the predefined sets of interruption words. In this embodiment, the speech recognition module 133 is unable to convert the human voice into the text content. When the speech recognition module 133 cannot convert the human voice and thus does not output the text content, this would mean that the text content corresponding to the human voice is not any one of the predefined sets of interruption words. In these cases, the text content may be complex text for example.

[0070] At step 324, the speech recognition module 133 converts the human voice into the text content based on the data models. The data models may include a pre-trained acoustic model, a pretrained language model, and a pre-established recognition word model. Methods for training the acoustic model and language model and for establishing the recognition word model are described below.

[0071] The acoustic model is trained using a first training sample dataset that includes multiple first training samples and multiple second training samples. Each first training sample includes a respective one of the predefined sets of interruption words and includes a respective first audio corresponding to the respective predefined set of interruption words. Each second training sample includes a training text without any of the predefined sets of interruption words and includes a respective second audio corresponding to the respective training text. The first training sample dataset is used to train a first data model, preferably a GMM-HMM model, until a first predetermined training stop condition is met, thereby obtaining the trained acoustic model. The first predetermined training stop condition may be, but is not limited to, the current training round reaching a first predetermined round threshold or the current training duration reaching a first predetermined duration threshold. The first predetermined round threshold may be, but is not limited to, 10 rounds, 20 rounds, 30 rounds, etc. The first predetermined duration threshold may be, but is not limited to, 1 hour, 5 hours, 10 hours, etc. It will be appreciated that known GMM-HMM model training methods may be used to train the GMM- HMM model using the first training sample dataset.

[0072] The language model is trained using a second training sample dataset that includes one or more of the predefined sets of interruption words and multiple other words related to the one or more of the predefined sets of interruption words. The second training sample dataset is used to train a second data model, preferably an N-gram language model, until a second predetermined training stop condition is met, thereby obtaining the trained language model. The second predetermined training stop condition may be, but is not limited to, the current training round reaching a second predetermined round threshold or the current training duration reaching a second predetermined duration threshold. The second predetermined round threshold may be, but is not limited to, 10 rounds, 20 rounds, 30 rounds, etc. The second predetermined duration threshold may be, but is not limited to, 1 hour, 5 hours, 10 hours, etc. It will be appreciated that known N-gram language model training methods may be used to train the N-gram language model using the second training sample dataset.

[0073] The recognition word model or network is established based on inter-word relationships corresponding to each of the predefined sets of interruption words. This may involve analyzing the composition and grammatical features of each predefined set of interruption words and then establishing an association network between the respective predefined set of interruption words. For example, these inter-word relationships may refer to grammatical, semantic, or phonetic associations between words. Establishing the recognition word model allows the speech recognition module 133 to constrain the inter-word relationships of the text content obtained from the conversion during speech recognition of the human voice, thereby improving the accuracy of the speech recognition module 133 in recognizing the human voice.

[0074] The data models may be adjusted and optimized to further improve the accuracy of the speech recognition module 133 in recognizing the human voice. For example, a predefined set of interruption words may still be accurately recognized when the human voice contains the predefined set of interruption words and meaningless interjections, since these meaningless interjections do not alter the essential meaning of the predefined set of interruption words and do not affect the recognition of the predefined set of interruption words. This ensures that the speech recognition module 133 can still successfully convert the human voice into the text content based on the data models.

[0075] If at step 324 the speech recognition module 133 determines that the text content corresponding to the human voice does not match any one of the predefined sets of interruption words, step 324 proceeds to step 326. At step 326, the soft switch subsystem 130 sends a stop playback instruction to the mobile network operator 120. At step 328, upon receiving the stop playback instruction, the mobile network operator 120 promptly stops playing the first reply voice to the user 1 10. The first reply voice is thus interrupted by stopping its playback in response to the interruption voice input from the user 1 10. If at step 324 the speech recognition module 133 determines that the text content corresponding to the human voice matches one of the predefined sets of interruption words, step 324 proceeds to step 330. At step 330, the interruption voice generation module 134 integrated in the soft switch subsystem 130 generates the second reply voice based on the matched set of interruption words. At step 332, the soft switch subsystem 130 sends the second reply voice and the stop playback instruction to the mobile network operator 120. At step 334, upon receiving the second reply voice and the stop playback instruction, the mobile network operator 120 promptly stops playing the first reply voice to the user 1 10 and starts playing the second reply voice to the user 110. The first reply voice is thus interrupted and replaced by the second reply voice in response to the interruption voice input from the user 110.

[0076] In some embodiments, at step 330, the natural language understanding module 135 generates an interruption reply text generated based on the matched set of interruption words, and the text-to-speech module 136 generates the second reply voice, or the interruption reply voice, based on the interruption reply text.

[0077] The natural language understanding module 135 may contain a first mapping relationship between each predefined set of interruption words and a corresponding predefined interruption reply text. The text-to-speech module 136 may contain a second mapping relationship between each predefined interruption reply text and a corresponding predefined interruption reply voice.

[0078] Specifically, at step 330, the matched set of interruption words is input into the natural language understanding module 135, which then outputs the predefined interruption reply text corresponding to the matched set of interruption words based on the first mapping relationship. The output predefined interruption reply text is then input into the text-to-speech module 136, which then outputs the predefined interruption reply voice corresponding to the predefined interruption reply text based on the second mapping relationship. Subsequently at step 332, the soft switch subsystem 130 sends the output predefined interruption reply voice as the second reply voice, together with the stop playback instruction, to the mobile network operator 120.

[0079] Different predefined sets of interruption words may map to different corresponding predefined interruption reply texts in the first mapping relationship. The natural language understanding module 135 can generate different interruption reply texts based on the respective predefined set of interruption words and the first mapping relationship, and the text-to-speech module 136 can subsequently generate different interruption reply voices. The interruption reply voices are generated through text-to-speech technology, such as using the same voice to read the interruption reply texts. Different interruption reply texts may map to different speech parameters, such as pitch and speaking speed, and may optionally result in aurally different voices. The mapping relationships can thus provide more personalized and accurate feedback based on the specific needs or instructions of the user 110.

[0080] Accordingly, the mobile network operator 120 may receive the stop playback instruction and the second reply voice at step 332 when the speech recognition module 133 determines at step 324 that the text content corresponding to the human voice matches one of the predefined sets of interruption words, or the mobile network operator 120 may receive the stop playback instruction without the second reply voice at step 332 when the speech recognition module 133 determines at step 324 that the text content corresponding to the human voice does not match any one of the predefined sets of interruption words.

[0081] The user 1 10 may issue the interruption voice input during the playback of the first reply voice by the mobile network operator 120 in the mobile call. This may happen in situations when the user 110 does not want to continue listening to the first reply voice, such as when the first reply voice does not answer the user’s initial query, and the user 110 wants to stop the first reply voice. As the modules 131 -136 are integrated in the soft switch subsystem 130, when the user 1 10 issues an interruption voice input during the playback of the first reply voice and the soft switch subsystem 130 receives the call audio from the mobile network operator 120, the voice detection module 132 can detect whether the call audio contains a human voice without network transmission delay.

[0082] Therefore, when the user 1 10 issues the interruption voice input, the soft switch subsystem 130 can promptly instruct the mobile network operator 120 to stop playing the first reply voice, thereby enhancing the user experience of the user 1 10. Moreover, when the interruption voice input matches one of the predefined sets of interruption words, the soft switch subsystem 130 can promptly instruct the mobile network operator 120 to stop playing the first reply voice and to start playing the second reply voice, thereby further enhancing the user experience of the user 1 10.

[0083] Embodiments of the present disclosure may be provided as a network of communicating devices (i.e. a “computerized network”). Embodiments of the invention may be also provided as a software application downloadable into a computer device to facilitate the method. The software application may be a computer program product, which may be stored on a non- transitory computer-readable medium on a tangible data-storage device (such as a storage device of a server, or one within a user device). FIG. 4 is a block diagram illustrating a technical architecture of a computing server for implementing the processes of FIG. 2 and FIG. 3, according to various embodiments of the present disclosure. The technical architecture 400 represents a computer server suitable for use as the soft switch subsystem 130 and the voice subsystem 140 according to various embodiments of the present disclosure. While a single computing server is shown, the methods may be implemented across multiple computers in a distributed computing environment.

[0084] The technical architecture 400 includes a processor 402 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 404 (such as disk drives), read only memory (ROM) 406, random access memory (RAM) 408. The processor 402 may be implemented as one or more CPU chips. The technical architecture may further comprise input / output (I / O) devices 410, and network connectivity devices 412.

[0085] The secondary storage 404 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 408 is not large enough to hold all working data. Secondary storage 404 may be used to store programs which are loaded into RAM 408 when such programs are selected for execution.

[0086] In this embodiment, the secondary storage 404 includes a component 404a comprising non- transitory instructions operative by the processor 402 to perform various operations of the call processing method 200 and communication sequence 300. Specifically, the component 404a implements the functionality of the various components and modules illustrated in FIG. 1 , including the communication module 131 , voice detection module 132, speech recognition module 133, voice generation module 134, natural language understanding module 135, and a text-to-speech module 136 of the soft switch subsystem 130, as well as the communication module 141 , voice detection module 142, speech recognition module 143, natural language generation module 144, natural language understanding module 145, text-to-speech module 146, and dialogue management module 147 of the voice subsystem 140.

[0087] The ROM 406 is used to store instructions and perhaps data which are read during program execution. The secondary storage 404, the RAM 408, and / or the ROM 406 may be referred to in some contexts as computer readable storage media and / or non-transitory computer readable media.

[0088] I / O devices 410 may include printers, video monitors, liquid crystal displays (LCDs), plasma displays, touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices. The processor 402 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk-based systems may all be considered secondary storage 404), flash drive, ROM 406, RAM 408, or the network connectivity devices 412. While only one processor 402 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor, the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors.

[0089] The network connectivity devices 412 enable the technical architecture 400 to communicate with external systems, which may include distributed computing resources or cloud-based services.

[0090] Although the technical architecture 400 is described with reference to a computer, it should be appreciated that the technical architecture 400 may be formed by two or more computers in communication with each other that collaborate to perform a task. For example, but not by way of limitation, an application may be partitioned in such a way as to permit concurrent and / or parallel processing of the instructions of the application. Alternatively, the data processed by the application may be partitioned in such a way as to permit concurrent and / or parallel processing of different portions of a data set by the two or more computers. In an embodiment, virtualization software may be employed by the technical architecture 400 to provide the functionality of a number of servers that is not directly bound to the number of computers in the technical architecture 400. In an embodiment, the functionality disclosed above may be provided by executing the application and / or applications in a cloud computing environment. Cloud computing may comprise providing computing services via a network connection using dynamically scalable computing resources. A cloud computing environment may be established by an enterprise and / or may be hired on an as-needed basis from a third- party provider.

[0091] By programming and / or loading executable instructions onto the technical architecture 400, at least one of the CPU 402, the RAM 408, and the ROM 406 are changed, transforming the technical architecture 400 in part into a specific purpose machine or apparatus having the novel functionality taught by the present disclosure. It is fundamental to the electrical engineering and software engineering arts that functionality that can be implemented by loading executable software into a computer can be converted to a hardware implementation by well-known design rules. Such hardware implementations may include field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), which can further enhance the system's performance for specific information processing tasks. In the foregoing detailed description, embodiments of the present disclosure in relation to a call processing system and method with voice interruption function are described with reference to the provided figures. The description of the various embodiments herein is not intended to call out or be limited only to specific or particular representations of the present disclosure, but merely to illustrate non-limiting examples of the present disclosure. The present disclosure serves to address at least one of the mentioned problems and issues associated with the prior art. Although only some embodiments of the present disclosure are disclosed herein, it will be apparent to a person having ordinary skill in the art in view of the present disclosure that a variety of changes and / or modifications can be made to the disclosed embodiments without departing from the scope of the present disclosure. It will be understood by those skilled in the art that many variations of the embodiments can be made within the scope of the following claims. Moreover, features of one or more embodiments may be mixed and matched with features of one or more other embodiments. Therefore, the scope of the disclosure as well as the scope of the following claims is not limited to embodiments described herein.

Claims

Claims1. A call processing system for processing a mobile call between a user and a mobile network operator, the call processing system comprising: a soft switch subsystem comprising: a communication module configured for receiving a call audio of the mobile call from the mobile network operator, the call audio sent by the user to the mobile network operator during playing of a first reply voice by the mobile network operator to the user in the mobile call; a voice detection module configured for detecting if the call audio contains a human voice; a speech recognition module configured for, if the call audio contains the human voice, determining if a text content corresponding to the human voice matches any one of a number of predefined sets of interruption words; and an interruption voice generation module configured for, if the text content matches one of the predefined sets of interruption words, generating a second reply voice based on the matched set of interruption words, wherein the communication module is further configured for: sending, if the text content does not match any one of the predefined sets of interruption words, a stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice; and sending, in response to said generation of the second reply voice, the stop playback instruction and the second reply voice to the mobile network operator for the mobile network operator to stop playing the first reply voice and to play the second reply voice to the user.

2. The call processing system of claim 1 , wherein the speech recognition module comprises a set of data models configured for converting, using a set of data models of the speech recognition module, the human voice into the text content, and wherein the text content matches one of the predefined sets of interruption words if said conversion of the human voice into the text content is successful.

3. The call processing system of claim 2, wherein said conversion of the human voice into the text content is successful if the text content is identical to one of the predefined sets of interruption words.

4. The call processing system of claim 2 or 3, wherein said conversion of the human voice into the text content is unsuccessful if the text content does not contain any one of the predefined sets of interruption words, or if the text content contains any one of the predefined sets of interruption words and one or more additional words.

5. The call processing system of any one of claims 2 to 4, wherein the data models comprise a pre-trained acoustic model, a pre-trained language model, and a pre-established recognition word model.

6. The call processing system of claim 5, wherein: the acoustic model is trained using a first training sample dataset comprising multiple first training samples and multiple second training samples: each first training sample comprises a respective one of the predefined sets of interruption words and comprising a respective first audio corresponding to the respective predefined set of interruption words; and each second training sample comprises a training text without any of the predefined sets of interruption words and comprising a respective second audio corresponding to the respective training text.

7. The call processing system of claim 5 or 6, wherein the language model is trained using a second training sample dataset comprising one or more of the predefined sets of interruption words and comprising multiple other words related to the one or more of the predefined sets of interruption words.

8. The call processing system of any one of claims 5 to 7, wherein the acoustic model is a GMM-HMM (Gaussian Mixture Model-Hidden Markov Model) model and / or the language model comprises an N-gram language model.

9. The call processing system of any one of claims 1 to 8, wherein the interruption voice generation module comprises a text-to-speech module configured for generating the second reply voice based on an interruption reply text generated based on the matched set of interruption words.

10. The call processing system of claim 9, wherein the interruption voice generation module comprises a natural language understanding module configured for generating the interruption reply text based on the matched set of interruption words.

11. A call processing method for processing a mobile call between a user and a mobile network operator, the call processing method comprising: receiving, by a communication module of a soft switch subsystem, a call audio of the mobile call from the mobile network operator, the call audio sent by the user to the mobile network operator during playing of a first reply voice by the mobile network operator to the user in the mobile call: detecting, by a voice detection module of the soft switch subsystem, if the call audio contains a human voice; determining, by a speech recognition module of the soft switch subsystem and if the call audio contains the human voice, if a text content corresponding to the human voice matches any one of a number of predefined sets of interruption words; sending, by the communication module and if the text content does not match any one of the predefined sets of interruption words, a stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice; generating, by an interruption voice generation module of the soft switch subsystem and if the text content matches one of the predefined sets of interruption words, a second reply voice based on the matched set of interruption words; and sending, by the communication module in response to said generation of the second reply voice, the second reply voice and the stop playback instruction to the mobile network operator for the mobile network operator to stop playing the first reply voice and to play the second reply voice to the user.

12. The call processing method of claim 1 1 , further comprising: converting, using a set of data models of the speech recognition module, the human voice into the text content; and determining, by the speech recognition module, that the text content matches one of the predefined sets of interruption words if said conversion of the human voice into the text content is successful.

13. The call processing method of claim 12, wherein said conversion of the human voice into the text content is successful if the text content is identical to one of the predefined sets of interruption words.

14. The call processing method of claim 12 or 13, wherein said conversion of the human voice into the text content is unsuccessful if the text content does not contain any one of thepredefined sets of interruption words, or if the text content contains any one of the predefined sets of interruption words and one or more additional words.

15. The call processing method of any one of claims 12 to 14, wherein the data models comprise a pre-trained acoustic model, a pre-trained language model, and a pre-established recognition word model.

16. The call processing method of claim 15, further comprising training the acoustic model using a first training sample dataset comprising multiple first training samples and multiple second training samples, wherein each first training sample comprises a respective one of the predefined sets of interruption words and comprising a respective first audio corresponding to the respective predefined set of interruption words; and wherein each second training sample comprises a training text without any of the predefined sets of interruption words and comprising a respective second audio corresponding to the respective training text.

17. The call processing method of claim 15 or 16, further comprising training the language model using a second training sample dataset comprising one or more of the predefined sets of interruption words and comprising multiple other words related to the one or more of the predefined sets of interruption words.

18. The call processing method of any one of claims 15 to 17, further comprising establishing the recognition word model based on inter-word relationships corresponding to each of the predefined sets of interruption words.

19. The call processing method of any one of claims 1 1 to 18, further comprising generating, by a text-to-speech module of the interruption voice generation module, the second reply voice based on an interruption reply text generated based on the matched set of interruption words.

20. The call processing method of claim 19, further comprising generating, by a natural language understanding module of the interruption voice generation module, the interruption reply text based on the matched set of interruption words.

Images