Concurrent reception of multiple user voice inputs for translation
By simultaneously listening and automatically detecting voice endpoints between mobile devices and auxiliary devices, a natural conversation exchange is realized, solving the problems of unnatural conversation delays and social prompts in the prior art, and improving the efficiency and intuitiveness of the translation experience.
Patent Information
- Application Number
- CN201980011421.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-02-09
- Filing Date
- 2019-02-06
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2039-02-06
AI Technical Summary
When existing mobile computing devices and auxiliary devices perform voice translation, the serial characteristics of the exchange cause increased session mode delay, and social prompts are unnatural, requiring both parties to pay close attention to who is speaking and whether the translation is output.
The microphones of mobile devices and auxiliary devices listen to voice input simultaneously, and automatically detect voice endpoints to achieve continuous listening and translation, using two independent wireless communication channels for data transmission, ensuring instant translation and output.
It realizes a more natural dialogue exchange between users and foreign language speakers, reduces conversation delays and social prompts discomfort, and improves the intuitiveness and efficiency of the translation experience.
Smart Images

Figure CN111684411B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 628,421, filed on February 9, 2018, the disclosure of which is incorporated herein by reference. Technical Field Background Art
[0003] Mobile computing devices such as mobile phones have available auxiliary devices that can provide additional functionality. For example, some earbuds provide a translation experience in which a user can touch one earbud, speak in a first language, and the phrase spoken is translated into a second language. The user can hear the translated phrase, for example, from a speaker in the mobile computing device. Similarly, a phrase spoken to a mobile phone in a second language can be translated into the first language and output through the earbud. This exchange is performed serially, so that each participant must wait for the other party's speech to be completed, translated, and output through the speaker of the mobile computing device or auxiliary device before speaking a response. The serial nature of the exchange significantly increases the delay of the conversational mode and introduces uncomfortable social cues. The interaction flow requires both parties to carefully pay attention to which person is speaking and whether the translation is being output on each device, and is therefore non-intuitive. Summary of the invention
[0004] The present disclosure provides a more natural dialogue exchange between a user and a foreign language speaker using the translation features of a mobile device and an auxiliary device. In particular, the microphones of both the mobile device and the auxiliary device are always listening for voice input. The mobile device and the auxiliary device can also determine whether the voice input is from the user or from the foreign language speaker. The mobile device and the auxiliary device can automatically determine when the spoken phrase is complete and ready for translation, send the spoken phrase for translation, and immediately start listening again. In this regard, even if another user is still speaking, in response to the translated voice of the other user, acknowledgments such as "OK", "Yes", "Oh No" or other social prompts can be captured, translated, and output through dialogue.
[0005] One aspect of the present disclosure provides a system for providing translation, which includes an auxiliary device and a mobile computing device coupled to the auxiliary device. For example, the mobile computing device can be wirelessly coupled to the auxiliary device. The auxiliary device includes: a microphone configured to continuously listen to a first input from a first user; and a communication interface adapted to send the first input. The microphone of the auxiliary device can be configured to continuously listen to the first input from the first user by continuing to listen when an endpoint in speech is detected. The mobile computing device includes: a mobile computing device microphone configured to continuously listen to a second input from a second user while the auxiliary device microphone listens to the first input from the first user; a communication interface adapted to receive the first input from the auxiliary device and send the information to the auxiliary device; one or more processors adapted to process the first input and the second input; and an output adapted to provide at least one of a processed first input or a processed second input. The microphone of the mobile computing device can be configured to continuously listen to the second input from the second user by continuing to listen when an endpoint in speech is detected. The processing of the first input and the second input by the one or more processors may include providing a translation of at least one of the first input and the second input. This may include the one or more processors of the mobile computing device performing the translation, or the one or more processors of the mobile computing device sending at least one of the first input and the second input to the remote device for translation.
[0006] The auxiliary device and the mobile device may be wirelessly coupled using two independent wireless communication channels including a first channel and a second channel, the first channel being used for transmission from the auxiliary device to the mobile device, and the second channel being used for transmission from the mobile device to the auxiliary device. The first channel and the second channel may be of different types. For example, the first channel may be an RFComm channel and the second channel may be an A2DP channel. According to some examples, any one or both of the auxiliary device and the mobile device may be configured to automatically determine the endpoint in the received voice and continue to listen to the subsequent voice immediately after the endpoint. In some examples, any one or both of the devices may also be configured to determine which of the first user or the second user is providing input. For example, the device may include a speaker recognition unit configured to determine at least one of the sound or the spoken language. In some examples, one or more processors of the mobile device are also configured to cross-reference the volume level of the input received through the auxiliary device microphone with the volume level of the input received through the mobile computing device microphone, such as to determine which of the first user or the second user is providing input. In other examples, one or more processors of the mobile device are also configured to cross-reference the waveform of the voice input received through the auxiliary device microphone with the waveform of the voice input received through the mobile computing device microphone.
[0007] Another aspect of the present disclosure provides an auxiliary device (such as earplugs), which includes: a microphone; a communication interface configured to wirelessly communicate with a computing device; an output source; and one or more processors in communication with the microphone, the communication interface, and the output source. The one or more processors can be configured to receive a first input from a first user through the microphone, detect an endpoint in the first input, receive a subsequent input from the first user after the endpoint, and send the first input to the computing device through the communication interface while receiving the subsequent input until the endpoint. According to some examples, the one or more processors can also be configured to receive a translated speech from a second user from the computing device, and output the translated speech through the output source. The communication interface can be configured to communicate with the computing device using two independent wireless communication channels including a first channel and a second channel having a different type from the first channel, the first channel being used to send information to the computing device, and the second channel being used to receive information from the computing device.
[0008] Another aspect of the present disclosure provides a mobile computing device (such as a mobile phone) comprising a microphone, a communication interface configured to wirelessly communicate with an electronic accessory, an output source, and one or more processors. The one or more processors may be configured to receive a first sound input through a microphone, detect an endpoint in the first sound input, translate the received first input until the endpoint, listen to subsequent inputs while translating, and send the translated first input to the electronic accessory for output. In some examples, the one or more processors are configured to distinguish a first sound input from a first user from a second sound input from a second user. For example, a speaker recognition unit may be configured to determine at least one of a sound or a spoken language. According to other examples, the one or more processors are also configured to receive a second input from an electronic accessory, and cross-reference the volume level of the second input with the volume level of the first input or cross-reference the waveform of the second input with the waveform of the first input.
[0009] Another aspect of the present disclosure provides a method for providing translation using an auxiliary device and a computing device wirelessly connected to the auxiliary device, the method comprising: continuously listening to a first input from a first user at a microphone of the auxiliary device, and sending the first input when the first input is received; while the auxiliary device microphone listens to the first input from the first user, continuously listening to a second input from a second user at a microphone of the computing device; receiving the first input sent from the auxiliary device at a communication interface of the computing device; processing the first input and the second input at the computing device to provide a translation of at least one of the first input and the second input; and outputting the translation of the at least one of the first input and the second input from the computing device. A non-transitory computer-readable storage medium comprising computer program instructions is also provided, which, when executed by a system comprising the auxiliary device and the computing device, causes the system to perform the steps of the method. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 is a schematic diagram of an example system in use by two users according to aspects of the present disclosure.
[0011] Figure 2 yes Figure 1 Schematic diagram of an example system.
[0012] Figure 3 It is shown Figure 1 A block diagram of components of an example system.
[0013] Figure 4 is a flow chart illustrating an example method according to aspects of the present disclosure.
[0014] Figure 5 is a flow chart illustrating another example method according to aspects of the present disclosure. DETAILED DESCRIPTION
[0015] The present disclosure provides an improved translation experience between a first user and a second user using a mobile computing device and an auxiliary device such as a pair of earbuds. The first user may be, for example, a person who speaks a foreign language, and the second user may be the owner of the mobile computing device and the auxiliary device. The microphones on both the mobile device and the auxiliary device capture input from the first user and the second user simultaneously, respectively, rather than alternating between the mobile device and the auxiliary device. Each device may determine when to create an endpoint for a voice block or send a voice block for translation, for example, based on a pause in the voice. Each device may send the received voice to an endpoint for translation and output accordingly, so that it is provided in a natural communication flow. Listening by the microphone is automatically resumed immediately after the endpoint is created, so that the voice will not be lost.
[0016] Example System
[0017] Figure 1 is a schematic diagram of an example system in use. A first user 101 provides a voice input 110. The voice input 110 may be in a foreign language for a second user 102. The voice input 110 is received at a mobile computing device 170 of the second user 102. The second user 102 also provides a second voice input 120 in a second language. The second voice input 120 may be received by both the mobile device 170 and an auxiliary device 180 worn by the second user 102. According to some examples, the auxiliary device 180 may be intended to primarily receive the second voice input 120, while the mobile device 170 is intended to primarily receive the first voice input 110. However, even in this example, the mobile device 170 may pick up some of the second voice input 120 as a side effect, and may filter out the second voice input 120. For example, the mobile device 170 may determine which input is received from the first user 101 and which input is received from the second user 102, and translate the voice from the first user 101. The auxiliary device 180 may also determine that the received input is from the second user 102, and send the input to the mobile device for translation. Each of the mobile device 170 and the auxiliary device 180 remains listening during the entire communication between the first user 101 and the second user 102 and provides translated outputs that are relevant to each other from time to time.
[0018] In the example shown, auxiliary device 180 is a pair of wireless earbuds. However, it should be understood that auxiliary device 180 can be any of a variety of different types of auxiliary devices. For example, auxiliary device 180 can be a pair of wired earbuds, headphones, a head-mounted display, a smart watch, a mobile assistant, etc.
[0019] Mobile computing device 170 may be, for example, a mobile phone, a tablet computer, a laptop computer, a gaming system, or any other type of mobile computing device. In some examples, mobile computing device 170 may be coupled to a network such as a cellular network, a wireless Internet network, etc. Translation capabilities may be stored on mobile computing device 170 or accessed from a remote source by mobile computing device 170. For example, mobile device 170 may interface with a cloud computing environment in which speech translation from a first language to a second language is performed and provided back to mobile device 170.
[0020] Figure 2 is a schematic diagram of an example system including auxiliary device 180 and mobile device 170 , and illustrates signal exchanges between mobile device 170 and auxiliary device 180 .
[0021] In this example, auxiliary device 180 is shown as a pair of earplugs, which may include, for example, a speaker portion 187 adjacent to an inner ear engagement surface 188 and an input portion 185 adjacent to an outer surface. In some examples, a user may enter input by pressing input portion 185 while speaking, or by tapping input portion 185 before speaking. In other examples, no manual user input is required, and the user may simply begin speaking. The user's voice may be received by a microphone in an earplug (not shown) or in mobile device 170. The user may hear the translated speech from another person through speaker portion 187.
[0022] Auxiliary device 180 is wirelessly coupled to mobile device 170. The wireless connection between the devices may include, for example, a short-range pairing connection, such as Bluetooth. Other types of wireless connections are also possible.
[0023] In such Figure 2 In some examples shown, different wireless protocols may be used for information transmission from earplug 180 to mobile device 170 compared to information transmission in the reverse path from mobile device 170 to earplug 180. Wireless pairing between mobile device and auxiliary device can usually use HFP / SCO channel, which allows lower quality two-way communication. This example allows replacing HFP / SCO channel with two concurrent channels. In some examples, concurrent channels can be of different types. The first channel 240 for transmitting information from mobile device 170 to auxiliary device 180 can be a high-quality one-way audio channel, such as A2DP. The second channel 250 for transmitting information from auxiliary device 180 to mobile device 170 can be, for example, an RFComm channel. This allows mobile device 170 to process two input audio streams at a time. In this regard, microphones on both mobile device 170 and auxiliary device 180 can be used concurrently, with audio being transmitted simultaneously in both directions. It should be understood that A2DP and RFComm are merely examples of different types of channels that can be used. In addition, in some examples, channels 240, 250 can be independent channels of the same type, each of which provides one-way communication.
[0024] Figure 3 An example block diagram of an auxiliary device 180 and a mobile device 170 is provided. Each device includes one or more processors 371, 381, memory 372, 382, and other components typically found in mobile computing devices and auxiliary devices. Although multiple components are shown, it should be understood that such components are merely non-limiting examples and other components may be included additionally or alternatively.
[0025] As described above, auxiliary device 180 can be any of various types of devices, such as earbuds, head-mounted devices, smart watches, etc. Mobile device 170 can also take various forms, such as smartphones, tablets, laptops, game consoles, etc.
[0026] The one or more processors 371, 381 may be any conventional processor, such as a commercially available microprocessor. Alternatively, the one or more processors may be a dedicated device, such as an application specific integrated circuit (ASIC) or other hardware-based processor. Figure 3 The processors, memories, and other elements of auxiliary device 180 and mobile computing device 170 are functionally shown as being within the same respective blocks, but one of ordinary skill in the art will appreciate that a processor or memory may actually include multiple processors or memories that may or may not be stored within the same physical housing. Similarly, a memory may be a hard drive or other storage medium located in a housing different from that of auxiliary device 180 or mobile computing device 170. Thus, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.
[0027] Memory 382 may store information accessible to processor 381, including data 384 and instructions 383 executable by processor 381. Memory 382 may be a type of memory operable to store information accessible to processor 381, including non-transitory computer-readable media or other media storing data readable by electronic devices such as hard drives, memory cards, read-only memories ("ROM"), random access memories ("RAM"), optical disks, and other writable read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of instructions 383 and data 384 are stored on different types of media.
[0028] Data 384 can be retrieved, stored or modified by processor 381 according to instruction 383. For example, although the present disclosure is not limited to a specific data structure, data 384 can be stored in a computer register, in a relational database as a table with multiple different fields and records, an XML document or a flat file. Data 384 can also be formatted in a computer readable format such as, but not limited to, a binary value, ASCII or a Unicode. As a further example only, data 384 can be stored as a bitmap composed of pixels, which are stored in compressed or uncompressed or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. In addition, data 384 can include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information used by functions to calculate relevant data.
[0029] Instructions 383 may be executed to facilitate translation performed by the mobile computing device. For example, instructions 383 allow listening and receiving a user's voice, such as through microphone 388. Microphone 388 may be beamformed so that it is directed to receive audio from the direction of the user's mouth. In this regard, auxiliary device 180 may recognize the received voice as the user's voice, rather than the voice of a foreign language speaker who is not wearing auxiliary device 180.
[0030] Instructions 383 may also allow for detecting endpoints in the received speech. For example, endpoints may be automatically determined based on any of pauses, key words, intonation, inflection, or a combination of these or other factors in the speech. Once an endpoint is detected, auxiliary device 180 may buffer the received speech while immediately resuming listening. In other examples, as an alternative or in addition to buffering, auxiliary device 180 may transmit the received speech to mobile device 170 for translation. For example, as described above in conjunction with Figure 2 As discussed, auxiliary device 180 can transmit voice via RFComm or other communication link. In some examples, all endpoint creation can be performed by mobile device 170. In such an example, auxiliary device 180 can transmit all audio streams to mobile device 170 which handles all endpoint creation, voice recognition and translation offline or online.
[0031] While the auxiliary device 180 is executing the instructions 383, the mobile device 170 may also be executing the instructions 373 stored in the memory 372 along with the data 374. For example, similar to the auxiliary device 180, the mobile device 170 may also include a memory 372 that stores data 374 and instructions 373 that may be executed by one or more processors 371. Similar to the memory 382 and data 384 of the auxiliary device 180, the memory 372 may be any of a variety of types and the data 374 may be any of a variety of formats. While the auxiliary device 180 is listening to and receiving speech from the user wearing the auxiliary device 180, the mobile device 170 may also be listening to and receiving speech through the microphone 378. The microphone 378 may not be beamformed and may receive speech from a foreign language speaker (e.g., Figure 1 ) and a user of the auxiliary device 180 (eg Figure 1 Thus, instructions 373 may allow for determining which user is providing voice input so that some voice input (e.g., from a foreign language speaker) may be translated for output via auxiliary device 180 and other voice input (e.g., from a user) may be filtered.
[0032] Any of a variety of voice recognition techniques can be used. As an example, the mobile device 170 can cross-reference the volume level between the auxiliary device microphone 388 and the mobile device microphone 378. If the sound received by the microphone 388 is quiet and the sound received by the microphone 378 is loud, it can be determined that the foreign speaker is providing voice input. On the contrary, if the sound received by the two microphones 388, 378 is loud, it can be determined that the owner / user is speaking. As another example technology, a voice recognition unit can be used. The voice recognition unit can be trained to recognize the voice of the user / owner of the auxiliary device 180 and the mobile device 170. Therefore, if the voice recognition unit detects the voice of the owner, it can ignore the voice. Similarly, the voice recognition unit can be trained to detect the language mainly spoken by the owner / user, and the voice detected in the language can be filtered out. As another example technology, audio echo cancellation technology can be used. For example, the mobile device 170 can listen to the two microphones 388, 378, detect overlapping audio, and recognize that the overlapping audio belongs to the owner. Overlapping audio may be detected by identifying similar waveforms or patterns of acoustic inputs, or detecting similar plosives or transient attacks.In some examples, any combination of the foregoing techniques or other techniques may be used.
[0033] When the detected speech is from a foreign language speaker, instruction 373 may further allow continued listening until an endpoint is detected. As described above, an endpoint may be detected based on pauses, keywords, intonation, or other factors. The received speech from the foreign language speaker is buffered, and microphone 378 resumes listening.
[0034] The mobile device 170 can perform translation of both the foreign language speaker input received by the microphone 378 and the owner input received by the communication from the auxiliary device 180. Such translation can be performed on the mobile device 170 itself, or can be performed using one or more remote computing devices (such as the cloud). For example, the mobile device 170 can upload the voice for translation to the remote computing network that performs the translation, and receive the response including the translated voice from the remote computing network. The translation of the voice from the foreign language speaker can be provided to the auxiliary device 180 to be output through the output 387. The translation from the owner's voice can be output through the output 377 of the mobile device 170. The outputs 377, 387 can each include, for example, one or more speakers adapted to provide audio output. In some examples, the outputs 377, 387 can also include one or more other types, such as a display, tactile feedback, etc.
[0035] It should be understood that the auxiliary device 180 and the mobile device 170 may each include other components not shown, such as charging inputs for batteries, signal processing components, etc. Such components may also be used in the execution of the instructions 383 , 373 .
[0036] Example Method
[0037] In addition to the operations described above and shown in the accompanying drawings, various operations will now be described. It should be understood that the following operations need not be performed in the exact order described below. Instead, the various steps may be processed in a different order or simultaneously, and steps may also be added or omitted.
[0038] Figure 4 The method 400 executed by the mobile device 170 is shown. In block 410, the mobile device listens for and receives voice input. Although the mobile device is intended to capture speech such as a foreign language speaker (e.g. Figure 1 101), but it can also capture the voice of the primary user (e.g. Figure 1 The device can continuously listen and receive input without manual interaction such as pressing a button.
[0039] In block 420, the mobile device determines whether the received sound input is from the mobile device owner or from a foreign language speaker. For example, the mobile device can use voice recognition, language recognition, etc. As another example, the mobile device can cross-reference the volume of the sound received at the mobile device with the volume of the sound received at the auxiliary device and relayed to the mobile device.
[0040] If it is determined in block 430 that the input is from the mobile device owner, the mobile device can ignore the input. Therefore, the method returns to block 410 to continue listening for input from a foreign language speaker. However, if it is determined that the input is from a foreign language speaker, the method proceeds to block 440.
[0041] In block 440, the mobile device determines whether an endpoint in the speech is detected. For example, if there is a pause in the speech input within a predetermined time period such as half a second, one second, two seconds, etc., the mobile device can determine that the speech endpoint has been reached. Other examples of detecting endpoints can include detecting changes in tone or intonation, or detecting keywords. Detecting endpoints helps to ensure the correct translation of complete phrases. For example, the translation of each word is often inaccurate. As an example only, while adjectives are usually said before their related nouns in English, in Spanish adjectives are said after the noun. By creating an endpoint for the speech after the complete statement, the phrase can be translated as a whole, and the words can be rearranged to be suitable for the language translated.
[0042] If no endpoint is detected, the mobile device keeps listening and waiting for the endpoint in block 445. However, if an endpoint is detected, the mobile device buffers the received input in block 450. It should be understood that buffering can be performed while receiving input before the endpoint is detected.
[0043] In block 460, the mobile device translates the buffered voice input to the endpoint. The translation may be performed at the mobile device or by a remote computing device.
[0044] In block 470, the translated input is provided to the auxiliary device for output. Figure 2 The translated input is provided via one of the two communication channels described in .
[0045] Although not shown, the mobile device also receives speech from the auxiliary device. For example, the auxiliary device may be combined with the following Figure 5 The device may receive speech from the owner and provide the received speech to the mobile device for translation. Once the translation is complete, the mobile device may output the translated speech for the foreign language speaker. For example, the mobile device may transmit an audible translation of the speech through a speaker for the foreign language speaker to hear, may provide a text translation of the speech on a display, and the like.
[0046] Figure 5 A method 500 is shown for execution by a secondary device to receive a translation. The method 500 may be executed by the secondary device concurrently with the mobile device executing the method 400.
[0047] In block 510, the auxiliary device listens and receives voice input from the owner. In the case where the microphone in the auxiliary device is beamformed, it can be determined that any input received is from the owner. In other examples, voice recognition technology such as the voice recognition technology described above can be used.
[0048] In block 520, the auxiliary device determines whether an endpoint has been reached. The endpoint can be automatically detected based on a pattern in the voice. In other examples, the endpoint can be manually entered by the owner, such as by pressing a button. If the endpoint is not detected, the device continues to listen in block 525 until the endpoint is detected. During and / or after reception, the received voice can be buffered at the auxiliary device. In some alternatives, instead of the auxiliary device performing endpoint creation, the auxiliary device continuously streams the received audio to the mobile device. The mobile device can run two sound recognizers simultaneously to detect sounds and endpoints accordingly.
[0049] In block 530, the received speech from the owner is transmitted to the mobile device for translation and output. Figure 2 As discussed, speech may be sent via the second communication channel. The speech may be sent simultaneously with speech received from the mobile device (block 540) for output by the auxiliary device (block 550).
[0050] By keeping the microphones on both the auxiliary device and the mobile device continuously open, speech from both the owner and the foreign speaker is continuously received. In this regard, the owner and the foreign speaker can have a more natural conversation including interjections, affirmations, acknowledgments, etc., and there are no awkward pauses while waiting for translation. Through automatic endpoint creation, phrases or other speech blocks can be detected and translated without manual input from the user. In addition, sound recognition technology can be used to determine which user is providing input, and therefore determine how the input should be translated. Therefore, less user manipulation of the device is required. Instead, the user can talk naturally, and the auxiliary device and the mobile device can automatically provide assistance, allowing near real-time translation.
[0051] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be considered as an illustration rather than a limitation of the subject matter defined by the claims. In addition, the provision of the examples described herein and the expressions "such as", "including", etc. should not be interpreted as limiting the subject matter of the claims to specific examples; rather, these examples are intended to illustrate only one of many possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.
Claims
1. A system for providing translation, comprising: Auxiliary equipment, including: an auxiliary device microphone configured to continuously listen for a first voice input from a first user; and an auxiliary device communication interface configured to transmit the first voice input; and A mobile computing device wirelessly coupled to an auxiliary device, the mobile computing device comprising: a mobile computing device microphone configured to continuously listen for a second voice input from a second user while the secondary device microphone listens for a first voice input from a first user; a mobile computing device communication interface configured to receive a first voice input from an auxiliary device and to send information to the auxiliary device; Processor, configured as: detecting an endpoint in a second voice input, processing the first voice input and the second voice input, and providing a translation of at least one of the first speech input and the second speech input; and output, configured to provide a translation of at least one of the first speech input or the second speech input, The processor of the mobile computing device is further configured to cross-reference a waveform of a voice input received through the auxiliary device microphone with a waveform of a voice input received through the mobile computing device microphone.
2. The system according to claim 1, wherein: The auxiliary device and the mobile computing device are wirelessly coupled using two independent wireless communication channels including a first channel for transmissions from the auxiliary device to the mobile computing device and a second channel for transmissions from the mobile computing device to the auxiliary device.
3. The system according to claim 2, wherein: The first channel and the second channel are of different types.
4. The system according to claim 3, wherein: The first channel is an RFComm channel and the second channel is an A2DP channel.
5. A system according to any preceding claim, wherein: The auxiliary device also includes at least one auxiliary device processor configured to determine endpoints in the first speech input.
6. The system according to claim 1, wherein: The processor of the mobile computing device is also configured to determine endpoints in the first voice input.
7. The system according to claim 6, wherein: The mobile computing device microphone is configured to continue listening immediately after the endpoint.
8. The system according to claim 1, wherein: The processor of the mobile computing device is configured to determine which of the first user or the second user is providing input based on the cross-reference.
9. The system according to claim 8, wherein: The mobile computing device also includes a speaker recognition unit configured to determine at least one of a voice or a spoken language.
10. The system according to claim 8, wherein: The processor of the mobile computing device is further configured to cross-reference a volume level of input received through the auxiliary device microphone with a volume level of input received through the mobile computing device microphone to determine which of the first user or the second user is providing the input.
11. An auxiliary device comprising: microphone; a communication interface configured to communicate wirelessly with a computing device; as well as a processor in communication with the microphone and the communication interface, the processor being configured to: receiving a first voice input from a first user through a microphone; Detecting endpoints in a first speech input; receiving subsequent voice input from the first user after the endpoint; as well as while receiving subsequent voice input, sending the first voice input to the computing device through the communication interface to the endpoint; Wherein the processor of the computing device is configured to cross-reference the waveform of the voice input received through the microphone of the auxiliary device with the waveform of the voice input received through the microphone of the computing device.
12. The auxiliary device according to claim 11, wherein: The processor of the auxiliary device is further configured to: receiving, from the computing device, translated speech from the second user; and The translated speech is output through a speaker or a display.
13. An auxiliary device according to claim 11 or 12, wherein: Assistive equipment includes earplugs.
14. The auxiliary device according to claim 11, wherein: The communication interface is configured to communicate with the computing device using two independent wireless communication channels including a first channel for transmitting information to the computing device and a second channel that is a different type than the first channel for receiving information from the computing device.
15. A mobile computing device comprising: microphone; a communication interface configured to communicate wirelessly with the electronic accessory; as well as Processor, configured as: receiving a first sound input via a microphone; detecting an endpoint in a first sound input; translating a received first sound input to a detected endpoint; listening to subsequent sound input while translating the first sound input received; as well as sending the translated first sound input to the electronic accessory for output, and The processor is further configured to: receiving a second sound input from the electronic accessory; and The waveform of the second sound input is cross-referenced with the waveform of the first sound input.
16. The mobile computing device of claim 15, wherein: The processor is configured to distinguish a first voice input from a first user from a second voice input from a second user based on the cross-reference.
17. The mobile computing device of claim 15, further comprising a speaker recognition unit configured to determine at least one of a voice or a spoken language.
18. The mobile computing device of claim 15, wherein: The processor is further configured to: receiving a second sound input from the electronic accessory; and The volume level of the second sound input is cross-referenced with the volume level of the first sound input to determine which of the first user or the second user is providing the input.
19. A method of providing translation using a computing device wirelessly coupled to an auxiliary device, the method comprising: listening at the microphone of the computing device for a second input from a second user while the auxiliary device microphone listens for the first input from the first user; receiving, at a communication interface of the computing device, a first input from a first user from an auxiliary device; cross-referencing a waveform of speech input received through a microphone of the auxiliary device with a waveform of speech input received through a microphone of the computing device; detecting an endpoint in at least a second input; processing the first input and the second input to provide a translation of at least one of the first input and the second input; as well as A translation of the at least one of the first input and the second input is output.
20. A non-transitory computer-readable storage medium comprising computer program instructions that, when executed by a computing device wirelessly coupled to an auxiliary device, cause the computing device to perform the method of claim 19.
