Voice input device and input voice processing method

The voice input device uses multiple microphones to adapt sound collection strategies based on ambient noise, enhancing voice recognition for AI assistants in noisy environments.

JP2024169667AInactive Publication Date: 2024-12-05JVC KENWOOD CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024166209
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-25
Publication Date
2024-12-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In loud ambient sound environments, voice input devices like wireless earphones struggle to accurately capture and transmit user voices to AI assistants due to interference from surrounding noise, leading to poor recognition rates.

Method used

A voice input device with multiple microphones positioned at different locations - outside and inside the ear canal - adjusts sound collection based on ambient noise levels, selectively using air-conducted or bone-conducted sound to enhance voice recognition.

Benefits of technology

The device maintains high voice recognition rates for AI assistants by minimizing ambient noise interference, ensuring clear voice transmission even in noisy conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169667000001_ABST
    Figure 2024169667000001_ABST
Patent Text Reader

Abstract

To provide a voice input device with a high recognition rate by an AI assistant concerning uttered voice even when peripheral sound is large.SOLUTION: A voice input device (91) comprises: a first microphone (M1) for collecting voice at a first position on an outside of the external auditory canal (E1) of a speaker (H) and transmitting a first input voice signal; a second microphone (M2) for collecting voice at a second position closer to a mouth than the first position on an outside of the external auditory canal (E1) and transmitting a second input voice signal (SN2); a third microphone (M3) for collecting voice inside the external auditory canal (E1) of the speaker (H) and transmitting a third input voice signal (SN3); and a control section (3) for detecting a sound pressure (Va) of the first input voice signal (SN1), setting respective reflection degrees (FR2, FR3) of the second input voice signal (SN2) and the third input voice signal (SN3) in response to the voice pressure (Va), and generating an output voice signal (SNt) including at least one of the second input voice signal (SN2) and the third input voice signal (SN3) on the basis of the reflection degrees (FR2, FR3).SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a voice input device and an input voice processing method. [Background technology]

[0002] Patent Document 1 describes a wireless headset equipped with a microphone and earphones that is used as a voice input device to transmit the user's voice to an AI assistant. Patent Document 2 describes a wireless earphone with a microphone that serves as a voice input device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-030780 A [Patent Document 2] JP 2019-195179 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a speaker speaks to an AI assistant using a voice input device such as a wireless earphone with a microphone while the surrounding noise is loud, the loud surrounding noise is picked up by the microphone of the voice input device along with the spoken voice and sent to the AI ​​assistant. As a result, there is a risk that the AI ​​assistant will not be able to recognize the user's voice and will not be able to respond appropriately.

[0005] Therefore, the problem to be solved by the present invention is to provide a voice input device and an input voice processing method that enable an AI assistant to recognize spoken voice at a high rate even when the ambient sound is loud. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention has the following configurations 1) to 3). 1) a first microphone that picks up a voice at a first position outside the ear canal of a speaker and outputs a first input voice signal; a second microphone that picks up a sound at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position and outputs a second input sound signal; a third microphone that picks up a voice in the ear canal of the speaker and outputs a third input voice signal; a control unit that detects a sound pressure of the first input audio signal, sets a reflection degree of each of the second input audio signal and the third input audio signal in accordance with the detected sound pressure, and generates an output audio signal including at least one of the second input audio signal and the third input audio signal based on the reflection degree; a communication unit that transmits the output audio signal to an external device; It is a voice input device equipped with. 2) acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of a speaker, and detecting a sound pressure of the first input audio signal; acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position; Acquiring a sound picked up in the ear canal of the speaker as a third input sound signal; The input audio processing method selectively sets one of the second input audio signal and the third input audio signal as an output audio signal according to the sound pressure of the first input audio signal, and transmits the output audio signal to an outside. 3) acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of the speaker, and detecting a sound pressure of the first input audio signal; acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position; Acquiring a sound picked up in the ear canal of the speaker as a third input sound signal; The input audio processing method mixes the second input audio signal and the third input audio signal at a sound pressure ratio according to the sound pressure of the first input audio signal, sets the mixed signal as an output audio signal, and transmits the output audio signal to the outside. Effect of the Invention

[0007] According to the present invention, the effect is that even if the surrounding noise is loud, the AI ​​assistant can recognize the spoken voice with a high rate. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic cross-sectional view showing an earphone 91 which is a first embodiment of a voice input device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of the earphone 91. [Diagram 3] FIG. 3 is a diagram showing the operation of the earphone 91. [Figure 4] FIG. 4 is a block diagram showing an earphone 91A which is a first variation of the earphone 91. As shown in FIG. [Diagram 5] FIG. 5 is a diagram showing the operation of the earphone 91A. [Figure 6] FIG. 6 is a diagram showing the operation of the second modified example of the earphone 91. [Figure 7] FIG. 7 is a diagram showing the operation of the third modified example of the earphone 91. [Figure 8] FIG. 8 is a diagram showing the operation of the fourth modified example of the earphone 91. [Figure 9] FIG. 9 is a block diagram showing an earphone system 91ST which is Example 2 of a voice input system according to an embodiment of the present invention. [Figure 10] FIG. 10 is a table showing the operation of the earphone system 91ST. [Figure 11] FIG. 11 is a schematic cross-sectional view showing an example of the arrangement of the third microphone M3 when it is a bone conduction microphone. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Example 1 A voice input device according to an embodiment of the present invention will be described using an earphone 91 of a first embodiment with reference to FIGS.

[0010] Fig. 1 is a vertical cross-sectional view showing an earphone 91 of Example 1. In Fig. 1, the earphone 91 is shown in use, being attached to the auricle E of a speaker H.

[0011] FIG. 2 is a block diagram of the earphone 91.

[0012] The earphone 91 has a main body 1 and an insertion portion 2 that protrudes from the main body 1 and is inserted into the ear canal E1.

[0013] The main body 1 includes a first microphone M1, a second microphone M2, a control unit 3, a communication unit 4, a drive unit 5, and a speaker unit 6.

[0014] The insertion section 2 has a third microphone M3.

[0015] The control unit 3 includes a sound pressure detection unit 3a and an input selection unit 3b.

[0016] The main body 1 has an air chamber 1a on the sound output side of the speaker unit 6.

[0017] The insertion section 2 has a sound emission path 2a that opens at the tip and communicates with the air chamber 1a.

[0018] When the earphone 91 is in use, sound output from the speaker unit 6 by the operation of the drive section 5 passes through the air chamber 1a and the sound output path 2a, and is output into the ear canal E1.

[0019] This allows the earphone 91 to receive an audio signal wirelessly transmitted from an external audio playback device via the communication unit 4 and reproduce the audio signal at the speaker unit 6 via the control unit 3 and the drive unit 5.

[0020] The first microphone M1 is disposed at a first position on the main body 1, which is a portion farther from the mouth of the speaker H, and picks up sounds around the main body 1.

[0021] The second microphone M2 is disposed at a second position on the main body 1 that is closer to the mouth of the speaker H, and mainly collects the sound uttered by the speaker H as air-conducted sound. That is, when the earphones 91 are in use, the second microphone M2 is located closer to the mouth of the speaker H than the first microphone M1.

[0022] Hereinafter, the sound around the main body 1 will also be simply referred to as the ambient sound.

[0023] The third microphone M3 is an air-conduction sound microphone and is arranged at a third position facing the sound emission path 2a of the insertion portion 2. The third microphone M3 picks up air-conduction sound generated when the sound emitted by the speaker H reaches the ear canal E1 as bone-conduction sound and reverberates in the ear canal E1 and the internal space Ev of the sound emission path 2a.

[0024] That is, the first position of the first microphone M1 is outside the ear canal of the speaker H. The second position of the second microphone M2 is outside the ear canal of the speaker H and closer to the mouth of the speaker H than the first position. The third microphone M3 is inside the ear canal of the speaker H.

[0025] The sound pressure detection section 3a of the control section 3 detects the sound pressure of the input sound signal SN1, which is the first input sound signal received from the first microphone M1, and outputs it as a detected first sound signal SN1a.

[0026] The sound pressure is detected, for example, as an equivalent continuous-tone sound level (LAeq). The sound pressure of the detected first sound signal SN1a detected as the equivalent continuous-tone sound level (LAeq) by the sound pressure detector 3a is hereinafter referred to as a sound pressure Va.

[0027] As described above, the first microphone M1 mainly picks up the ambient sound, and therefore the sound pressure Va can be regarded as the sound pressure of the ambient sound.

[0028] As shown in FIG. 2, the input selection unit 3b receives an input audio signal SN2 which is the second input audio signal from the second microphone M2, an input audio signal SN3 which is the third input audio signal from the third microphone M3, and a detected first audio signal SN1a from the sound pressure detection unit 3a.

[0029] The input selection unit 3b of the control unit 3 generates and outputs an output audio signal SNt to the communication unit 4.

[0030] At this time, the input selection unit 3b of the control unit 3 sets the reflection degrees RF1 and RF2 of the input sound signals SN2 and SN3 in the output sound signal SNt, respectively, based on the sound pressure Va of the detected first sound signal SN1a.

[0031] The reflection degree RF1 and the reflection degree RF2 are indexes indicating the degree of reflection of the input audio signal SN2 and the input audio signal SN3, respectively, to the output audio signal SNt.

[0032] The index is, for example, the magnitude of sound pressure.

[0033] In this example, the input selection unit 3b sets the reflection degree RF1 and the reflection degree RF2 as an alternative between one being reflected and the other not being reflected.

[0034] That is, the input selection unit 3b selectively sets and generates the output audio signal SNt to switch to either the input audio signal SN2 or the input audio signal SN3 based on the sound pressure Va of the detected first audio signal SN1a. As a result, the output audio signal SN1 includes at least one of the input audio signal SN2 and the input audio signal SN3.

[0035] The communication unit 4 wirelessly transmits the output audio signal SNt output from the input selection unit 3b to the outside of the earphone 91. The wireless transmission is performed by, for example, Bluetooth (registered trademark).

[0036] Next, the input voice processing method by the operation of the input selection unit 3b will be described in detail with reference to FIG.

[0037] FIG. 3 is a diagram in which the horizontal axis represents sound pressure Va and the vertical axis represents the input audio signal SN2 of the second microphone M2 and the input audio signal SN3 of the third microphone M3, which are alternatively selected as the output audio signal SNt.

[0038] As arbitrary values ​​of the sound pressure Va, a first sound pressure, a lower switching sound pressure Va1, and a second sound pressure, an upper switching sound pressure Va2, which is higher than the lower switching sound pressure Va1, are preset.

[0039] When the sound pressure Va is less than the switching lower sound pressure Va1, the input selection unit 3b selects and sets the input sound signal SN2 as the output sound signal SNt to be generated, and when the sound pressure Va is greater than the switching upper sound pressure Va2, the input selection unit 3b selects and sets the input sound signal SN3 as the output sound signal SNt to be generated.

[0040] When the input audio signal SN2 is selected and set as the output audio signal SNt, if the sound pressure Va increases and exceeds the upper switching sound pressure Va2, the input selection unit 3b switches the input audio signal SN2 to the input audio signal SN3.

[0041] When the input audio signal SN3 is selected and set as the output audio signal SNt, if the sound pressure Va decreases and becomes less than the switching lower sound pressure Va1, the input selection unit 3b switches the input audio signal SN3 to the input audio signal SN2.

[0042] That is, when the ambient sound is low, the earphone 91 outputs the voice of the speaker H, which is picked up by the second microphone M2 as air-conducted sound outside the ear canal E1, to the outside as an output audio signal SNt.

[0043] Furthermore, when the ambient sound is loud, the earphone 91 transmits to the outside the voice of the speaker H, which is picked up by the third microphone M3 as air-conducted sound that has passed through bone conduction in the external auditory canal E1, as an output audio signal SNt.

[0044] The voice of speaker H picked up in the ear canal E1 as air-conducted sound via bone conduction or as bone-conducted sound is less clear than the voice picked up outside the ear canal E1 as air-conducted sound, but it is obtained with a stable sound pressure with almost no influence from ambient sounds.

[0045] Therefore, even if the surrounding sound is loud, the earphone 91 can output the output audio signal SNt with a high sound pressure of the voice of the speaker H without being buried in the surrounding sound.

[0046] Furthermore, when the ambient sound is low, the voice of the speaker H is picked up outside the ear canal E1 as air-conducted sound, the sound pressure of the voice of the speaker H is relatively high, and a clearer output voice signal SNt can be sent out.

[0047] 3, in the earphone 91, the upper switching sound pressure Va2 at which the input selection unit 3b switches the output audio signal SNt from the input audio signal SN2 to the input audio signal SN3 and the lower switching sound pressure Va1 at which the input audio signal SN3 switches to the input audio signal SN2 are set to different values. Specifically, the upper switching sound pressure Va2 is set higher than the lower switching sound pressure Va1.

[0048] By making the value of the upper switching sound pressure Va2 different from the value of the lower switching sound pressure Va1, when the ambient sound picked up by the first microphone M1 frequently fluctuates between the lower switching sound pressure Va1 and the upper switching sound pressure Va2, it is possible to avoid a phenomenon in which the input audio signal SN2 and the input audio signal SN3 frequently switch and the sound pressure or sound quality of the output audio signal SNt becomes unstable. This prevents a deterioration in the voice recognition of the AI ​​assistant 81 due to fluctuations in the sound pressure of the ambient sound of the earphone 91.

[0049] In addition, by setting the upper switching sound pressure Va2 to be greater than the lower switching sound pressure Va1, a problem is prevented in which the input audio signal to be selected is not switched to when the increase / decrease fluctuation of the sound pressure Va is reversed between the lower switching sound pressure Va1 and the upper switching sound pressure Va2.

[0050] The lower switching sound pressure Va1 and the upper switching sound pressure Va2 are appropriately set by the manufacturer so as to maintain a high recognition rate of the AI ​​assistant 81 in accordance with the usage environment of the earphones 91. In addition, without being limited to this, the speaker H may be allowed to adjust the lower switching sound pressure Va1 and the upper switching sound pressure Va2 in accordance with the usage environment of the earphones 91.

[0051] As described above, the earphone 91 maintains the voice of the speaker H at a high sound pressure in the output audio signal SNt generated by the control unit 3 and transmitted from the communication unit 4, regardless of the volume of the sound around the main body 1.

[0052] As a result, the AI ​​assistant 81 that receives the output voice signal SNt can improve the recognition rate of the voice uttered by the speaker H.

[0053] The first embodiment described above in detail is not limited to the above-mentioned configuration and procedure, and may be modified without departing from the gist of the present invention. (Variation 1) FIG. 4 is a block diagram showing an earphone 91A which is a variation of the earphone 91, and FIG. 5 is a diagram showing the operation of the earphone 91A.

[0054] As shown in FIG. 4, the earphone 91A is configured in such a way that the input selection unit 3b of the earphone 91 is replaced with an input mixing unit 3c, but the rest of the configuration is the same as that of the earphone 91.

[0055] The input mixer 3c receives the input audio signal SN2 from the second microphone M2, the input audio signal SN3 from the third microphone M3, and the detected first audio signal SN1a from the sound pressure detector 3a.

[0056] The input mixer 3c mixes the input audio signal SN2 and the input audio signal SN3 at a sound pressure ratio according to the sound pressure Va of the detected first audio signal SN1a, and outputs the mixed signal to the communication unit 4 as an output audio signal SNt.

[0057] The input mixer 3c of the control unit 3 sets the reflection degree RF2 and the reflection degree RF3 of the input audio signal SN2 and the input audio signal SN3 in the output audio signal SNt by the ratio of their sound pressures. The sound pressure ratio is the ratio of the sound pressures of the input audio signal SN2 and the input audio signal SN3 included in the output audio signal SNt.

[0058] The input voice processing method by the operation of the input mixer 3c will be described with reference to FIG.

[0059] In FIG. 5, the horizontal axis is a linear axis of sound pressure Va, the left vertical axis is a linear axis of mixed sound pressure V of input sound signal SN2 and input sound signal SN3, and the right vertical axis is a linear axis of total sound pressure Vt of output sound signal SNt.

[0060] The total sound pressure Vt is the sound pressure of the sound signal obtained by mixing the input sound signal SN2 and the input sound signal SN3, and includes the case where either one of them is 0 (zero).

[0061] 5, a mixed lower limit sound pressure Va3 and a mixed upper limit sound pressure Va4 that is greater than the mixed lower limit sound pressure Va3 are set in advance as arbitrary values ​​for the sound pressure Va. Hereinafter, the range of the sound pressure Va that is equal to or greater than the mixed lower limit sound pressure Va3 and equal to or less than the mixed upper limit sound pressure Va4 is also referred to as the mixed range R of the sound pressure Va.

[0062] Furthermore, a minimum mixed sound pressure Vmin, which is the minimum sound pressure to be mixed, and a maximum mixed sound pressure Vmax, which is the maximum sound pressure to be mixed, are set in advance for the input audio signal SN2 and the input audio signal SN3. The minimum mixed sound pressure Vmin may be 0 (zero).

[0063] When the sound pressure Va is less than the mixing lower limit sound pressure Va3, the input mixing unit 3c sets the input sound signal SN2 to the mixing maximum sound pressure Vmax and sets the input sound signal SN3 to the mixing minimum sound pressure Vmin.

[0064] When the sound pressure Va exceeds the mixed upper limit sound pressure Va4, the input mixing unit 3c sets the input sound signal SN2 to the mixed minimum sound pressure Vmin and sets the input sound signal SN3 to the mixed maximum sound pressure Vmax.

[0065] In the mixing range R of the sound pressure Va, the input mixing unit 3c decreases the mixed sound pressure V for the input sound signal SN2 as the sound pressure Va increases, and increases the mixed sound pressure V for the input sound signal SN3 as the sound pressure Va increases.

[0066] That is, as the sound pressure Va increases, the input mixer 3c decreases the reflection degree FR2 of the input sound signal SN2 and increases the reflection degree FR3 of the input sound signal SN3.

[0067] The input mixer 3c increases or decreases the mixed sound pressure V relative to the sound pressure Va, for example, linearly, within the mixing range R of the sound pressure Va.

[0068] As a result, the input mixing unit 3c mixes the input audio signal SN2 and the input audio signal SN3 at an arbitrary sound pressure Vax in the mixing range R at the sound pressure Va with a mixed sound pressure V2x and a mixed sound pressure V3x corresponding to the sound pressure Vax, respectively, to generate an output audio signal SNt, and outputs the generated output audio signal SNt to the communication unit 4.

[0069] By the operation of the input mixer 3c described above, the total sound pressure Vt of the output audio signal SNt becomes a constant total sound pressure Vtc, regardless of the magnitude of the sound pressure Va.

[0070] The mixed lower limit sound pressure Va3, the mixed upper limit sound pressure Va4, the mixed minimum sound pressure Vmin, and the mixed maximum sound pressure Vmax are appropriately set by the manufacturer depending on the usage environment of the earphones 91A, etc., so that the voice recognition rate of the AI ​​assistant 81 is maintained high.

[0071] The lower mixture sound pressure limit Va3, the upper mixture sound pressure limit Va4, the minimum mixture sound pressure Vmin, and the maximum mixture sound pressure Vmax may be adjustable by the speaker H.

[0072] According to the earphone 91A of the first variant example, when the sound pressure Va of the ambient sound is within a mixing range R between the lower mixing limit sound pressure Va3 and the upper mixing limit sound pressure Va4, the input audio signal SN2 and the input audio signal SN3 are mixed at a sound pressure ratio of the reflection degrees FR2, FR3 according to the sound pressure Va, and the ratio of the mixed sound pressures changes gradually linearly according to the increase or decrease in the sound pressure of the ambient sound of the main body 1.

[0073] For example, the reflection degree FR2 in the output sound signal SNt is represented by Vmax / Vmin when the sound pressure Va is Va3, by V2x / V3x when the sound pressure Vax, and by Vmin / Vmax when the sound pressure Va4.

[0074] The reflection degree FR3 in the output audio signal SNt is expressed as Vmin / Vmax when the sound pressure Va is the sound pressure Va3, as V3x / V2x when the sound pressure Vax, and as Vmax / Vmin when the sound pressure Va4.

[0075] As a result, the change in sound quality of the output audio signal SNt in response to an increase or decrease in the ambient sound becomes gradual and smooth, and the recognition rate of the voice uttered by the speaker H by the AI ​​assistant 81 is maintained high regardless of the sound pressure of the ambient sound of the main body unit 1.

[0076] Furthermore, with the earphone 91A, the total sound pressure Vt of the output audio signal SNt remains constant and does not change suddenly regardless of whether the ambient sound increases or decreases, so the recognition rate of the voice uttered by the speaker H by the AI ​​assistant 81 is maintained at a higher level. (Variation 2) The earphone 91A may be an earphone 91B (see FIG. 4) of variant example 2 in which, instead of the input mixing section 3c that keeps the total sound pressure Vt of the output audio signal SNt constant regardless of the sound pressure Va, an input mixing section 3cB (see FIG. 4) that changes the total sound pressure Vt according to the sound pressure Va, as shown in FIG. 6.

[0077] For example, in the mixing range R of the sound pressure Va, the input mixer 3cB increases the total sound pressure Vt as the sound pressure Va increases.

[0078] In detail, the input mixer 3cB performs the mixing operation with the mixed maximum sound pressure V2max of the input audio signal SN2 and the mixed maximum sound pressure V3max of the input audio signal SN3 being different values, as shown in Fig. 6. For example, the mixed maximum sound pressure V3max is set to be greater than the mixed maximum sound pressure V2max.

[0079] As a result, the sound pressure of the output sound signal SNt increases and decreases between a total sound pressure Vt1 at the mixture lower limit sound pressure Va3 and a total sound pressure Vt2 that is greater than the total sound pressure Vt1 at the mixture upper limit sound pressure Va4.

[0080] When the total sound pressure Vt is constant, if the sound pressure Va is large, that is, the ambient sound is large, the sound pressure ratio of the ambient sound contained to a certain extent as background noise in the input sound signal SN2 becomes high. Therefore, the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output sound signal SNt becomes relatively high.

[0081] In contrast, in the earphone 91B, the mixing ratio of the sound pressure of the input sound signal SN3 to the input sound signal SN2 increases as the sound pressure Va increases in the mixing range R of the sound pressure Va. This suppresses an increase in the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output sound signal SNt.

[0082] This allows the speech recognition rate by the AI ​​assistant 81 that receives the output speech signal SNt to be stably maintained. (Variation 3) The earphone 91A of the first modification may be modified to an earphone 91C of a third modification (see FIG. 4) in which the input mixer 3c is replaced with an input mixer 3cC (see FIG. 4) that nonlinearly decreases and increases the input.

[0083] As shown in Figure 7, in the input mixing unit 3cC, in the mixing range R at the sound pressure Va, the sound pressure Va5 that mixes the input sound signal SN2 and the input sound signal SN3 at an equal sound pressure when the sound pressure Va decreases over time is set closer to the mixing lower limit sound pressure Va3 than the midpoint between the mixing lower limit sound pressure Va3 and the mixing upper limit sound pressure Va4.

[0084] That is, the input mixer 3cC mixes the input audio signal SN2 and the input audio signal SN3 when the sound pressure Va decreases, based on the nonlinear characteristic lines LN2b and LN3b.

[0085] On the other hand, the input mixing unit 3cC has a sound pressure Va6 that mixes the input audio signals SN2 and SN3 at equal sound pressure when the sound pressure Va increases over time, and is set closer to the mixing upper limit sound pressure Va4 than the midpoint between the mixing lower limit sound pressure Va3 and the mixing upper limit sound pressure Va4.

[0086] That is, the input mixer 3cC mixes the input audio signal SN2 and the input audio signal SN3 when the sound pressure Va increases, based on the nonlinear characteristic lines LN2a and LN3a.

[0087] In addition, when the sound pressure Va increases but does not reach the upper mixed sound pressure Va4 and starts to decrease, the input mixing unit 3cC changes the mixing ratio on the characteristic lines LN2a and LN3a. In addition, when the sound pressure Va decreases but does not reach the lower mixed sound pressure Va3 and starts to increase, the input mixing unit 3cC changes the mixing ratio on the characteristic lines LN3b and LN2b.

[0088] Moreover, the input mixer 3cC controls the mixing ratio of the input audio signal SN2 and the input audio signal SN3 so that the total sound pressure Vt of the output audio signal SNt becomes a constant total sound pressure Vtc regardless of the magnitude of the sound pressure Va.

[0089] The nonlinear characteristics of the input audio signals SN2 and SN3 in FIG. 7 are preset by the manufacturer of the earphone 91 or are set by adjustment of the speaker H.

[0090] When the sound pressure Va of the ambient sound is maintained relatively small and is close to the lower limit sound pressure Va3 of the mixing range R, the earphone 91C of variant example 3 generates an output audio signal SNt by mixing so that the ratio of the input audio signal SN2 to the input audio signal SN3 is higher, thereby prioritizing clarity of the sound.

[0091] In addition, when the sound pressure Va of the ambient sound is maintained relatively high and is close to the upper mixed sound pressure Va4 of the mixing range R, the earphone 91C generates the output audio signal SNt by mixing so that the ratio of the input audio signal SN3 to the input audio signal SN2 is higher, thereby prioritizing the increase in sound pressure of the audio.

[0092] In this way, the earphone 91C of the third modification generates an output sound signal SNt that is more suitable for voice recognition in accordance with the magnitude tendency of the ambient sound pressure Va. This maintains a high recognition rate of the AI ​​assistant 81 for the voice uttered by the speaker H. (Variation 4) The earphone 91C may be modified to an earphone 91D (see FIG. 4) of Modification 4 in which the input mixer 3cC is replaced with an input mixer 3cD (see FIG. 4) that changes the total sound pressure Vt in response to the sound pressure Va, as shown in FIG.

[0093] For example, in a mixing range R for a sound pressure Va, the input mixer 3cD increases the total sound pressure Vt as the sound pressure Va increases.

[0094] In detail, the input mixer 3cD performs the mixing operation by setting the mixed maximum sound pressure V2max of the input audio signal SN2 and the mixed maximum sound pressure V3max of the input audio signal SN3 to different values ​​as shown in Fig. 8. For example, the mixed maximum sound pressure V3max is set to be larger than the mixed maximum sound pressure V2max.

[0095] As a result, the sound pressure of the output sound signal SNt increases and decreases between a total sound pressure Vt1 at the mixture lower limit sound pressure Va3 and a total sound pressure Vt2 that is greater than the total sound pressure Vt1 at the mixture upper limit sound pressure Va4.

[0096] As a result, as in Modification 2, the mixing ratio of the sound pressure of the input sound signal SN3 to the input sound signal SN2 increases as the sound pressure Va increases, which suppresses an increase in the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output sound signal SNt, and the voice recognition rate of the AI ​​assistant 81 that receives the output sound signal SNt is stably maintained.

[0097] When earphones 91, 91A to 91C are sold as products, they may be sold not only as single items but also as a set of two or more. If earphones 91, 91A to 91C are designed to be wearable on both the left and right ears, they may be sold as a set of two for the left and right ears.

[0098] Furthermore, the earphones 91, 91A to 91C may be sold as a set of any number of earphones with a microphone for one ear to be worn by multiple employees of a large store. Example 2 Second Embodiment A voice input system according to an embodiment of the present invention will be described using an earphone system 91ST according to a second embodiment with reference mainly to FIGS.

[0099] Fig. 9 is a block diagram of the earphone system 91ST, and Fig. 10 is a table showing the operation of the earphone system 91ST. As shown in Fig. 9, the earphone system 91ST is configured as a pair of an earphone 91L which is a first audio input device and an earphone 91R which is a second audio input device. The earphone 91L is worn on the left ear of a speaker H, and the earphone 91R is worn on the right ear of the speaker H.

[0100] As shown in FIG. 1, the earphone 91L has a main body 1L and an insertion portion 2, and the earphone 91R has a main body 1R and an insertion portion 2.

[0101] In the earphone 91L and the earphone 91R, the configurations and positions of the first to third microphones M1 to M3, the driver 5, and the speaker unit 6 are the same as those in the earphone 91 of the first embodiment.

[0102] Hereinafter, the same reference numerals will be used to designate the same members as those in the earphone 91, and different members will be distinguished by adding L and R to the end of the reference numerals, respectively.

[0103] As shown in FIGS. 1 and 9, the earphones 91L and 91R have control units 3L and 3R, respectively, instead of the control unit 3 of the earphone 91, and have communication units 4L and 4R, respectively, instead of the communication unit 4 of the earphone 91.

[0104] In the earphone 91L, the main body 1L has a first microphone M1, a second microphone M2, a control unit 3L, a communication unit 4L, a drive unit 5, and a speaker unit 6. The insertion portion 2 has a third microphone M3.

[0105] In the earphone 91R, the main body 1R has a first microphone M1, a second microphone M2, a control unit 3L, a communication unit 4L, a drive unit 5, and a speaker unit 6. The insertion portion 2 has a third microphone M3.

[0106] As shown in FIG. 1, the main body 1L and the main body 1R have an air chamber 1a on the sound output side of the speaker unit 6.

[0107] The insertion section 2 has a sound emission path 2a that opens at the tip and communicates with the air chamber 1a.

[0108] When the earphones 91L, 91R are in use, sound output from the speaker unit 6 by the operation of the drive unit 5 passes through the air chamber 1a and the sound output path 2a, and is output into the external ear canals E1 of the left and right ears, respectively.

[0109] As a result, the earphones 91L, 91R can receive audio signals wirelessly transmitted from an external audio playback device via the communication units 4L, 4R, respectively, and play the signals in the speaker unit 6 via the control units 3L, 3R and the drive unit 5.

[0110] The earphones 91L and 91R are capable of communicating with each other via the communication units 4L and 4R.

[0111] The first microphone M1 provided in each of the main bodies 1L and 1R is disposed at a portion of the main bodies 1L and 1R farther from the mouth of the speaker H, and picks up sounds around the main bodies 1L and 1R.

[0112] The second microphone M2 provided in each of the main bodies 1L and 1R is disposed at a portion of the main bodies 1L and 1R closer to the mouth of the speaker H. In other words, when the earphones 91L and 91R are in use, the second microphone M2 is located closer to the mouth of the speaker H than the first microphone M1.

[0113] The third microphone M3 is an air-conduction sound microphone and is arranged at a third position facing the sound emission path 2a of the insertion portion 2. The third microphone M3 picks up air-conduction sound generated when the sound emitted by the speaker H reaches the ear canal E1 as bone-conduction sound and reverberates in the ear canal E1 and the internal space Ev of the sound emission path 2a.

[0114] That is, the first position of the first microphone M1 is outside the ear canal of the speaker H. The second position of the second microphone M2 is outside the ear canal of the speaker H and closer to the mouth of the speaker H than the first position. The third microphone M3 is inside the ear canal of the speaker H.

[0115] As shown in FIG. 9, a control unit 3L of an earphone 91L has a sound pressure detection unit 3aL, an input selection unit 3bL, and a sound pressure difference evaluation unit 3d.

[0116] The control unit 3R of the earphone 91R has a sound pressure detection unit 3aR, an input selection unit 3bR, and an output control unit 3e.

[0117] In the earphone 91L, the sound pressure detection unit 3aL detects the sound pressure of the audio signal SN1L received from the first microphone M1, and outputs it as a detected first audio signal SNL to both the input selection unit 3bL and the sound pressure difference evaluation unit 3d.

[0118] In the earphone 91R, the sound pressure detector 3aR detects the sound pressure of the audio signal SN1R received from the first microphone M1, and outputs it as a detected first audio signal SNR to both the input selector 3bR and the output controller 3e.

[0119] The sound pressures of the audio signals SN1L and SN1R are detected, for example, as an equivalent noise level (LAeq). The sound pressures of the detected first audio signals SNL and SNR detected by the sound pressure detectors 3aL and 3aR as the equivalent noise level (LAeq) are hereinafter referred to as sound pressures VL and VR, respectively.

[0120] The first microphone M1 provided in the main body 1L picks up ambient sound around the main body 1L, and the first microphone M1 provided in the main body 1R picks up ambient sound around the main body 1R.

[0121] Therefore, the sound pressure VL can be regarded as the sound pressure of the surrounding sound for the left earphone 91L, and the sound pressure VR can be regarded as the sound pressure of the surrounding sound for the right earphone 91R.

[0122] The output control unit 3e of the earphone 91R sends sound pressure information JR1 including the sound pressure VR of the received detected first sound signal SNR, and communication control information JR2 (details of which will be described later), to the communication unit 4R.

[0123] The input sound processing method by the operation of the input selection units 3bL and 3bR of the earphones 91L and 91R is similar to that of the input selection unit 3b of the first embodiment.

[0124] 3, the input selection unit 3bL of the earphone 91L selects and sets the input audio signal SN2L as the output audio signal SNtL when the sound pressure VL of the detected first audio signal SNL output from the sound pressure detection unit 3aL is less than at least a preset lower switching sound pressure Va1, and selects and sets the input audio signal SN3L as the output audio signal SNtL when the sound pressure VR exceeds at least the upper switching sound pressure Va2.

[0125] The input selection unit 3bL outputs the output audio signal SNtL set as described above to the communication unit 4L.

[0126] In this way, the input selection unit 3bL sets the reflection degrees RF1L and RF2L of the input sound signals SN2L and SN3L in the output sound signal SNtL, respectively, based on the sound pressure Va of the detected first sound signal SN1a.

[0127] In this example, the input selection unit 3bL sets the reflection degree RF1L and the reflection degree RF2L as an alternative between one being reflected and the other not being reflected.

[0128] Moreover, the input selection unit 3bR of the earphone 91R selects and sets the input audio signal SN2R as the output audio signal SNtR when the sound pressure VR of the detected first audio signal SNR output from the sound pressure detection unit 3aR is less than at least the preset lower switching sound pressure Va1, and selects and sets the input audio signal SN3R as the output audio signal SNtR when the sound pressure VR exceeds at least the upper switching sound pressure Va2.

[0129] The input selection unit 3bR outputs the output audio signal SNtR set as described above to the communication unit 4R.

[0130] In this way, the input selection unit 3bR sets the reflection degrees RF1R and RF2R of the input sound signals SN2R and SN3R in the output sound signal SNtR, respectively, based on the sound pressure Va of the detected first sound signal SN1a.

[0131] In this example, the input selection unit 3bR sets the reflection degree RF1R and the reflection degree RF2R as an alternative between one being reflected and the other not being reflected.

[0132] The communication unit 4R wirelessly transmits the sound pressure information JR1 received from the output control unit 3e to the outside of the earphone 91R. The wireless transmission method is, for example, Bluetooth (registered trademark).

[0133] Here, whether or not the communication unit 4R wirelessly transmits the output audio signal SNtR output from the input selection unit 3bR is controlled by communication control information JR2 input from the output control unit 3e.

[0134] That is, the communication control information JR2 includes a command to either permit or prohibit wireless transmission of the output audio signal SNtR, and the wireless transmission of the output audio signal SNtR in the communication unit 4R is controlled based on this command.

[0135] The communication unit 4L of the earphone 91L receives the sound pressure information JR1 wirelessly transmitted from the communication unit 4R of the earphone 91R, and sends it to the sound pressure difference evaluation unit 3d.

[0136] The sound pressure difference evaluation unit 3d acquires the sound pressure VR from the sound pressure information JR1 sent from the communication unit 4L, and compares the sound pressure VR with the sound pressure VL of the detected first sound signal SNL acquired from the sound pressure detection unit 3aL.

[0137] The sound pressure difference evaluation unit 3d sets one of the output sound signal SNtL and the output sound signal SNtR, which are preset in accordance with the magnitude relationship between the sound pressures VL and VR, as the output sound signal SNst to be wirelessly transmitted to the outside as the earphone system 91ST.

[0138] Next, the sound pressure difference evaluation unit 3d outputs communication control information JL2 that identifies the signal set in the output sound signal SNtL to the communication unit 4L, and the communication unit 4L wirelessly transmits the communication control information JL2 to the communication unit 4R of the earphone 91R.

[0139] The communication unit 4R receives the communication control information JL2 and sends the received communication control information JL2 to the output control unit 3e.

[0140] The operation of the sound pressure difference evaluation unit 3d will be described in detail with reference to FIG.

[0141] FIG. 10 is a table showing the relationship between the magnitude of the sound pressure VL and the sound pressure VR and the output audio signal SNst wirelessly transmitted to the outside as the earphone system 91ST.

[0142] As shown in FIG. 10, when the sound pressure difference evaluation unit 3d determines that the sound pressure VR is greater than the sound pressure VL, it sets the output sound signal SNtL to the output sound signal SNst to be wirelessly transmitted as the earphone system 91ST.

[0143] Moreover, the sound pressure difference evaluation unit 3d sends the communication control information JL2 to the communication unit 4L, including an instruction to the communication unit 4L to wirelessly transmit the output sound signal SNtL.

[0144] When the sound pressure difference evaluation unit 3d determines that the sound pressure VR is smaller than the sound pressure VL, it sends the communication control information JL2 to the communication unit 4L including an instruction to the communication unit 4L to stop wireless transmission of the output sound signal SNtL.

[0145] The communication unit 4L transmits the communication control information JL2 to the communication unit 4R, and executes or stops wireless transmission of the output audio signal SNtL based on an instruction for the communication unit 4L contained in the communication control information JL2.

[0146] On the other hand, the communication section 4R receives the communication control information JL2 transmitted from the communication section 4L and sends it to the output control section 3e.

[0147] When the output control unit 3e causes the communication unit 4L to wirelessly transmit the output audio signal SNtL based on the communication control information JL2, the output control unit 3e generates communication control information JR2 including an instruction to stop wireless transmission of the output audio signal SNtR in the communication unit 4R.

[0148] Furthermore, when the output control unit 3e does not want the communication unit 4L to wirelessly transmit the output audio signal SNtL, the output control unit 3e generates communication control information JR2 including an instruction to wirelessly transmit the output audio signal SNtR in the communication unit 4R.

[0149] The output control unit 3e sends the generated communication control information JR2 to the communication unit 4R.

[0150] The communication unit 4R executes or stops wireless transmission of the output audio signal SNtR based on the communication control information JR2 sent from the output control unit 3e.

[0151] As is apparent from the above-described audio input device and input audio processing method, the earphone system 91ST selects an output audio signal from the earphone 91L, 91R, whichever has the smaller ambient sound, and wirelessly transmits the selected output audio signal to the outside.

[0152] As a result, the earphone system 91ST improves the recognition rate of the voice uttered by the speaker H in the AI ​​assistant 81.

[0153] The second embodiment described above in detail is not limited to the above-mentioned configuration and procedure, and may be modified without departing from the gist of the present invention.

[0154] As described in the first modification of the first embodiment, the earphones 91L and 91R may have, instead of the input selection units 3bL and 3bR, input mixing units 3cL and 3cR (see FIG. 9) that perform the same operation as the input mixing unit 3c.

[0155] For example, in the input mixing unit 3cL, the input audio signal SN2L and the input audio signal SN3 are mixed at a sound pressure ratio of the reflection degrees FR2L and FR3L, respectively, which correspond to the sound pressure Va, and the ratio of the mixed sound pressures gradually changes linearly in response to the increase or decrease in the sound pressure of the ambient sound of the main body unit 1.

[0156] The reflection degree RF1L and the reflection degree RF2L are indexes indicating the degree of reflection of the input audio signal SN2L and the input audio signal SN3L to the output audio signal SNtL, respectively. The index is, for example, the magnitude of sound pressure. Therefore, the sound pressure ratio is the ratio of the sound pressure of the input audio signal SN2L and the input audio signal SN3L included in the output audio signal SNtL.

[0157] For example, as shown in FIG. 5, the reflection degree FR2L in the output sound signal SNtL is represented by Vmax / Vmin when the sound pressure Va is Va3, V2x / V3x when the sound pressure Vax, and Vmin / Vmax when the sound pressure Va4.

[0158] The reflection degree FR3 in the output audio signal SNtL is expressed as Vmin / Vmax when the sound pressure Va is the sound pressure Va3, as V3x / V2x when the sound pressure Vax, and as Vmax / Vmin when the sound pressure Va4.

[0159] When the earphones 91L and 91R have the input mixers 3cL and 3cR instead of the input selectors 3bL and 3bR, respectively, the sound quality of the output audio signal SNst changes gradually and smoothly in response to an increase or decrease in the ambient sound.

[0160] This allows the recognition rate of the voice uttered by the speaker H by the AI ​​assistant 81 to be maintained high regardless of the sound pressure of the surrounding sound of the main body 1L or the main body 1R.

[0161] Furthermore, because the earphone 91L and the earphone 91R have the input mixing units 3cL and 3cR, respectively, the total sound pressure of the output audio signal SNst remains constant and does not change suddenly regardless of an increase or decrease in the ambient sound, so that the recognition rate of the voice uttered by the speaker H by the AI ​​assistant 81 is maintained at a higher level.

[0162] In the case where the earphone 91L and the earphone 91R have the input mixing unit 3cL and the input mixing unit 3cR, respectively, the earphone system 91ST can apply the second to fourth modifications of the first embodiment to the extent possible.

[0163] The wireless communication method of the communication units 4, 4L, and 4R is not limited to the above-mentioned Bluetooth (registered trademark), and various methods can be applied. Furthermore, the communication units 4, 4L, and 4R are not limited to wireless communication with the outside, and may be wired.

[0164] In the earphones 91, 91A to 91D, 91L, and 91R that are the voice input devices of the first and second embodiments, the third microphone M3 is not limited to the air conduction sound microphone described above, and may be a bone conduction microphone that picks up bone conduction sound.

[0165] Fig. 11 is a diagram illustrating an example of the arrangement position of the third microphone M3 when it is a bone conduction microphone. As shown in Fig. 11, the third microphone M3 is a bone conduction microphone, and when the insertion portion 2 is inserted into the ear canal E1, it is located at a third position where it is in close contact with the inner surface of the ear canal E1, and picks up the bone conduction sound of the voice uttered by the speaker H.

[0166] In the voice input system 91ST of the second embodiment, the use of the earphone 91L as the first voice input device and the earphone 91R as the second voice input device is not limited to a use mode in which the earphones are worn on one ear and the other ear of the speaker H. For example, the earphone 91L may be worn on the ear of a first speaker, and the earphone 91R may be worn on the ear of a second speaker different from the first speaker.

[0167] The index of the reflection degrees RF1L, RF2L, RF1, and RF2 is not limited to sound pressure, but may be a physical quantity related to sound quality or the like. [Explanation of symbols]

[0168] 1 Main unit 1a Air chamber 2 Insertion section 2a Sound emission path 3,3L,3R control section 3a, 3aL, 3aR Sound pressure detection section 3b, 3bL, 3bR Input selection section 3c, 3cB, 3cC, 3cD Input Mixing Section 3d Sound pressure difference evaluation section 3e Output control section 4,4L,4R Communication Department 5 Drive unit 6 Speaker unit 81 AI Assistant 91, 91A, 91B, 91C, 91D, 91L, 91R Earphones (audio input devices) 91ST Earphone System (Audio Input System) E. Auricle E1 ear canal Ev Inner Space FR2,FR3 reflection degree H Speaker JR1 Sound pressure information JL2, JR2 communication control information M1 First Microphone M2 Second Microphone M3 3rd microphone LN2b,LN3b,LN2a,LN3a characteristic line R Mixed range SN1~SN3,SN1L~SN3L,SN1R~SN3R Input audio signal SN1a, SNL, SNR Detected first audio signal SNt, SNtL, SNtR, SNst output audio signals V, V2x, V3x mixed sound pressure Va, Vax, Va5, Va6 sound pressure Va1 switching sound pressure Va2 Sound pressure above switching Va3 Mixed lower limit sound pressure Va4 Mixed upper limit sound pressure Vmax Maximum mixed sound pressure Vmin Mixed minimum sound pressure VL, VR Sound pressure Vt, Vt1, Vt2, Vtc Total sound pressure V2max, V3max mixed maximum sound pressure

Claims

1. a first microphone that picks up a voice at a first position outside the ear canal of a speaker and outputs a first input voice signal; a second microphone that picks up a voice at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position and outputs a second input voice signal; a third microphone that picks up a voice in the ear canal of the speaker and outputs a third input voice signal; a control unit that detects a sound pressure of the first input audio signal, mixes the second input audio signal and the third input audio signal at a sound pressure ratio such that a ratio of the sound pressure of the third input audio signal to the sound pressure of the second input audio signal increases as the sound pressure of the first input audio signal increases, and generates an output audio signal including at least one of the second input audio signal and the third input audio signal; A voice input device comprising:

2. acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of a speaker, and detecting a sound pressure of the first input audio signal; acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position; Acquiring a sound picked up in the ear canal of the speaker as a third input sound signal; The second input audio signal and the third input audio signal, mixing the first input audio signal and the second input audio signal so that a ratio of the sound pressure of the third input audio signal to the sound pressure of the second input audio signal increases as the sound pressure of the detected first input audio signal increases, and generating an output audio signal including at least one of the second input audio signal and the third input audio signal. Input audio processing method.

Citation Information

Patent Citations

  • Microphone equipment

    JP1990114708A

  • Handset

    JP2000261534A

  • Voice processing apparatus and voice processing method

    JP2003264883A

  • Combined microphone and earphone audio headset having means for denoising near speech signal, in particular for "hands-free" telephony system

    JP2012231468A

  • Earhole attachment-type sound pickup device, signal processing device, and sound pickup method

    JP2013121106A