Microphone device and communication device equipped with microphone device

Abstract

To provide a microphone device for generating sounds based on air conduction sounds and bone conduction sounds and a communication device including the microphone device capable of reliably achieving microphone pickup even when any abnormality occurs in either a bone conduction microphone or an air conduction microphone.SOLUTION: A microphone device 1 includes: a bone conduction microphone 11 for recording bone conduction sounds and bone-conducted ambient sounds of a user; an air conduction microphone 10 for recording air conduction sounds and air-conducted ambient sounds of the user; and a signal processing unit 4 that generates signals to be output based on recording result by the bone conduction microphone and the air conduction microphone. The signal processing unit is configured so as to acquire the level of the bone conduction sound and the level of the air conduction sound, and when the difference between the bone conduction sound level and the air conduction sound level is equal to or greater than a predetermined difference threshold, determine a generation method of the signal to be output based on either the recording results of the bone conduction microphone or the air conduction microphone.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a microphone device equipped with an air conduction microphone and a bone conduction microphone, and a communication device equipped with the microphone device. [Background technology]

[0002] There has been active research into technologies for effectively transmitting a user's voice to others and enabling voice recognition even in a noisy environment around the user. In response to this, a signal processing device that uses both a bone conduction microphone and an air conduction microphone has been proposed in recent years.

[0003] A known signal processing device having such a configuration includes an air conduction microphone, a bone conduction microphone, an output adjustment means for adjusting the sound pressure ratio of the sound signal output from the air conduction microphone and the sound signal output from the bone conduction microphone based on the intensity of an external noise signal (so-called ambient sound level) mixed into the sound signal output from the air conduction microphone, and outputting the adjusted sound pressure ratio and an addition means for adding the two sound signals outputted from the air conduction microphone and outputting the added signal to a voice recognition device (see Patent Document 1).

[0004] Furthermore, a signal processing device having such a configuration is known which has two sound collection sections, an air conduction microphone and a bone conduction microphone, a frequency component extraction means for extracting the low and high frequency components of the sound collection signal of the air conduction microphone and the low frequency component of the sound collection signal of the bone conduction microphone, an ambient sound level estimation means for estimating the ambient sound level when the sound collection sections are in use, and a synthesis means for changing the mixing ratio of the low and high frequency components of the sound collection signal of the air conduction microphone and the low frequency component of the sound collection signal of the bone conduction microphone based on the estimation result of the ambient sound level estimation means, and synthesizing the low and high frequency components of the sound collection signal of the air conduction microphone and the low frequency component of the sound collection signal of the bone conduction microphone as a transmission signal (see Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2003-264883 A [Patent Document 2] Patent No. 3095214 Summary of the Invention [Problem to be solved by the invention]

[0006] As described in Patent Documents 1 and 2, a voice processing device using an air conduction microphone and a bone conduction microphone is known. Hereinafter, the voice of the user picked up by the bone conduction microphone is referred to as bone conduction voice, and the ambient sound picked up by the bone conduction microphone is referred to as bone conduction ambient sound. In addition, the voice of the user and the ambient sound are collectively referred to as a sound signal. That is, the sound signal may include voice (also referred to as the user's vocalization) and ambient sound. Furthermore, here, as necessary, the air conduction voice and the air conduction ambient sound are collectively referred to as air conduction sound, and the bone conduction voice and the bone conduction ambient sound are collectively referred to as bone conduction sound.

[0007] Although the sound signal picked up by an air conduction microphone is clear up to the high frequencies, it is easily affected by surrounding sounds because it propagates through the air. Therefore, in order to achieve a good signal-to-noise ratio (also called SNR) based on the user's voice and surrounding sounds, the user's speaking volume must be above a certain level. On the other hand, the sound signal picked up by a bone conduction microphone is unclear, but it is less affected by surrounding sounds because it does not propagate through the air. Therefore, a relatively good SNR can be ensured regardless of whether the user's speaking volume is loud or quiet.

[0008] In other words, when the user speaks very quietly (i.e., when speaking softly), the sound signal picked up by the air conduction microphone is affected by the surrounding noise because the user's voice (signal) is small, and the SNR decreases, resulting in unclear sound quality. On the other hand, the bone conduction microphone is less affected by the surrounding noise, so even when speaking softly, the SNR is less likely to decrease, resulting in clearer sound quality than the air conduction microphone.

[0009] In the technology of Patent Document 1, air conduction sound and bone conduction sound are synthesized based on the ambient sound level so that the sound pressure ratio between the audio signal output from the air conduction microphone and the audio signal output from the bone conduction microphone is, for example, a value between 4:6 and 2:8.

[0010] In the technology of Patent Document 1, the synthesis ratio is determined based on the ambient sound level, regardless of the volume of the user's voice, so when the user speaks softly and the ambient sound is loud, the air conduction microphone produces the aforementioned unclear sound quality. In other words, under conditions where the user speaks softly and the ambient sound is loud, the air conduction microphone has inferior sound quality compared to the bone conduction microphone, and is therefore not suitable for synthesizing air conduction sound and bone conduction sound. In other words, under the above conditions, it is desirable to actively use the bone conduction microphone.

[0011] Similarly, when the user's voice volume is high (i.e., when the user speaks loudly) and the ambient sound is below a certain level, the air conduction microphone provides the above-mentioned clear sound quality. In other words, under conditions where the user speaks loudly and the ambient sound is below a certain level, the bone conduction microphone has inferior sound quality compared to the air conduction microphone and is therefore not suitable for combining air conduction sound and bone conduction sound. In other words, under the above-mentioned conditions, it is desirable to actively use the air conduction microphone.

[0012] As described above, the sound pressure ratio is set based on the ambient sound level in the technology of Patent Document 1. Therefore, there is a problem that the sound quality may differ from the expected value depending on the voice volume of the user.

[0013] In addition, in the technology of Patent Document 2, as in Patent Document 1, the mixture ratio is set based on the ambient noise level. Therefore, the mixture ratio is not set according to the voice volume of the user, but an inappropriate mixture ratio depending on the frequency is set, which may cause, for example, the balance of voice formants to be lost. Therefore, even with the technology of Patent Document 2, it is possible that the sound quality may differ from the expected value depending on the voice volume of the user.

[0014] Furthermore, the techniques of Patent Documents 1 and 2 do not take into consideration cases where the bone conduction microphone is not in contact with the user or where the bone conduction microphone and air conduction microphone are not picking up sound normally due to reasons such as a malfunction of the air conduction microphone.

[0015] Therefore, the main objective of the present disclosure is to provide a microphone device that generates sound based on air conduction sound and bone conduction sound, and a communication device equipped with the microphone device, which achieves good microphone sound quality even when there is an abnormality in the sound pickup of either the bone conduction microphone or the air conduction microphone. [Means for solving the problem]

[0016] The microphone device of the present disclosure is a microphone device that collects sound signals including a user's voice and ambient sound, and generates a signal to be output, and includes a bone conduction microphone that collects the user's bone-conducted voice and bone-conducted ambient sound, an air conduction microphone that collects the user's air-conducted voice and air-conducted ambient sound, and a signal processing device that generates a signal to be output based on the sound collection results by the bone conduction microphone and the air conduction microphone, and the signal processing device acquires a level of the bone conduction voice and a level of the air conduction voice, and when a level difference between the level of the bone conduction voice and the level of the air conduction voice is equal to or greater than a predetermined difference threshold, A method for generating a signal to be output based on the sound pickup result of the air conduction microphone is determined. The composition is as follows.

[0017] A communication device according to the present disclosure includes the microphone device described above. Effect of the Invention

[0018] According to the present disclosure, in a microphone device that generates sound based on air conduction sound and bone conduction sound, and in a communication device equipped with the microphone device, good microphone sound quality can be achieved even when there is an abnormality in the sound collection of either the bone conduction microphone or the air conduction microphone, or when there is an abnormality in the sound collection of either the bone conduction microphone or the air conduction microphone. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a perspective view showing a state in which a communication device provided with a microphone device according to an embodiment is used; [Diagram 2] Exploded view of earphones [Diagram 3] Functional block diagram of sound processing device [Figure 4] A table showing examples of combinations of microphone types and filter processing [Diagram 5] Flowchart of abnormality detection process [Figure 6] Flowchart of output setting process [Figure 7] A table for explaining the determination results of the sound section and the soundless section and the signal-to-noise ratio when the ambient sound is different in Pattern 1. [Figure 8] 1 is a table for explaining the determination results of the sound section and the soundless section and the signal-to-noise ratio when the ambient sound is different in pattern 2. [Figure 9] 1 is a table for explaining the determination results of the sound section and the soundless section and the signal-to-noise ratio when the ambient sound is different in pattern 3. [Figure 10] 1 is a table for explaining the determination results of the sound section and the soundless section and the signal-to-noise ratio when the ambient sound is different in pattern 4. [Figure 11] (A) A map showing the selection ranges of an air conduction microphone and a bone conduction microphone according to an embodiment, (B) a first modified example, (C) a second modified example, and (D) a third modified example of the map. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] A first invention made to solve the above problem is a microphone device that picks up a sound signal including a user's voice and ambient sound and generates a signal to be output, the microphone device comprising: a bone conduction microphone that picks up the user's bone-conducted voice and bone-conducted ambient sound; an air conduction microphone that picks up the user's air-conducted voice and air-conducted ambient sound; and a signal processing device that generates a signal to be output based on the sound collection results by the bone conduction microphone and the air conduction microphone. The signal processing device acquires a level of the bone conduction voice and a level of the air conduction voice, and when a level difference between the level of the bone conduction voice and the level of the air conduction voice is equal to or greater than a predetermined difference threshold, A method for generating a signal to be output based on the sound pickup result of the air conduction microphone is determined. The composition is as follows.

[0021] This prevents the generation of signals to be output based on both the bone conduction sound and the air conduction sound when there is an abnormality in the sound pickup by either the bone conduction microphone or the air conduction microphone, thereby realizing reliable microphone sound pickup even when there is an abnormality in the sound pickup by either the bone conduction microphone or the air conduction microphone. Furthermore, when there is an abnormality in the sound pickup by the bone-conduction microphone, the method of generating the signal to be output can be determined without relying on the bone-conduction microphone.

[0022] The second invention is, A microphone device that collects sound signals including a user's voice and ambient sounds and generates a signal to be output, the microphone device comprising: a bone conduction microphone that collects the user's bone-conducted voice and bone-conducted ambient sounds; an air conduction microphone that collects the user's air-conducted voice and air-conducted ambient sounds; and a signal processing device that generates a signal to be output based on the sound collection results by the bone conduction microphone and the air conduction microphone, the signal processing device acquiring a level of the bone conduction voice and a level of the air conduction voice, and determining a method of generating a signal to be output based on the sound collection results by the air conduction microphone after noise cancellation when a level difference between the level of the bone conduction voice and the level of the air conduction voice is equal to or greater than a predetermined difference threshold. The composition is as follows.

[0023] According to this, When there is an abnormality in the sound pickup by either the bone conduction microphone or the air conduction microphone, it is possible to prevent the generation of a signal to be output based on both the bone conduction sound and the air conduction sound. This makes it possible to realize reliable microphone sound pickup even when there is an abnormality in the sound pickup by either the bone conduction microphone or the air conduction microphone. In addition, when there is an abnormality in the sound pickup by the bone conduction microphone, a method for generating a signal to be output is determined so as to reduce the influence of noise. It is possible.

[0028] Also, 3 In the invention, the signal processing device is configured to change the frequency characteristics and determine a method for generating a signal to be output when the level difference between the bone-conducted sound level and the air-conducted sound level is less than the difference threshold.

[0029] This can increase the clarity of the signal produced.

[0030] Also, 4 The present invention relates to a communication device comprising the microphone device described above.

[0031] This makes it possible to provide a communication device that can reliably pick up sound from the microphone even when there is an abnormality in the sound pickup by either the bone conduction microphone or the air conduction microphone.

[0032] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0033] The microphone device 1 constitutes a part of a communication device 2 for communicating between users. Fig. 1 shows a perspective view illustrating a state in which the communication device 2 is in use. Fig. 2 is an exploded perspective view of the communication device 2.

[0034] 1, the communication device 2 is a device for converting sound into an electric signal, transmitting the electric signal to a remote location, and outputting the signal at the remote location. The communication device 2 includes at least one earphone 3 with a microphone and a signal processing device 4, and the microphone device 1 is composed of the earphone 3 and the signal processing device 4.

[0035] The earphone 3 is inserted into the user's ear when used. In FIG. 1, an example is shown in which the earphone 3 is inserted into the user's right ear when used, but the earphone 3 may be inserted into the left ear when used. In this case, the earphone 3 appears symmetrical to that shown in the figure. The communication device 2 may also include a pair of earphones 3, one on the left and one on the right.

[0036] 1 shows an example in which the earphones 3 and the signal processing device 4 are connected by a signal line 5 and can communicate with each other. However, the earphones 3 and the signal processing device 4 may be connected to each other so as to be able to communicate with each other via wireless signals based on Wifi (registered trademark) or Bluetooth (registered trademark). However, the earphones 3 and the signal processing device 4 may be integrated into the housing of the earphones 3.

[0037] As shown in FIG. 2, the earphone 3 includes an exterior housing 9, an air conduction microphone 10 and a bone conduction microphone 11 housed in the exterior housing 9, an air conduction receiver 12, an internal housing 13, and a microphone rubber 15.

[0038] The exterior housing 9 constitutes the outer surface of the earphone 3. The exterior housing 9 includes a case 9A that constitutes the inner part (deeper part) of the user's ear, and a cover 9B that constitutes the outer part of the user's ear. The exterior housing 9 is formed by overlapping the cover 9B on the case 9A from the outside.

[0039] The air conduction microphone 10 and the bone conduction microphone 11 are each housed inside the exterior housing 9. The air conduction microphone 10 and the bone conduction microphone 11 each include an element (e.g., a vibration detection element) for converting sound vibration into an electrical signal. The air conduction microphone 10 and the bone conduction microphone 11 each output the converted electrical signal to the signal processing device 4.

[0040] The air conduction receiver 12 outputs the output signal from the signal processing device 4 as audio (output sound). The air conduction receiver 12 may be configured with a known speaker. For example, when the signal processing device 4 outputs a signal corresponding to the voice of the other party of the user's call, the air conduction receiver 12 outputs the sound corresponding to the voice of the other party. Note that the sound output by the air conduction receiver 12 is not limited to the voice of the other party of the call, and may include, for example, ambient sounds and on-hold sounds.

[0041] The internal housing 13 is a member for guiding the reproduced sound of the air conduction receiver 12 to the ear canal, and is provided with a passage 13A for guiding the reproduced sound of the air conduction receiver 12 to the ear canal. The air conduction receiver 12 is disposed on one end side of the passage 13A, and the air conduction receiver 12 is housed in the exterior housing 9 together with a part of the internal housing 13 that constitutes the one end side of the passage 13A.

[0042] At the other end of the passage 13A, a cylindrical tube portion 13B is provided that extends to the outside of the exterior housing 9 through an opening 9C provided in the exterior housing 9 (specifically, the case 9A). In this embodiment, a tip rubber 16 that abuts against the user's ear canal over its entire circumference is provided at the extending end of the tube portion 13B. By having the tip rubber 16 abut against the ear canal over its entire circumference, airtightness of the ear canal is ensured, and the reproduced sound from the air conduction receiver 12 is effectively transmitted to the user.

[0043] The microphone rubber 15 is also housed in the exterior housing 9 and elastically holds the bone-conduction microphone 11. The microphone rubber 15 connects the part where vocal cord vibration is transmitted via the bone near the ear (vocal cord vibration transmission part) to the bone-conduction microphone 11. As a result, the vocal cord vibration transmitted to the vocal cord vibration transmission part is transmitted to the bone-conduction microphone 11, and the bone-conduction microphone 11 picks up sound by detecting the vocal cord vibration propagating to the bones and skin around the ear via the microphone rubber 15.

[0044] On the other hand, the air conduction microphone 10 is disposed between the exterior housing 9 and the air conduction receiver 12. In order to prevent sound vibrations generated by the air conduction receiver 12 from being transmitted to the air conduction microphone 10, a resin member 17 is provided between the air conduction microphone 10 and the air conduction receiver 12. The air conduction microphone 10 mainly transmits the user's voice through the air and picks it up at the air conduction microphone 10 near the ear.

[0045] In this way, the air conduction microphone 10 is a device that is placed at the user's ear so as to mainly pick up acoustic waves in the air, and picks up the user's voice and ambient sounds generated around the user.

[0046] The bone conduction microphone 11 is a device that is attached to the user's ear so as to pick up vocal cord vibrations that are transmitted mainly to the vocal cord vibration transmission area, and picks up the user's speech and ambient sounds generated around the user.

[0047] The air conduction microphone 10 and the bone conduction microphone 11 are each connected to the signal processing device 4 via a different signal line 5. The air conduction microphone 10 outputs the picked up sound as an electric signal to the signal processing device 4. The bone conduction microphone 11 also outputs the picked up sound signal as an electric signal to the signal processing device 4. The exterior housing 9 is provided with a cable cap 18 for leading the signal line 5 to the outside.

[0048] The bone-conduction microphone 11 picks up vocal cord vibrations propagating inside the user's body, so it is less susceptible to the effects of the user's surroundings. On the other hand, the components in the high-frequency range (treble range) are attenuated, resulting in unclear sound quality.

[0049] On the other hand, the air conduction microphone 10 mainly picks up acoustic waves that propagate through the air, so it is easily affected by the user's surroundings. On the other hand, high-frequency components are less likely to attenuate, so the sound quality is clearer than that of the bone conduction microphone 11.

[0050] Based on the characteristics of the air conduction microphone 10 and the bone conduction microphone 11, the signal processing device 4 generates a signal to be output by the communication device 2 of the other party so that only the user's voice can be delivered to the other party while suppressing the influence of ambient sound and increasing clarity. The signal generated by the signal processing device 4 is output, for example, from an air conduction receiver 12 or various speakers built into the earphone 3 used by the other party.

[0051] The signal processing device 4 is configured by a processor such as a CPU, a memory such as a RAM or ROM, a storage such as an SSD or HDD, and a terminal (smartphone) equipped with an audio input / output terminal. However, the configuration of the signal processing device 4 is not limited to this embodiment, and may be configured by, for example, a tablet or various computers equipped with an audio input / output terminal. Furthermore, the signal processing device 4 may also be realized by a combination of a logic device and various analog devices.

[0052] As shown in Figure 3, the signal processing device 4 includes an amplifier unit 31, a pre-processing unit 32 and a noise cancellation unit 33 which process the outputs of the air conduction microphone 10 and the bone conduction microphone 11, a mixing unit 34 (mixer) which combines the outputs of the air conduction microphone 10 and the bone conduction microphone 11, and a signal setting unit 35 which sets the signal to be output based on the outputs of the bone conduction microphone 11 and the air conduction microphone 10.

[0053] In this embodiment, the signal processing device 4 is equipped with an AD converter (not shown) that performs AD conversion to convert the output of the air conduction microphone 10 and the output of the bone conduction microphone 11 into digital signals, and a microphone output unit 37 that transmits the output of the mixing unit 34 to the external signal processing device 4.

[0054] The amplifier 31 amplifies the output of a corresponding microphone (the bone conduction microphone 11 or the air conduction microphone 10). In this embodiment, the amplifier 31 amplifies the digital signal converted by the AD converter by performing arithmetic processing. Alternatively, the amplifier 31 may be configured by a known electric circuit (analog circuit) including an operational amplifier or the like.

[0055] The pre-processing units 32 each perform various processes such as noise removal and smoothing on the signal output from the amplification unit 31. In this embodiment, the pre-processing units 32 execute filtering processes such as a high-pass filter (HPF) and a low-pass filter (LPF) on the signal output from the amplification unit 31. The pre-processing units 32 may also be configured with a known electric circuit using a capacitor, resistor, coil, etc.

[0056] The noise canceling units 33 each remove noise other than the signal due to the user's voice by noise cancellation based on various known techniques from the output of the pre-processing units 32.

[0057] Alternatively, the noise cancellation performed by the noise cancellation unit 33 may be based on a spectrum subtraction method (SS method) that subtracts the power spectrum of noise estimated by another method from the power spectrum of the signal output from the preprocessing unit 32 and performs an inverse Fourier transform on the power spectrum after subtraction to restore the signal from which the noise has been removed. Alternatively, the noise removal performed by the noise cancellation unit 33 may be based on a deep neural network method (DNN method) that uses a deep learning network.

[0058] The signal setting section 35 controls whether or not the noise canceling section 33 performs noise cancellation and the method of performing the noise cancellation.

[0059] The mixer 34 mixes the output of the air conduction microphone 10 and the output of the bone conduction microphone 11 at a predetermined mixing ratio (also called a mixing ratio) by a known method and outputs the result. The mixer 34 can mix the output of the air conduction microphone 10 and the output of the bone conduction microphone 11 at different mixing ratios for each frequency band. The mixing ratio (including the mixing ratio for each frequency band) used by the mixer 34 when performing mixing is set by the signal setting unit 35.

[0060] The signal setting unit 35 includes a mounting state detection unit 41 that detects the mounting state of the microphone, a sound collection state detection unit 42 that detects the sound collection state by the microphone, and a path setting unit 43 that sets the path through which the output of the air conduction microphone 10 and the output signal of the bone conduction microphone 11 should pass.

[0061] In order to determine the wearing state of the microphones, the wearing state detection unit 41 implements an abnormality determination method for determining whether the air conduction microphone 10 and the bone conduction microphone 11 are each normally detecting sound. In order to implement the abnormality determination method, the wearing state detection unit 41 executes a predetermined program to perform an abnormality detection process for determining whether the bone conduction microphone 11 and the air conduction microphone 10 are each normally detecting sound.

[0062] In the abnormality detection process, the wearing state detection unit 41 judges whether the bone conduction sound level and the air conduction sound level are equal to or higher than the corresponding thresholds and whether there is a correlation between the bone conduction sound level and the air conduction sound level. The wearing state detection unit 41 judges that the headphones are normally worn when the bone conduction sound level and the air conduction sound level are equal to or higher than the corresponding thresholds and the difference between the two is equal to or lower than a predetermined difference threshold.

[0063] The level here refers to the strength of the sound signal picked up by the microphone, for example, the magnitude of sound pressure. The level may be expressed in decibels using the common logarithm of the ratio of the magnitude of sound pressure to a reference value.

[0064] The wearing state detection unit 41 obtains the level of the bone conduction sound and the level of the air conduction sound based on the sound collection result by the air conduction microphone 10 and the sound collection result by the bone conduction microphone 11 when the user is speaking, for example, and calculates the difference between the two. After that, when the difference is equal to or greater than a predetermined threshold (difference threshold), the wearing state detection unit 41 determines that there is an abnormality in the sound collection by either the air conduction microphone 10 or the bone conduction microphone 11. When the wearing state detection unit 41 determines that there is an abnormality in the sound collection by either one of the microphones, it sets the microphone to output a sound signal based on the sound collected by the other microphone that has no abnormality. In addition, the wearing state detection unit 41 sets the noise cancellation of the microphone that has no abnormality to ON.

[0065] For example, the wearing state detection unit 41 may determine that there is an abnormality in the sound collection by the bone-conduction microphone 11 when the level of the bone-conduction sound is lower than the level of the air-conduction sound collected at the same timing and the difference between the two sound levels is equal to or greater than a difference threshold.

[0066] Since the bone-conduction microphone 11 collects sound through the vocal cord vibration transmission site, it is often dependent on the state of wearing by the user. Therefore, when the difference between the level of the bone-conduction sound and the level of the air-conduction sound collected at the same timing is equal to or greater than a difference threshold, the wearing state detection unit 41 determines that there is an abnormality in the sound collection by the bone-conduction microphone 11, and sets the microphone 10 so that a signal is output based only on the air-conduction microphone 10.

[0067] At this time, the wearing state detection unit 41 may also set on the noise cancellation of the air conduction microphone 10. This causes noise cancellation to be performed on the air conduction sound picked up by the air conduction microphone 10, and a signal to be output by the communication device 2 of the other party is generated based on the air conduction sound after the noise cancellation.

[0068] For example, when the level of the air conduction sound is below a predetermined threshold and the difference between the level of the air conduction sound and the level of the bone conduction sound picked up at the same time is equal to or greater than the difference threshold, the wearing status detection unit 41 determines that there is an abnormality in the sound pickup by the air conduction microphone 10, and sets the signal to be output based only on the bone conduction microphone 11.

[0069] The wearing state detection unit 41 may also determine the presence or absence of an abnormality in the air conduction microphone 10 and the bone conduction microphone 11 by determining the presence or absence of a correlation between the level of the bone conduction sound and the level of the air conduction sound picked up at the same timing. Specifically, the wearing state detection unit 41 may calculate a correlation coefficient between the level of the bone conduction sound and the level of the air conduction sound picked up at the same timing, and determine that there is an abnormality in the sound pickup of either one of them when the correlation coefficient is equal to or less than a predetermined threshold value less than 1. Since the sound pickup by the bone conduction microphone 11 tends to depend on the wearing state, the wearing state detection unit 41 may determine that there is an abnormality in the sound pickup by the bone conduction microphone 11 when the correlation coefficient is equal to or less than a predetermined threshold value.

[0070] The wearing state detection unit 41 further obtains the frequency characteristic of the bone-conducted sound by performing a Fourier transform on the level of the bone-conducted sound over a predetermined time. Next, the wearing state detection unit 41 obtains the frequency characteristic of the level of the air-conducted sound. After that, the wearing state detection unit 41 compares the two frequency characteristics and obtains the level difference in the low frequency region below a predetermined threshold and the high frequency region above a predetermined threshold.

[0071] The sound collection state detection unit 42 detects the sound collection state based on the sound collection results by the air conduction microphone 10 and the bone conduction microphone 11. Here, the sound collection state detection unit 42 acquires the level of the ambient sound (ambient sound level) of the user and the SNR based on the sound collection results by the air conduction microphone 10 and the bone conduction microphone 11.

[0072] Here, the sound collection state detection unit 42 detects the sound collection state based on the sound collection results of the air conduction microphone 10 and the bone conduction microphone 11, and implements a determination method for determining a generation method for a signal to be output. The process of determining the generation method may include so-called microphone selection, in which the generation of the signal to be output is based only on the sound signal collected by the air conduction microphone 10 or only on the sound signal collected by the bone conduction microphone 11, and setting of the mixing ratio of the sound signals collected by both microphones. In order to implement this determination method, the sound collection state detection unit 42 executes a predetermined program and performs output setting processing.

[0073] The signal-to-noise ratio refers to the ratio of the signal level to the noise level, and the signal here refers to the voice of the user, that is, the sound. The noise level increases as the noise in the user's surrounding environment increases. In other words, the lower the signal-to-noise ratio, the higher the noise level in the surrounding environment, meaning that the user's voice is difficult to hear. Therefore, the lower the signal-to-noise ratio, the more desirable it is to prioritize the bone-conduction microphone 11. However, if the bone-conduction microphone 11 is prioritized despite a high signal-to-noise ratio, sound with unclear quality will be output.

[0074] Therefore, the signal processing device 4 performs appropriate filtering on the sound signals detected by the bone conduction microphone 11 and the air conduction microphone 10, and then generates a signal to be output by selecting one of the sound signals or mixing them in a predetermined ratio.

[0075] Here, the sound collection state detection unit 42 performs output setting processing to generate sound that matches the external environment, and determines the selection of one of the sound signals, the mixing ratio of bone-conducted sound and air-conducted sound, and the filter processing method based on the level of the ambient sound (hereinafter, ambient sound level) and the signal-to-noise ratio, and determines the method of generating the signal to be output.

[0076] The ambient sound level is determined based on either the level of air-conducted ambient sound or the level of bone-conducted ambient sound. The signal level used to calculate the signal-to-noise ratio is determined based on either the level of bone-conducted sound or the level of air-conducted sound. The noise level used to calculate the signal-to-noise ratio is determined based on either the air-conducted ambient sound or the bone-conducted ambient sound used for the ambient sound level, or the other. In other words, the type of ambient sound used for the ambient sound level and the type of ambient sound used for the noise level of the signal-to-noise ratio may be the same or different.

[0077] The sound collection state detection unit 42 can acquire bone-conducted ambient sound by performing high-pass filter (HPF) processing on the sound signal collected by the bone-conduction microphone 11 during speech, extracting only frequency components equal to or higher than a predetermined cutoff frequency. The reason why it is desirable to perform high-pass filter processing is that bone-conducted sound is a sound in which high-frequency components are attenuated and low-frequency components are emphasized as it passes through the body, so that by appropriately setting the cutoff frequency, it is possible to enhance the high-frequency components contained in the ambient sound.

[0078] The sound collection state detection unit 42 may obtain the level of the bone-conducted sound by performing low-pass filter (LPF) processing on the sound signal collected by the bone-conduction microphone 11 during speech, extracting only frequency components equal to or lower than a predetermined cutoff frequency. By appropriately setting the cutoff frequency, the influence of sources other than the user's voice can be reduced.

[0079] When acquiring the ambient sound level and the signal-to-noise ratio, the sound collection state detection unit 42 may receive a notification from the air conduction receiver 12 so that the air conduction microphone 10 or the bone conduction microphone 11 collects the sound.

[0080] The sound collection status detection unit 42 may be configured to determine whether the user is speaking (determine whether there is a sound or silent section) by setting a predetermined threshold based on the level of the sound signal collected by the bone conduction microphone 11 or the air conduction microphone 10. In this case, the sound collection status detection unit 42 may determine a time period in which the level of the sound signal collected by the bone conduction microphone 11 or the air conduction microphone 10 is equal to or higher than a predetermined threshold as a sound section (a time period when the user is speaking), and a time period in which the level is less than the threshold as a silent section (a time period when the user is not speaking).

[0081] The sound collection state detection unit 42 can regard the level of the sound signal collected in the silent section as the noise level used to calculate the signal-to-noise ratio or the ambient sound level. However, the microphones used to obtain the noise level of the signal-to-noise ratio and the ambient sound level may be different. For example, the noise level of the signal-to-noise ratio may be the level of the bone-conducted ambient sound collected by the bone-conducted microphone 11, and the ambient sound level may be the level of the air-conducted ambient sound collected by the air-conducted microphone 10.

[0082] The sound collection state detection unit 42 may also use the level of the sound collected in the sound section as the signal level of the signal-to-noise ratio. That is, the signal level may be the level of the bone-conducted sound in the sound section, or may be the level of the air-conducted sound.

[0083] Hereinafter, the signal level of the signal-to-noise ratio will be simply referred to as the voice level. That is, the signal-to-noise ratio is calculated by the ratio between the voice level and the noise level.

[0084] The sound collection state detection unit 42 may acquire an average value or a representative value of the bone-conducted sound or air-conducted sound level during sound intervals as the sound level, and may acquire an average value or a representative value of the bone-conducted ambient sound or air-conducted ambient sound during silent intervals as the noise level or ambient sound level.

[0085] FIG. 4 shows, in the form of a table, examples of combinations (patterns) of microphone types and filter processes used when determining whether a voice or silent section is active and when acquiring an ambient sound level, a voice level, and a noise level.

[0086] In pattern 1, the sound signal collected by the bone conduction microphone 11 after low-pass filter processing is used to determine whether there is sound or not. Also, the sound signal of the silent section collected by the air conduction microphone 10 is used to obtain the ambient sound level. The sound signal of the silent section collected by the air conduction microphone 10 is used to obtain the voice level, and the sound signal of the silent section collected by the air conduction microphone 10 is used to obtain the noise level.

[0087] In pattern 2, the sound signal collected by the bone-conduction microphone 11 after low-pass filtering is used to determine whether sound is present or absent. Also, the sound signal of the silent section collected by the bone-conduction microphone 11 that has been subjected to high-pass filtering is used to obtain the ambient sound level. The sound signal of the silent section collected by the bone-conduction microphone 11 that has been subjected to low-pass filtering is used to obtain the voice level, and the sound signal of the silent section collected by the bone-conduction microphone 11 that has been subjected to high-pass filtering is used to obtain the noise level.

[0088] In pattern 3, the signal collected by the bone-conduction microphone 11 after low-pass filtering is used to determine whether there is sound or not. Also, the sound signal of the silent section collected by the bone-conduction microphone 11 that has been subjected to high-pass filtering is used to obtain the ambient sound level. The sound signal of the sound section collected by the bone-conduction microphone 11 that has been subjected to low-pass filtering is used to obtain the sound level, and the sound signal of the sound section collected by the bone-conduction microphone 11 that has been subjected to high-pass filtering is used to obtain the noise level.

[0089] In pattern 4, the sound signal collected by the bone conduction microphone 11 after low-pass filtering is used to determine whether sound is present or absent. Also, the sound signal of the silent section collected by the air conduction microphone 10 is processed by high-pass filtering to obtain the ambient sound level. The sound signal of the silent section collected by the bone conduction microphone 11 is processed by low-pass filtering to obtain the voice level, and the sound signal of the silent section collected by the bone conduction microphone 11 is processed by high-pass filtering to obtain the noise level.

[0090] When the sound collection state detection unit 42 acquires the ambient sound level and the signal-to-noise ratio, it selects a microphone, sets the mixing ratio of the sound signals collected by both microphones, and turns noise cancellation on and off based on the ambient sound level and the signal-to-noise ratio.

[0091] When the ambient sound level is equal to or higher than a predetermined level threshold (i.e., when the noise level is high), the sound collection state detection unit 42 selects the bone-conduction microphone 11 and determines a method for generating an output signal so that a signal is output based on the bone-conduction sound. At this time, the sound collection state detection unit 42 switches the path (also called a route or path) through which the signal passes and sets it on and off by a known method so that an output is made based only on the sound signal collected by the bone-conduction microphone 11. Furthermore, the sound collection state detection unit 42 sets the noise cancellation of the bone-conduction microphone 11 to on.

[0092] However, the present disclosure is not limited to this aspect, and the sound collection state detection unit 42 may set the mixing ratio so that the proportion of bone conduction sound is sufficiently higher than that of air conduction sound when the ambient sound level is equal to or higher than a predetermined threshold.

[0093] When the ambient sound level is less than the level threshold and the signal-to-noise ratio is less than a predetermined first threshold, the sound collection state detection unit 42 selects the bone-conduction microphone 11 and determines a method for generating an output signal so that a signal is output based on the bone-conduction sound. At this time, the sound collection state detection unit 42 switches paths and sets ON / OFF by a known method so that output is based only on the sound signal collected by the bone-conduction microphone 11. Furthermore, the sound collection state detection unit 42 sets the noise cancellation of the bone-conduction microphone 11 to ON.

[0094] The sound collection state detection unit 42 determines a method for generating an output signal so that a signal is output based on air conduction sound when the ambient sound level is below the level threshold and the signal-to-noise ratio is greater than a second threshold. At this time, the sound collection state detection unit 42 switches paths and sets ON / OFF by a known method so that an output is based only on the sound signal collected by the air conduction microphone 10. Furthermore, the sound collection state detection unit 42 sets the noise cancellation of the air conduction microphone 10 to ON.

[0095] When the ambient sound level is below a level threshold and the signal-to-noise ratio is equal to or greater than a first threshold and equal to or less than a second threshold, the sound collection state detection unit 42 determines a method for generating an output signal so that a signal is output based on both bone conduction sound and air conduction sound.

[0096] When the ambient sound level is below the level threshold and the signal-to-noise ratio is equal to or greater than the first threshold and equal to or less than the second threshold, the sound collection state detection unit 42 sets the mixing ratio so that as the signal-to-noise ratio increases, the proportion of air conduction sound becomes greater than that of bone conduction sound.

[0097] At this time, the sound collection state detection unit 42 sets the noise cancellation of the air conduction microphone 10 to ON, and sets the noise cancellation of the bone conduction microphone 11 to OFF.

[0098] When there is a frequency band in which the frequency characteristics of the bone-conducted sound level and the frequency characteristics of the air-conducted sound level have a difference of more than a predetermined threshold, the sound collection state detection unit 42 may set a mixing ratio for each frequency band in which there is a difference of more than the predetermined threshold.

[0099] For example, when the level of the bone-conducted sound is lower than that of the air-conducted sound in the high frequency range, the sound collection state detection unit 42 may set a gain to increase the bone-conducted sound in the high frequency range and correct the mixing ratio. This causes the bone-conducted sound to be amplified and mixed (synthesized) with the air-conducted sound in the high frequency range, compensating for attenuation of the bone-conducted sound in the high frequency range and increasing the clarity of the output signal.

[0100] In this way, when the ratio of air conduction sound is not zero, the sound collection state detection unit 42 sets the noise cancellation of the air conduction microphone 10 to ON and sets the noise cancellation of the bone conduction microphone 11 to OFF. Also, when the ratio of air conduction sound is zero, the sound collection state detection unit 42 sets the noise cancellation of the bone conduction microphone 11 to ON.

[0101] The path setting unit 43 controls the noise cancellation unit 33, the mixer 34, and the switch 45 to realize the microphone selection, the mixing ratio, and the noise cancellation setting determined by the sound collection state detection unit 42. In this way, the path setting unit 43 controls the path (route) through which the output signals of the bone-conduction microphone 11 and the air-conduction microphone 10 pass so that sound is output from the selected microphone.

[0102] The path setting unit 43 controls various switches 45 and the mixer 34 to realize an output based only on the sound signal collected by the air conduction microphone 10, an output based only on the sound signal collected by the bone conduction microphone 11, or an output at a predetermined mixing ratio based on both the air conduction microphone 10 and the bone conduction microphone 11. The switch 45 is provided on a path from the air conduction microphone 10 to the mixer 34 (see FIG. 3) and a path from the bone conduction microphone 11 to the mixer 34 (not shown). By controlling the switch 45, the path setting unit 43 can block the output of the bone conduction microphone 11 (switch the switch 45 in FIG. 3) and control the path through which the signal passes so that an output is based only on the output of the air conduction microphone 10. Similarly, the path setting unit 43 can set a path so that an output is based only on the output of the bone conduction microphone 11 by controlling the switch 45.

[0103] In addition, when the sound collection state detection unit 42 sets the noise cancellation of the air conduction microphone 10 to ON, the path setting unit 43 sets a path to input the output of the air conduction microphone 10 to the noise cancellation unit 33 and performs noise cancellation processing. After that, the path setting unit 43 inputs the output of the noise cancellation unit 33 to the mixer 34 and appropriately mixes it with the output of the bone conduction microphone 11. In this way, the path setting unit 43 controls the path through which the signal passes so that the output signal of the air conduction microphone 10 passes through the noise cancellation unit 33 and is input to the mixer 34.

[0104] The microphone output unit 37 outputs a signal corresponding to the mixed sound signal output from the mixer 34 to the communication device 2 of the other party. As a result, the communication device 2 of the other party generates a mixed signal corresponding to the acquired signal from the air conduction receiver 12, a speaker, etc.

[0105] Next, the operation of the microphone device 1 and the communication device 2 thus configured will be described.

[0106] When the user performs a predetermined input to the signal processing device 4 to start using the communication device 2, the processor of the signal processing device 4 displays a notification that initial settings will be performed and a setting start button on the touch panel. At this time, when the user touches the setting start button, the processor of the signal processing device 4 executes a predetermined program (setting program) to perform setting processing, and sets noise cancellation settings, microphone selection, and mixing ratio. Figure 5 shows a flowchart showing an example of the setting processing.

[0107] The processor of the signal processing device 4 first selects one pattern from the combinations of microphone types and filter processes. At this time, the processor may select one pattern from the patterns shown in Fig. 4. The processor may also display a selection screen on a touch panel and accept input from the user regarding the pattern selection.

[0108] When the pattern selection is completed, the processor causes the touch panel to display a notification urging the user to wear the earphones 3 and speak.

[0109] Once the display is complete, the processor first acquires, from the air conduction microphone 10 and the bone conduction microphone 11, respectively, a sound signal when the user is speaking (sound signal when there is sound) and a sound signal when the user is not speaking (sound signal when there is no sound) (more specifically, signals corresponding to each sound signal).

[0110] At this time, the processor judges whether there is sound or no sound based on the selected pattern. Specifically, for example, when pattern 1 in Fig. 4 is selected, the processor judges whether there is sound or no sound based on a signal obtained by low-pass filtering of a sound signal collected by the bone-conduction microphone 11, and judges that the signal is sound when it is equal to or greater than a predetermined threshold value, and that the signal is no sound when it is less than the threshold value. At this time, the processor may perform low-pass filtering of the sound signal collected by the bone-conduction microphone 11 by controlling the pre-processing unit 32 and various switches.

[0111] The processor acquires the sound signal collected by the air conduction microphone 10 when there is sound as air conduction sound, and acquires the sound signal collected by the air conduction microphone 10 when there is no sound as air conduction ambient sound. The processor acquires the sound signal collected by the bone conduction microphone 11 when there is sound as bone conduction sound, and acquires the sound signal collected by the bone conduction microphone 11 when there is no sound as bone conduction ambient sound.

[0112] After that, the processor (the mounting state detection unit 41) first performs an abnormality detection process.

[0113] FIG. 5 shows a flowchart of the abnormality detection process.

[0114] In the first step of the abnormality detection process, the processor determines whether or not there is an abnormality in the sound collection by the air conduction microphone 10 (ST101). Specifically, when the level of the air conduction sound is less than the corresponding threshold, the processor determines that there is an abnormality in the sound collection by the air conduction microphone 10, and when the level of the air conduction sound is equal to or greater than the corresponding threshold, the processor determines that there is no abnormality in the sound collection by the air conduction microphone 10.

[0115] When it is determined that there is an abnormality in the sound collection by the air conduction microphone 10 (Yes in ST101), the processor selects the bone conduction microphone 11, sets a path so that output is performed based only on the sound signal collected by the bone conduction microphone 11, and sets the noise cancellation of the bone conduction microphone 11 to ON (ST102). When the setting is completed, the processor ends the abnormality detection process.

[0116] When it is determined that there is no abnormality in the sound collection by the air conduction microphone 10 (No in ST101), the processor determines whether there is an abnormality in the sound collection by the bone conduction microphone 11 (ST103). Specifically, when the level of the bone conduction sound is below the corresponding threshold, the processor determines that there is an abnormality in the sound collection by the bone conduction microphone 11 due to reasons such as the microphone device 1 not being properly attached, and when it is equal to or above the threshold, the processor determines that there is no abnormality in the sound collection by the bone conduction microphone 11.

[0117] When it is determined that there is an abnormality in the sound collection by the bone conduction microphone 11 (Yes in ST103), the processor selects the air conduction microphone 10, sets a path so that output is performed based only on the sound signal collected by the air conduction microphone 10, and sets the noise cancellation of the air conduction microphone 10 to ON (ST104). When the setting is completed, the processor ends the abnormality detection process.

[0118] When it is determined that there is no abnormality in the sound collection by the bone-conduction microphone 11 (No in ST103), the processor acquires the level difference between the bone-conduction sound and the air-conduction sound. Then, the processor determines whether the level difference between the bone-conduction sound and the air-conduction sound is equal to or greater than a difference threshold (ST105).

[0119] When the level difference between the bone-conduction sound and the air-conduction sound is equal to or greater than the difference threshold (Yes in ST105), the processor determines that there is an abnormality in the sound pickup by the bone-conduction microphone 11, selects the air-conduction microphone 10, sets a path so that a signal is output based only on the air-conduction microphone 10, and sets the noise cancellation of the air-conduction sound to ON (ST106). When the settings are complete, the processor ends the abnormality detection process.

[0120] When the level of the bone-conducted sound is equal to or higher than the level of the air-conducted sound, or when the level difference between the bone-conducted sound and the air-conducted sound is less than a predetermined threshold (No in ST105), the processor acquires the frequency characteristics of the air-conducted sound level and the bone-conducted sound level, and calculates the level difference in the low and high ranges.Then, the processor updates the frequency correction table to set an appropriate gain for each frequency based on the calculated level difference, and changes (corrects) the frequency characteristics (ST107).The frequency correction table may be, for example, a table in which frequencies and appropriate gains corresponding to the frequencies are recorded.

[0121] For example, the processor may update the frequency correction table so as to reduce the level difference in a frequency band in which the level difference between bone-conducted sound and air-conducted sound is large.

[0122] When the gain setting is completed, the processor ends the abnormality detection process.

[0123] In the abnormality detection process, if no abnormality is detected in the sound collection by the air conduction microphone 10 and the bone conduction microphone 11, the processor (sound collection state detection unit 42) starts the output setting process. In the output setting process, the processor sets the method of generating the signal to be output by selecting the microphone, setting the mixing ratio of the air conduction sound and the bone conduction sound, and turning on / off the noise cancellation.

[0124] FIG. 6 shows a flowchart of the output setting process.

[0125] When the processor starts the output setting process, it first obtains the ambient sound level and the signal-to-noise ratio based on the selected pattern (ST111).

[0126] For example, when pattern 1 is selected, the processor acquires the level of the sound signal collected by the air conduction microphone 10 when there is sound (sound section) as the ambient sound level. The processor also acquires the voice level from the level of the sound signal collected by the air conduction microphone 10 when there is sound (sound section), and acquires the noise level from the level of the sound signal collected by the air conduction microphone 10 when there is no sound (silent section). After that, the processor acquires the signal-to-noise ratio (SNR) by, for example, dividing the voice level by the noise level and taking the logarithm.

[0127] At this time, the processor determines whether a sound section or a silent section is present based on a signal obtained by low-pass filtering the output of the bone-conduction microphone 11. Specifically, the processor performs low-pass filtering on the output of the bone-conduction microphone 11, and determines that there is sound when the level of the processed signal is equal to or higher than a predetermined threshold, and determines that there is no sound when the level is less than the threshold.

[0128] After the acquisition of the ambient sound level and the signal-to-noise ratio is completed, the processor determines whether the ambient sound level is greater than a predetermined threshold (ST112).

[0129] When the ambient sound level is greater than a predetermined threshold (level threshold), i.e., when the ambient sound is loud (Yes in ST112), the processor selects the bone conduction microphone 11, sets a path so that a signal is output based only on the bone conduction microphone 11, and sets the noise cancellation of the bone conduction microphone 11 to on (ST113).

[0130] Next, the processor changes the frequency characteristics based on the frequency correction table (ST114). However, the frequency characteristics are changed only for the bone-conduction microphone 11, and the processor changes (also called corrects) only the frequency characteristics of the bone-conduction microphone 11. When the processor has completed changing the frequency characteristics, it ends the output setting process.

[0131] When the ambient sound level is equal to or lower than the predetermined threshold, that is, when the ambient sound is small (No in ST112), the processor determines whether or not the signal-to-noise ratio is equal to or lower than a first threshold (ST115).

[0132] When the signal-to-noise ratio is equal to or lower than the first threshold (Yes in ST115), the processor selects the bone-conduction microphone 11, sets a path so that a signal is output based only on the sound pickup results by the bone-conduction microphone 11, and sets the noise cancellation of the bone-conduction microphone 11 to on (ST116).

[0133] Next, the processor changes the frequency characteristics based on the frequency correction table (ST117). However, the frequency characteristics are changed only for the bone-conduction microphone 11, and the processor changes (also called corrects) only the frequency characteristics of the bone-conduction microphone 11. When the processor has completed changing the frequency characteristics, it ends the output setting process.

[0134] When the signal-to-noise ratio is greater than the first threshold (No in ST115), the processor determines whether the signal-to-noise ratio is equal to or less than a second threshold (ST118).

[0135] When the processor determines that the signal-to-noise ratio is equal to or less than the second threshold (Yes in ST118, i.e., when the signal-to-noise ratio is greater than the first threshold and equal to or less than the second threshold), it sets the mixture ratio of air conduction sound and bone conduction sound based on the signal-to-noise ratio (ST119). At this time, the processor may set the mixture ratio so that the proportion of air conduction sound increases as the signal-to-noise ratio increases.

[0136] In this embodiment, at this time, the processor sets the noise cancellation of the air conduction microphone 10 to ON and sets the noise cancellation of the bone conduction microphone 11 to OFF.

[0137] Next, the processor changes the frequency characteristics based on the frequency correction table (ST120). However, the frequency characteristics are changed only for the bone-conduction microphone 11, and the processor changes (also called corrects) only the frequency characteristics of the bone-conduction microphone 11. When the processor has completed changing the frequency characteristics, it ends the output setting process.

[0138] When the processor determines that the signal-to-noise ratio is greater than the second threshold (No in ST118), it selects the air conduction microphone 10, sets a path so that sound is output based only on the sound pickup result by the air conduction microphone 10, and sets the noise cancellation of the air conduction microphone 10 to on (ST121). When the path setting (microphone selection) and the noise cancellation on / off setting are completed, the processor ends the output setting process.

[0139] When the output setting process is completed, the processor controls the amplifier 31, preprocessor 32, noise canceller 33, mixer 34, and switch 45 to output an audio signal based on the microphone selection, mixing ratio, on / off of noise cancellation, and gain set in the output setting process, and synthesizes the audio signal. After that, the processor causes the microphone output unit 37 to transmit a signal corresponding to the synthesized voice to the communication device 2 of the user's call partner.

[0140] The communication device 2 of the communication partner converts the signal transmitted from the microphone output unit 37 into voice and outputs it using the air conduction receiver 12, a speaker, and the like.

[0141] Next, the effects of the microphone device 1 and the communication device 2 thus configured will be described.

[0142] When the ambient sound level is higher than the threshold (Yes in ST112) or when the signal-to-noise ratio is equal to or lower than the first threshold (Yes in ST114), the bone-conduction microphone 11 is selected (ST113, ST116). As a result, when the user is in an environment with a high ambient sound level, the bone-conduction microphone 11, which is less susceptible to the influence of ambient sound, is selected.

[0143] When the ambient sound level is equal to or lower than the threshold and the signal-to-noise ratio is greater than the second threshold (No in ST112, No in ST115, No in ST118), the air conduction microphone 10 is selected. As a result, when the ambient sound level is low and the signal-to-noise ratio is large, the air conduction microphone 10 is output with priority (ST121). In other words, when the ambient sound level is low and the signal-to-noise ratio is large, the air conduction microphone 10, which is less likely to attenuate up to a high frequency band and has high clarity, is selected.

[0144] In this way, the microphone device 1 synthesizes signals to be output by prioritizing bone conduction sound when the noise level is high and air conduction sound when the signal level is high. In this way, the microphone device 1 according to the present disclosure optimally selects between the air conduction microphone 10 and the bone conduction microphone 11 according to the noise environment around the user so as to output an audio signal that is less susceptible to noise and has higher clarity.

[0145] 7 shows the output signal of the bone-conduction microphone 11 in three environments of low, medium, and high ambient sound when pattern 1 is selected, the signal after the output signal is low-pass filtered, the distinction between sound and silent periods, and the output signal of the air-conduction microphone 10. In the graph of the output signal of the bone-conduction microphone 11, the time periods when the user speaks are indicated by double-headed arrows. As can be seen from FIG. 7, even when the ambient sound is relatively high, the time periods when the user speaks can be extracted by low-pass filtering the output signal of the bone-conduction microphone 11, and the distinction between sound and silent periods can be performed well.

[0146] 7 also shows, by a solid line and a thin line, a representative value of the level of the sound signal picked up by the air conduction microphone 10 during a sound section, and a measured value of the level of the sound signal picked up by the air conduction microphone 10 during a silent section. The former corresponds to the sound level used in calculating the SNR, and the latter corresponds to the ambient sound level and the noise level used in calculating the SNR. Furthermore, in FIG. 7, the SNR calculated during a sound section is shown in two levels (OK: 10 dB or more, NG: 10 dB or less).

[0147] In pattern 1, when the ambient sound is low, the signal-to-noise ratio is higher than when the ambient sound is high. Therefore, when the ambient sound is low, the air conduction microphone 10 is used with priority, and when the ambient sound is high, the bone conduction microphone 11 is used with priority.

[0148] 8 shows the output signal of the bone-conduction microphone 11 in three environments of low, medium, and high ambient sound when pattern 2 is selected, a signal obtained by low-pass filtering the output signal, a distinction between sound and silence periods, and a signal obtained by high-pass filtering the output signal of the bone-conduction microphone 11. In the graph of the output signal of the bone-conduction microphone 11, the time periods when the user spoke are indicated by double-headed arrows, as in FIG. 7.

[0149] Fig. 8 also shows, by a solid line and a thin line, a representative value of the level of the sound signal collected by the bone-conduction microphone 11 in a sound section and a representative value of the level of the sound signal collected by the bone-conduction microphone 11 in a silent section. The former corresponds to the sound level used in calculating the SNR, and the latter corresponds to the ambient sound level and the noise level used in calculating the SNR. Fig. 8 also shows three levels of SNR calculated in a sound section (10 dB or more is good, 0 to 10 dB is acceptable, and 0 dB or less is unacceptable).

[0150] In this case, as in the case of pattern 1, when the ambient sound is low, the signal-to-noise ratio is higher than when the ambient sound is high. Therefore, when the ambient sound is low, the air conduction microphone 10 is used preferentially, and when the ambient sound is high, the bone conduction microphone 11 is used preferentially.

[0151] 9 shows the output signal of the bone-conduction microphone 11 in three environments of low, medium, and high ambient sound when pattern 3 is selected, a signal obtained by low-pass filtering the output signal, a distinction between sound and silence periods, and a signal obtained by high-pass filtering the output signal of the bone-conduction microphone 11. In the graph of the output signal of the bone-conduction microphone 11, the time periods when the user spoke are indicated by double-headed arrows, as in FIG. 7.

[0152] Fig. 9 also shows the signal after high-pass filtering of the output signal of the bone-conduction microphone 11 and the ambient sound level. The ambient sound level is obtained by high-pass filtering the output of the bone-conduction microphone 11 in the silent section and acquiring a representative value. Fig. 9 also shows the SNR calculated in the silent section.

[0153] In this case, as in the case of pattern 1, when the ambient sound is low, the signal-to-noise ratio is higher than when the ambient sound is high. Therefore, when the ambient sound is low, the air conduction microphone 10 is used preferentially, and when the ambient sound is high, the bone conduction microphone 11 is used preferentially.

[0154] In addition, in the pattern 3, the noise level is obtained by subjecting the output signal of the bone-conduction microphone 11 to high-pass filtering, and the voice level is obtained by subjecting the output signal of the bone-conduction microphone 11 to low-pass filtering (see FIG. 4). Therefore, the SNR can be obtained in real time.

[0155] 10 shows the output signal of the bone-conduction microphone 11 in three environments of low, medium, and high ambient sound when pattern 4 is selected, the signal after the output signal is subjected to low-pass filter processing, the distinction between sound and silence periods, and the output signal of the air-conduction microphone 10. In the graph of the output signal of the bone-conduction microphone 11, the time periods when the user spoke are indicated by double-headed arrows, as in FIG.

[0156] Fig. 10 also shows the measured value of the level of the sound signal collected by the air conduction microphone 10 during the silent section. The measured value of the level of the sound signal collected by the air conduction microphone 10 during the silent section corresponds to the ambient sound level. Fig. 10 also shows the SNR calculated during the silent section.

[0157] In this case, as in the case of pattern 1, when the ambient sound is low, the signal-to-noise ratio is higher than when the ambient sound is high. Therefore, when the ambient sound is low, the air conduction microphone 10 is used preferentially, and when the ambient sound is high, the bone conduction microphone 11 is used preferentially.

[0158] In addition, in the pattern 4, similarly to the pattern 3, the noise level is obtained by subjecting the output signal of the bone-conduction microphone 11 to high-pass filtering, and the voice level is obtained by subjecting the output signal of the bone-conduction microphone 11 to low-pass filtering (see FIG. 4). Therefore, the SNR can be obtained in real time.

[0159] In this way, the microphone device 1 and the communication device 2 according to the present disclosure select the best one between the air conduction microphone 10 and the bone conduction microphone 11 and mix the sounds collected by each microphone based on the necessary frequency band setting focusing on human vocalization and the results of sound collection state detection such as the ambient sound level and SNR. This allows ideal sound collection performance to be achieved with the minimum necessary configuration including the air conduction microphone 10 and the bone conduction microphone 11.

[0160] In this way, the present disclosure makes it possible to realize a microphone device 1 and a communication device 2 that can deliver "only" the user's conversation to the other party under any conditions or environment, regardless of the user's speaking volume or surrounding environment, and that can achieve microphone sound quality that is far superior to that of conventional devices and that is good.

[0161] In this way, the present disclosure provides a configuration for making the best selection between the air conduction microphone 10 and the bone conduction microphone 11, and demonstrates the versatility of SNR judgment in detecting the sound collection condition. With the present invention, it is now possible, for the first time, to achieve good microphone sound quality that surpasses conventional microphone sound quality.

[0162] As described above, when the user's speaking volume is high (i.e., when speaking loudly), it is desirable to use only the air conduction microphone 10, and when the user's speaking volume is low (i.e., when speaking softly), it is desirable to use only the bone conduction microphone 11. The present disclosure realizes a configuration in which the air conduction microphone 10 is used preferentially when speaking loudly and the bone conduction microphone 11 is used preferentially when speaking softly, i.e., the best selection between the air conduction microphone 10 and the bone conduction microphone 11 is realized.

[0163] Fig. 11(A) shows an example of a map indicating the selection range of the air conduction microphone 10 and the bone conduction microphone 11 according to the present disclosure. Fig. 11(A) shows the type of microphone to be selected when the horizontal axis indicates the sound level in decibels (dB) and the vertical axis indicates the ambient sound level in decibels (dB). In the example shown in Fig. 11(A), when the SNR (S / N) is equal to or higher than a predetermined threshold, the air conduction microphone 10 is used preferentially.

[0164] As a result, when speaking loudly (i.e., when the voice level is high), the range in which only the air conduction microphone 10 is used is set to be wide, and the air conduction microphone 10 is used preferentially. When speaking softly (i.e., when the voice level is low), the range in which only the bone conduction microphone 11 is used is set to be wide, and the bone conduction microphone 11 is used preferentially. In other words, as the generated volume increases, the air conduction microphone 10 is used preferentially, and as the generated volume decreases, the bone conduction microphone 11 is used preferentially. Thus, the optimal selection between the air conduction microphone 10 and the bone conduction microphone 11 is realized.

[0165] However, the map showing the selection range of the air conduction microphone 10 and the bone conduction microphone 11 is not limited to the embodiment of FIG. 11(A), and various modifications are possible. FIGS. 11(B), (C), and (D) show a first modified example, a second modified example, and a third modified example of the map, respectively. As in FIG. 11(A), FIGS. 11(B), (C), and (D) also show the type of microphone to be selected when the sound level in decibels (dB) is the horizontal axis and the ambient sound level in decibels (dB) is the vertical axis. In the second modified example of FIG. 11(B), the air conduction microphone 10 and the bone conduction microphone 11 are used together, and the region where the output of the air conduction microphone 10 and the output of the bone conduction microphone 11 are mixed is described as "mixing."

[0166] 11(B), (C), and (D), when speaking loudly, the range in which only the air conduction microphone 10 is used is set to be wide, and the air conduction microphone 10 is used preferentially. When speaking softly, the range in which only the bone conduction microphone 11 is used is set to be wide, and the bone conduction microphone 11 is used preferentially. This makes it possible to optimally select between the air conduction microphone 10 and the bone conduction microphone 11.

[0167] In the present invention, since the voice level and the ambient sound level are expressed in units of decibels (dB), the SNR in the present disclosure may be defined as SNR=(α×S)-(N+β) (α and β are unique correction coefficients for defining the SNR). In addition, the SNR is not limited to a linear function of S and N, and a quadratic function or a logarithmic function may also be applied.

[0168] Fig. 11(A) shows an example in which switching between the bone-conduction microphone 11 and the air-conduction microphone 10 is performed by one threshold, and the threshold is expressed as SNR=SN where α=1 and β=0. Fig. 11(B) shows an example in which switching between the bone-conduction microphone 11 and a mixture of the bone-conduction microphone 11 and the air-conduction microphone 10 is performed by a first threshold, and switching between the mixture of the bone-conduction microphone 11 and the air-conduction microphone 10 and the air-conduction microphone 10 is performed by a second threshold. The first threshold is expressed as SNR=S-(N+a) where α=1 and β=a (where a is an arbitrary value), and the second threshold is expressed as SNR=S-(N+b) where α=1 and β=b (where b is an arbitrary value such that a>b). FIG. 11(C) shows an example in which switching between the bone-conduction microphone 11 and the air-conduction microphone 10 is performed by one threshold value, and the threshold value is expressed as SNR=S-(N+c) where α=1 and β=c (where c is an arbitrary value). This corresponds to changing the offset of the vertical axis N. FIG. 11(D) shows an example in which switching between the bone-conduction microphone 11 and the air-conduction microphone 10 is performed by one threshold value, and the threshold value is expressed as SNR=(d×S)-(N+e) where α=d and β=e (where d is an arbitrary value other than 1 and e is an arbitrary value). This corresponds to changing the slope of the horizontal axis S.

[0169] As described above, the embodiment has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which modifications, replacements, additions, omissions, etc. are performed. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. In addition, the microphone device 1 in the present application can be applied to, for example, a wired type headset, a wireless type headset such as Bluetooth (registered trademark), a full wireless headset, a transceiver, an intercommunication (intercom), a head mounted display, etc. In addition, the microphone device 1 in the present application can be applied not only to telephone calls but also to applications such as a voice recorder (recording).

[0170] In the above embodiment, the processor (sound collection state detection unit 42) of the signal processing device 4 is configured to set the mixing ratio after acquiring the ambient sound level and the signal-to-noise ratio, but this is not limited to the above embodiment. For example, the processor (sound collection state detection unit 42) of the signal processing device 4 may first acquire only the ambient sound level, and set a path so that an audio signal is output based only on the bone-conduction microphone 11 (or so that the bone-conduction microphone 11 is prioritized) when the acquired ambient sound level is equal to or higher than a predetermined threshold (or is greater than the predetermined threshold). In this case, since the signal-to-noise ratio is not acquired, it is possible to reduce the processing to be performed by the processor (sound collection state detection unit 42) of the signal processing device 4 when the ambient sound level is high.

[0171] In the above embodiment, an example was described in which the first threshold and the second threshold are each a predetermined constant, but the processor (sound collection state detection unit 42) of the signal processing device 4 may set the first threshold and the second threshold based on the ambient sound level before comparing the signal-to-noise ratios.

[0172] In step ST105 of the above embodiment, the determination is made based on the level difference between the air conduction sound and the bone conduction sound, but the present invention is not limited to this, and the microphone device 1 and the communication device 2 may be configured to make a determination based on the level difference between the air conduction sound and the bone conduction sound. In addition, when the level difference is equal to or greater than a level threshold, the air conduction microphone 10 is configured to be prioritized in step ST106, but when the level of the air conduction sound is lower than the level of the bone conduction sound by a predetermined value or more, the bone conduction microphone 11 may be prioritized, and a signal to be output based only on the bone conduction sound may be generated.

[0173] In the above embodiment, it has been described that the air conduction microphone 10 and the bone conduction microphone 11 are selected and output is based on the sound signal picked up by either one of them, but this selection also includes a case where the proportion of the sound picked up by the air conduction microphone 10 is sufficiently high and the mixing ratio is set so that the sound is essentially only from the air conduction microphone 10. Similarly, it also includes a case where the proportion of the sound picked up by the bone conduction microphone 11 is sufficiently high and the mixing ratio is set so that the sound is essentially only from the bone conduction microphone 11.

[0174] In Figure 6, when the ambient sound level is below the level threshold and the signal-to-noise ratio is below the first threshold, output is made based on the sound collection results of the bone conduction microphone 11, when the signal-to-noise ratio is equal to or greater than the first threshold and equal to or less than the second threshold, output is made based on the sound collection results of the bone conduction microphone 11 and the air conduction microphone 10, and when the signal-to-noise ratio is greater than the second threshold, output is made based on the sound collection results of the air conduction microphone 10, but this is not limited to the above embodiment.

[0175] For example, when the ambient sound level is below the level threshold and the signal-to-noise ratio is below a first threshold, output may be performed based on the sound collection results of the bone-conduction microphone 11, and when the signal-to-noise ratio is greater than the first threshold, output may be performed based on the sound collection results of the bone-conduction microphone 11 and the air-conduction microphone 10. In addition, when the ambient sound level is below the level threshold and the signal-to-noise ratio is equal to or less than a second threshold, output may be performed based on the sound collection results of the bone-conduction microphone 11 and the air-conduction microphone 10, and when the signal-to-noise ratio is greater than the second threshold, output may be performed based on the sound collection results of the air-conduction microphone 10. [Industrial Applicability]

[0176] The microphone device and the communication device equipped with the microphone device according to the present disclosure are useful as a microphone device that generates sound based on air conduction sound and bone conduction sound, and a communication device equipped with a microphone device, which realizes reliable microphone sound pickup when there is an abnormality in the sound pickup of either the bone conduction microphone or the air conduction microphone, and a communication device equipped with a microphone device. [Explanation of symbols]

[0177] 1: Microphone device 2:Telephone device 3: Earphones 4: Signal processing device 5: Signal line 9: Exterior housing 9A: Case 9B: Cover 9C: Opening 10: Air conduction microphone 11: Bone conduction microphone 12: Air conduction receiver 13: Internal housing 13A:Aisle 13B:Cylinder part 15: Microphone Lover 16: Tip rubber 17: Resin material 18: Cable cap 31: Amplification section 32: Preprocessing section 33: Noise cancellation section 34: Mixing section 35: Signal setting section 37: Microphone output section 41: Wearing state detection unit 42: Decision section 43: Path setting section 45: Switch

Claims

1. A microphone device that picks up a sound signal including a user's voice and ambient sound and generates a signal to be output, A bone-conducted microphone for collecting the user's bone-conducted voice and bone-conducted ambient sound; An air conduction microphone that collects the air conduction voice and the air conduction ambient sound of the user; a signal processing device that generates a signal to be output based on the sound collection results by the bone conduction microphone and the air conduction microphone; The signal processing device includes: obtaining a level of the bone-conducted sound and a level of the air-conducted sound; A microphone device that determines a method for generating a signal to be output based on the sound pickup result of the air conduction microphone when a level difference between the bone conduction sound level and the air conduction sound level is equal to or greater than a predetermined difference threshold.

2. A microphone device that picks up a sound signal including a user's voice and ambient sound and generates a signal to be output, A bone-conducted microphone for collecting the user's bone-conducted voice and bone-conducted ambient sound; An air conduction microphone that collects the air conduction voice and the air conduction ambient sound of the user; a signal processing device that generates a signal to be output based on the sound collection results by the bone conduction microphone and the air conduction microphone; The signal processing device includes: obtaining a level of the bone-conducted sound and a level of the air-conducted sound; A microphone device that determines a method for generating a signal to be output based on the sound pickup result of the air conduction microphone after noise cancellation when the level difference between the bone conduction sound level and the air conduction sound level is greater than or equal to a predetermined difference threshold.

3. The signal processing device includes:

3. The microphone device according to claim 1, wherein when a level difference between the bone-conducted sound level and the air-conducted sound level is less than the difference threshold, a frequency characteristic is changed and a method for generating a signal to be output is determined.

4. A communication device equipped with a microphone device as described in claim 1 or claim 2.

Images