Microphone array and signal conversion device

Abstract

Provided is a microphone array for obtaining information which is the same as an ambisonics-format acoustic signal. This microphone array has two securing parts which are secured to the two ears of a user, and has microphones, two or more of which are held by each securing part. The microphone array is configured in a manner such that when the securing parts are secured to the two ears, the positions of the microphones arranged on one ear and the positions of the microphones arranged on the other ear are asymmetrical.

Description

[Technical field]

[0001] The present invention relates to a technique for obtaining information equivalent to an audio signal in Ambisonics format. [Background technology]

[0002] In recent years, in addition to speech recognition and sound source localization of speech, research into acoustic event detection has been actively conducted. This is based on the background that there is a demand for means that require recognition of surrounding sounds regardless of speech. For example, a method such as that shown in Non-Patent Document 1 has been proposed as a technology (SELD: sound event localization and detection) that simultaneously performs sound event detection and sound source localization. This is a method of detecting acoustic events from sounds recorded by an Ambisonics microphone. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Masahiro Yasuda, Yuma Koizumi, Shoichiro Saito, Hisashi Uematsu, Keisuke Imoto, "SOUND EVENT LOCALIZATION BASED ON SOUND INTENSITY VECTOR REFINED BY DNN-BASED DENOISING AND SOURCE SEPARATION," IEEE ICASSP 2020. Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are applications and environments in which the use of dedicated Ambisonics-format microphones is impractical.

[0005] "Binaural recording" is a highly practical means of collecting ambient sounds with wearable devices. However, binaural recording focuses on whether the recorded sounds sound natural when heard by humans, and is not necessarily suitable for acoustic processing such as acoustic event detection and sound source localization. For example, in binaural recording, one microphone is placed in each ear to faithfully reproduce the state in which a human hears a sound, and it does not necessarily obtain sufficient information for acoustic processing.

[0006] The present invention has been made in consideration of these points, and has an object to provide a microphone array that has a highly practical configuration and obtains information equivalent to an Ambisonics format acoustic signal, which has sufficient information for acoustic processing. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a microphone array for obtaining information equivalent to an Ambisonics format acoustic signal is provided, the microphone array having two fixed parts fixed to both ears of a user and at least two microphones held in each fixed part, and configured such that when the fixed parts are fixed to both ears, the position of the microphone arranged on one ear side and the position of the microphone arranged on the other ear side are asymmetrical. Effect of the Invention

[0008] This makes it possible to realize a microphone array that can obtain information equivalent to an Ambisonics format acoustic signal, which has sufficient information for acoustic processing, in a highly practical configuration. [Brief description of the drawings]

[0009] [Figure 1] Fig. 1A is a conceptual diagram illustrating the configuration of a microphone array system 1 according to a first embodiment. Fig. 1B is a block diagram illustrating the functional configuration of a signal conversion device in Fig. 1A. [Diagram 2]FIG. 2 is a conceptual diagram illustrating the configuration of the microphone array according to the first embodiment. [Diagram 3] 3A and 3B are conceptual diagrams illustrating the configuration of the microphone array according to the first embodiment. [Figure 4] FIG. 4 is a conceptual diagram illustrating the configuration of the microphone array according to the first embodiment. [Diagram 5] FIG. 5 is a conceptual diagram illustrating the configuration of the microphone array according to the first embodiment. [Figure 6] FIG. 6 is a conceptual diagram illustrating the directivity achieved by the microphone array of the first embodiment. [Figure 7] FIG. 7 is a conceptual diagram illustrating the directivity achieved by the microphone array of the first embodiment. [Figure 8] FIG. 8 is a conceptual diagram illustrating the configuration of a microphone array according to the second embodiment. [Figure 9] 9A and 9B are conceptual diagrams illustrating the configuration of a microphone array according to the second embodiment. [Figure 10] FIG. 10 is a block diagram illustrating a hardware configuration of the signal conversion device 13 according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [First embodiment] First, a first embodiment of the present invention will be described. As illustrated in FIG. 1A, the microphone array system 1 of the first embodiment includes a microphone array 11 and a signal conversion device 13.

[0011] <Microphone Array 11> As illustrated in Figures 1A, 2, 3A, 3B, 4 and 5, the microphone array 11 of this embodiment is intended to obtain information equivalent to an Ambisonics format acoustic signal, and has two fixed parts 11R, 11L fixed to both ears 110R, 110L of a user 100, and at least two microphones 11RF, 11RB, 11LF, 11LB held in each of the fixed parts 11R, 11L.

[0012] The microphones 11RF, 11RB, 11LF, and 11LB of this embodiment are, for example, non-directional (omnidirectional) microphones. The microphones 11RF and 11RB may be fixed to the fixed part 11R or may be incorporated into the fixed part 11R. Similarly, the microphones 11LF and 11LB may be fixed to the fixed part 11L or may be incorporated into the fixed part 11L.

[0013] The fixed part 11R is configured to be fixed (worn) to the right ear 110R (one ear) of the user 100. The microphones 11RF and 11RB are arranged at positions where, when the fixed part 11R is fixed to the right ear 110R, the sound observed by the microphones 11RF and 11RB is affected by the head of the user 100 but is not (not greatly) affected by the pinna. For example, when the fixed part 11R is fixed to the right ear 110R, the sound collecting ends of the microphones 11RF and 11RB are not arranged on the inner side of the pinna of the right ear 110R, but are arranged on the outer side of the pinna of the right ear 110R. Preferably, when the fixed part 11R is fixed to the right ear 110R, the sound collecting ends of the microphones 11RF and 11RB are configured to face outward of the user 100. For example, microphones 11RF and 11RB are provided on the outer side (one side) of fixed part 11R, the sound collecting ends of these microphones 11RF and 11RB face the outer side and protrude toward the outer side, and the other side (other side) of fixed part 11R is configured in a shape that can be attached to right ear 110R. For example, the other side of fixed part 11R is configured to be attached to the pinna or hole (ear canal) of right ear 110R.

[0014] The fixed part 11L is configured to be fixed (worn) to the left ear 110L (the other ear) of the user 100. The microphones 11LF, 11LB are arranged at positions where, when the fixed part 11L is fixed to the left ear 110L, the sound observed by the microphones 11LF, 11LB is affected by the head of the user 100 but is not (is not greatly) affected by the pinna. For example, when the fixed part 11L is fixed to the left ear 110L, the sound collecting ends of the microphones 11LF, 11LB are not arranged on the inner side of the pinna of the left ear 110L, but are arranged on the outer side of the pinna of the left ear 110L. Preferably, when the fixed part 11L is fixed to the left ear 110L, the sound collecting ends of the microphones 11LF, 11LB are configured to face outward of the user 100. For example, microphones 11LF, 11LB are provided on the outer side (one side) of the fixed part 11L, the sound collecting ends of these microphones 11LF, 11LB face the outer side and protrude toward the outer side, and the other side (the other side) of the fixed part 11L is configured in a shape that can be attached to the left ear 110L. For example, the other side of the fixed part 11L is configured to be attached to the auricle or hole of the left ear 110L.

[0015] 2, 3A, 3B, 4, and 5, when the user 100 wears the microphone array 11 and the fixing parts 11R and 11L are fixed (worn) to both ears 110R and 110L, respectively, the positions of the microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) and the positions of the microphones 11LF and 11LB arranged on the left ear 110L side (the other ear side) are configured to be asymmetric. For example, the positions of the microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) and the positions of the microphones 11LF and 11LB arranged on the left ear 110L side (the other ear side) are configured to be asymmetrical (not plane-symmetrical) with respect to a reference plane P1 (first reference plane) located between the right ear 110R side (one ear side) and the left ear 110L side (the other ear side).

[0016] As shown in FIG. 5, when the fixing parts 11R and 11L are fixed to the ears 110R and 110L, respectively, a straight line L passing through the two microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) is formed.RF-RB The (first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d1 (first rotation direction) centered on the axis L1 with respect to a reference plane P2 (second reference plane) including an axis L1 of a straight line passing through the right ear 110R (one ear) and the left ear 110L (the other ear). The reference plane P2 (second reference plane) in this case includes, for example, the center line of the user 100 (for example, a line connecting the top and bottom of the head). In addition, the straight line L passing through the two microphones 11LF, 11LB arranged on the left ear 110L side (the other ear side) is inclined at an angle θ1° (first angle) in a rotation direction d1 (first rotation direction) centered on the axis L1. LF-LB The (second straight line) is inclined at an angle θ2° (second angle) in a rotation direction d2 (second rotation direction) centered on the axis L1 with respect to the reference plane P2 (second reference plane). Here, the rotation direction d2 (second rotation direction) is the opposite rotation direction of the rotation direction d1 (first rotation direction), and the angle θ2° (second angle) is substantially the same as the angle θ1° (first angle). Note that θ1 and θ2 are positive real numbers that satisfy 0<θ1, θ2<90. In this embodiment, the rotation direction d1 is a left rotation direction and the rotation direction d2 is a right rotation direction, but the rotation direction d1 may be a right rotation direction and the rotation direction d2 may be a left rotation direction. For example, θ1 and θ2 are 45(°) or close to 45(°), but this does not limit the present invention, and θ1 does not have to be the same value as θ2 or close to θ2, and θ1 and θ2 may be other than 45(°). When θ1 and θ2 are 45° or close to 45°, the positional relationship between the microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) is perpendicular or substantially perpendicular to the positional relationship between the microphones 11LF and 11LB arranged on the left ear 110L side (the other ear side). RF-RB (First line) and line L LF-LB (the second straight line) is perpendicular or substantially perpendicular to the first straight line.

[0017] Hereinafter, a description will be given using an orthogonal coordinate system with an x-axis, a y-axis, and a z-axis. For convenience, when the fixing parts 11R, 11L are fixed to both ears 110R, 110L, respectively, the axis parallel to the axis L1 of the straight line passing through the right ear 110R and the left ear 110L is defined as the y-axis, and the direction from the right ear 110R to the left ear 110L is defined as the positive direction of the y-axis. The x-axis and z-axis are orthogonal to the y-axis, and the axis parallel to the front-to-back direction of the user 100 is defined as the x-axis, the forward direction of the user 100 is defined as the positive direction of the x-axis, the axis parallel to the center line of the user 100 (the axis parallel to the up-down direction of the user 100) is defined as the z-axis, and the upward direction of the user 100 is defined as the positive direction of the z-axis.

[0018] As illustrated in FIG. 2, FIG. 3A, FIG. 3B, FIG. 4, and FIG. 5, the fixed parts 11R and 11L are plate-shaped members (for example, disk-shaped members), and when the fixed parts 11R and 11L are fixed to both ears 110R and 110L, respectively, the plate surfaces of the fixed parts 11R and 11L are arranged along the xz plane. At this time, the microphones 11RF and 11RB held by the fixed part 11R are arranged on the outer side (negative direction of the y axis) of the fixed part 11R, and the microphones 11LF and 11LB held by the fixed part 11L are arranged on the outer side (positive direction of the y axis) of the fixed part 11L. As illustrated in FIG. 5, the direction from the axis L1 of the microphone 11RB held by the fixed part 11R arranged along the xz plane is a direction rotated by (90-θ1)° in the rotation direction d2 from the reference plane P3 (a plane including the axis L1 and parallel to the xy plane). Also, the direction from the axis L1 of the microphone 11RB held by the fixed part 11R is a direction rotated by (90-θ1)° in the rotation direction d2 from the reference plane P3. Similarly, the direction from the axis L1 of the microphone 11LB held by the fixed part 11L arranged along the xz plane is a direction rotated by (90-θ2)° in the rotation direction d1 from the reference plane P3. Also, the direction from the axis L1 of the microphone 11LB held by the fixed part 11L is a direction rotated by (90-θ2)° in the rotation direction d1 from the reference plane P3.

[0019] The positions of the microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) and the positions of the microphones 11LF and 11LB arranged on the left ear 110L side (the other ear side) are asymmetric. For example, the positions of the microphones 11RF and 11RB and the positions of the microphones 11LF and 11LB are asymmetric with respect to a reference plane P1 (first reference plane) parallel to the yz plane located between the right ear 110R side (one ear side) and the left ear 110L side (the other ear side).

[0020] As shown in FIG. 5, a straight line L passing through the microphones 11RF and 11RB is RF-RB The (first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d1 (first rotation direction) centered on the axis L1 with respect to a reference plane P2 (second reference plane) that is parallel to the yz plane and includes the axis L1 and the center line of the user 100. LF-LB The (second straight line) is inclined at an angle θ2° (second angle) in a rotation direction d2 (second rotation direction) centered on the axis L1 with respect to the reference plane P2 (second reference plane). Here, the rotation direction d2 (second rotation direction) is the opposite rotation direction to the rotation direction d1 (first rotation direction), and the angle θ2° (second angle) is substantially the same as the angle θ1° (first angle). In other words, the positions (images) of the microphones 11RF and 11RB projected onto the xz plane are different from the positions of the microphones 11LF and 11LB projected onto the xz plane, and these are line-symmetric with respect to a straight line parallel to the z-axis passing through the axis L1.

[0021] Next, the purpose of arranging the microphones 11LF, 11LB, 11RF, and 11RB in this manner will be described with reference to Figs. 6 and 7. F ,L B ,R F ,R B That is, L F is an acoustic signal obtained by the microphone 11RF arranged on the positive x-axis side (first direction side, forward direction of the user 100) out of the two microphones 11RF and 11RB arranged on the right ear 110R side (one ear side), and L Bis an acoustic signal obtained by the microphone 11RB arranged on the negative side of the x-axis (the second direction side opposite to the first direction side) of the two microphones 11RF and 11RB, and R F is an acoustic signal obtained by the microphone 11LF arranged on the positive x-axis side (first direction side) of the two microphones 11LF and 11LB arranged on the left ear 110L side (the other ear side), and R B is an acoustic signal obtained by the microphone 11LB, which is disposed on the negative x-axis side (second direction side) of the two microphones 11LF and 11LB.

[0022] First, when the microphones 11RF, 11RB, 11LF, and 11LB are omnidirectional microphones, the acoustic signal R obtained by the microphones 11RF and 11RB on the right ear 110R side is F ,R B Difference R F -R B , and the acoustic signal L obtained by the microphones 11LF and 11LB on the left ear 110L side F ,L B Difference L F -L B can be regarded as the acoustic signals observed by bidirectional microphones (Fig. 6). Furthermore, the difference R F -R B and the difference L F -L B The combination of can be regarded as an acoustic signal observed by a microphone having directionality in the x-axis and z-axis directions.

[0023] In addition, the acoustic signal R obtained by the microphones 11RF and 11RB on the right ear 110R side F ,R B Sum of R F +R B can be regarded as an acoustic signal observed by a microphone having gentle directivity in the positive direction of the y-axis, while on the negative side of the y-axis, the higher the frequency, the stronger the obstruction by the head of the user 100 becomes, and the lower the sensitivity becomes. Similarly, the acoustic signal L obtained by the microphones 11LF and 11LB on the left ear 110L side F ,L B Sum of LF +L B can be regarded as an acoustic signal observed by a microphone with gentle directivity in the negative direction of the y-axis, while on the positive side of the y-axis, the higher the frequency, the stronger the obstruction by the head of the user 100 becomes, and the lower the sensitivity becomes. F +R B and Japanese F +L B The difference between the two can be regarded as an acoustic signal observed by a microphone having a pseudo y-axis direction.

[0024] Based on the above, for example, using the following equations (1)-(4), the acoustic signal R F ,R B ,L F ,L B It is possible to generate pseudo 1st order Ambisonics B format signals (X,Y,Z,W) from X=L F -L B +R F -R B (1) Y=L F -R B +L B -R F (2) Z=L F -L B +R B -R F (3) W=L F +L B +R B +R F (4) Here, X represents the directional component in the x-axis direction, Y represents the directional component in the y-axis direction, Z represents the directional component in the z-axis direction, and W represents the omnidirectional component. In reality, the observation points at both ears are far apart, and there is an effect of the head of the user 100 as a hard sphere, so the results of equations (1)-(4) do not strictly match the first-order Ambisonics B-format signal (X, Y, Z, W). However, it can be seen that acoustic information can be obtained from above, below, left, right, front, and back of the user 100 by using microphones 11LF, 11LB and microphones 11RF, 11RB arranged asymmetrically.

[0025] <Signal conversion device 13> 1B, the signal conversion device 13 of this embodiment includes an input unit 131, a storage unit 132, a conversion unit 133, and an output unit 134. F ,R B ,L F ,L B is input to the input unit 131 and stored in the storage unit 132. The conversion unit 133 converts the acoustic signal R F ,R B ,L F ,L B and using these, X=L according to equations (1)-(4). F -L B +R F -R B ,Y=L F -R B +L B -R F ,Z=L F -L B +R B -R F ,W=L F +L B +R B +R F The output unit 134 outputs the obtained signals (X, Y, Z, W).

[0026] Alternatively, a model may be obtained that eliminates or reduces the deviation between the signal (X, Y, Z, W) obtained according to the formulas (1)-(4) and the ideal first-order Ambisonics B-format signal by using the impulse responses from each direction (known direction) obtained by the microphones 11RF, 11RB, 11LF, and 11LB. In this case, the conversion unit 133 may apply the signals (X, Y, Z, W) to the model to obtain and output the signals (X', Y', Z', W') that eliminate or reduce the deviation. In this case, the output unit 134 outputs the signals (X', Y', Z', W').

[0027] <Features of the first embodiment> The microphone array 11 of this embodiment has two fixed parts 11R, 11L fixed to both ears 110R, 110L of the user 100, and at least two microphones 11RF, 11RB, 11LF, 11LB held on each of the fixed parts 11R, 11L, and is configured so that when the fixed parts 11R, 11L are fixed to both ears 110R, 110L, respectively, the positions of the microphones 11RF, 11RB arranged on the right ear 110R (one ear side) and the positions of the microphones 11LF, 11LB arranged on the left ear 110L (the other ear side) are asymmetric. As a result, the acoustic signal R obtained by the microphones 11RF, 11RB, 11LF, 11LB is F ,R B ,L F ,L B By using these, it is possible to perform acoustic processing such as detecting acoustic events in the surrounding environment of the user 100, localizing the sound source, and detecting directional information based on machine learning or the like.

[0028] Furthermore, the microphones 11RF, 11RB, 11LF, and 11LB are attached to both ears 110R and 110L of the user 100 via the fixing parts 11R and 11L, and are compatible with wearable devices and the like, making them highly practical.

[0029] Furthermore, the microphones 11RF, 11RB, 11LF, and 11LB of this embodiment are placed in positions where the sounds observed by them are affected by the head of the user 100 but not by the pinna. This makes it possible to suppress individual differences in the acoustic signals obtained by the microphones 11RF, 11RB, 11LF, and 11LB due to the physical characteristics of the user 100.

[0030] [Modification of the first embodiment] As shown in FIG. 5, in this embodiment, when the fixing parts 11R and 11L are fixed to the ears 110R and 110L, respectively, a straight line L passing through the microphones 11RF and 11RB is aligned with a reference plane P2 (second reference plane) including the center line of the user 100, that is, a reference plane P2 (second reference plane) parallel to the z-axis. RF-RB(first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d1 (first rotation direction) centered on an axis L1, and a straight line L passing through the microphones 11LF and 11LB LF-LB In the example shown, the line L (second straight line) passing through the microphones 11RF and 11RB is inclined at an angle θ2° (second angle) in a rotation direction d2 (second rotation direction) centered on the axis L1 with respect to the reference plane P2 (second reference plane). However, this does not limit the present invention. RF-RB (first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d1 (first rotation direction) centered on an axis L1, and a straight line L passing through the microphones 11LF and 11LB LF-LB The (second straight line) may be inclined at an angle θ2° (second angle) in a rotation direction d2 (second rotation direction) about the axis L1 with respect to a reference plane P2' (second reference plane).

[0031] [Second embodiment] Next, a second embodiment of the present invention will be described. The second embodiment is a modification of the first embodiment, and uses both a glasses-type device and a microphone boom for microphone placement. The following description will focus on the differences from the first embodiment, and the same reference numbers will be used to simplify the description of the matters already described.

[0032] In the microphone array system of the second embodiment, the microphone array 11 of the microphone array system 1 of the first embodiment is replaced with a microphone array 21. The configuration of the microphone array 21 of this embodiment will be described below.

[0033] <Microphone Array 11> As illustrated in Fig. 8 and Fig. 9, the microphone array 21 of this embodiment has a fixing part 21R (first fixing part) fixed to the right ear 110R (one ear) of the user 100, a fixing part 21L (second fixing part) fixed to the left ear 110L (the other ear) of the user 100, a glasses-type device 22 held by at least both ears 110R, 110L of the user 100, and a plurality of microphones 11RF, 11RB, 11LF, and 11LB. The microphone 11RB (at least one of the microphones) is held by the fixing part 21R (first fixing part), the microphones 11LF, 11LB (at least two of the microphones) are held by the fixing part 21L (second fixing part), and the microphone 11RF (at least one of the microphones) is held by the glasses-type device 22. In the example of this embodiment, the microphone 11RF is held by a frame 22FR on the right side of the glasses-type device 22. For example, the microphone 11RF is held at an end (the end on the side where the lens is attached) of the frame 22FR on the right side of the eyeglass-type device 22. In the example of this embodiment, the fixing part 21L (second fixing part) has a base 21LA fixed to the left ear 110L (the other ear) and a rod-shaped microphone boom 21LB (extension part) extending from the base 21LA. The microphone 11LB (at least one of the microphones) is held at the base 21LA, and the microphone 11LF (at least one of the microphones) is held at the microphone boom 21LB (extension part). For example, the microphone 11LB is held at the end on the tip side of the microphone boom 21LB (extension part).

[0034] The fixed part 21R (second fixed part) is configured to be fixed (worn) to the right ear 110R (one ear) of the user 100. As in the first embodiment, the microphone 11RB is disposed at a position where, when the fixed part 21R is fixed to the right ear 110R, the sound observed by the microphone 11RB is affected by the head of the user 100 but is not (not greatly) affected by the pinna. For example, when the fixed part 21R is fixed to the right ear 110R, the sound collecting end of the microphone 11RB is not disposed on the inner side of the pinna of the right ear 110R, but is disposed on the outer side of the pinna of the right ear 110R. Preferably, when the fixed part 21R is fixed to the right ear 110R, the sound collecting end of the microphone 11RB is configured to face outward of the user 100. For example, the microphone 11RB is provided on the outer side (one side) of the fixed part 21R, the sound collecting ends of the microphones 11RB face the outer side and protrude toward the outer side, and the other side (other side) of the fixed part 21R is configured to have a shape that can be attached to the right ear 110R. For example, the other side of the fixed part 21R is configured to be attached to the pinna or hole (ear canal) of the right ear 110R.

[0035] The base 21LA of the fixing part 21L is configured to be fixed (worn) to the left ear 110L (the other ear) of the user 100. As in the first embodiment, the microphone 11LF is disposed at a position where, when the base 21LA is fixed to the left ear 110L, the sound observed by the microphone 11LB is affected by the head of the user 100 but is not (is not significantly) affected by the pinna. For example, when the base 21LA is fixed to the left ear 110L, the sound collecting end of the microphone 11LB is not disposed on the inner side of the pinna of the left ear 110L, but is disposed on the outer side of the pinna of the left ear 110L. Preferably, when the base 21LA is fixed to the left ear 110L, the sound collecting end of the microphone 11LB is configured to face outward of the user 100. For example, the microphone 11LB is provided on the outer side (one side) of the base 21LA, the sound collecting ends of the microphones 11LB face the outer side and protrude toward the outer side, and the other side (the other side) of the base 21LA is configured to have a shape that can be attached to the left ear 110L. For example, the other side of the base 21LA is configured to be attached to the auricle or hole of the left ear 110L.

[0036] Since the microphone 11RF is held by the eyeglass-type device 22, the sound observed by the microphone 11RF is affected by the head of the user 100 but is not (not significantly) affected by the pinna. Similarly, since the microphone 11LF is held by the microphone boom (extension 21LB), the sound observed by the microphone 11LF is affected by the head of the user 100 but is not (not significantly) affected by the pinna.

[0037] As illustrated in FIGS. 8, 9A, and 9B, when the user 100 wears the microphone array 21, the fixed part 21R (first fixed part) is fixed to the right ear 110R (one ear), the fixed part 21L (second fixed part) is fixed to the left ear 110L (the other ear), and the eyeglass-type device 22 is held by both ears 110R and 110L (and the nose), the microphones respectively held by the fixed part 21R (first fixed part) and the eyeglass-type device 22 are The microphones 11RF and 11RB are arranged on the right ear 110R (one ear) side, and the microphones 11LF and 11LB held by the fixing part 21L (second fixing part) are arranged on the left ear 110L (other ear) side, so that the positions of the microphones 11RF and 11RB arranged on the right ear 110R (one ear) side and the positions of the microphones 11LF and 11LB arranged on the left ear 110L (other ear) side are asymmetric. For example, the positions of the microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) and the positions of the microphones 11LF and 11LB arranged on the left ear 110L side (other ear side) are asymmetric with respect to a reference plane P1 (first reference plane) located between the right ear 110R side (one ear side) and the left ear 110L side (other ear side).

[0038] As illustrated in FIGS. 8, 9A, and 9B, when the fixing part 21R (first fixing part) is fixed to the right ear 110R (one ear) and the fixing part 21L (second fixing part) is fixed to the left ear 110L (the other ear) and the eyeglass-type device 22 is held by both ears 110R and 110L (and the nose), a straight line L passing through the two microphones 11RF and 11RB arranged on the right ear 110R side (one ear side) forms a straight line L. RF-RBThe (first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d3 (first rotation direction) centered on the axis L1 with respect to a reference plane P3 (second reference plane) including an axis L1 of a straight line passing through the right ear 110R (one ear) and the left ear 110L (the other ear). The reference plane P3 (second reference plane) in this case includes, for example, a straight line in the front-to-back direction of the user 100 (a straight line parallel to the x-axis), and is parallel to, for example, the xy plane. In addition, the straight line L passing through the two microphones 11LF, 11LB arranged on the left ear 110L side (the other ear side) is inclined at an angle θ1° (first angle) in a rotation direction d3 (first rotation direction) centered on the axis L1. LF-LB The (second straight line) is inclined at an angle θ2° (second angle) in a rotation direction d4 (second rotation direction) centered on the axis L1 with respect to the reference plane P3 (second reference plane). Here, the rotation direction d4 (second rotation direction) is the opposite rotation direction to the rotation direction d3 (first rotation direction), and the angle θ2° (second angle) is substantially the same as the angle θ1° (first angle).

[0039] Acoustic signal L picked up by microphones 11LF, 11LB, 11RF, and 11RB F ,L B ,R F ,R B is sent to the signal conversion device 13, and as described in the first embodiment, the signal conversion device 13 converts X=L according to the equations (1)-(4). F -L B +R F -R B ,Y=L F -R B +L B -R F ,Z=L F -L B +R B -R F ,W=L F +L B +R B +R F Alternatively, the signal conversion device 13 applies the signals (X, Y, Z, W) to the above-mentioned model to obtain and output signals (X', Y', Z', W') that eliminate or reduce the deviation from an ideal first-order Ambisonics B format signal.

[0040] <Features of the second embodiment> The microphone array 21 of this embodiment has a fixed part 21R (first fixed part) fixed to the right ear 110R (one ear) of the user 100, a fixed part 21L (second fixed part) fixed to the left ear 110L (the other ear) of the user 100, a glasses-type device 22 held by at least both ears 110R and 110L of the user 100, and a plurality of microphones 11RF, 11RB, 11LF, and 11LB. The microphone 11RB (at least one of the microphones) is held by the fixed part 21R (first fixed part), the microphones 11LF and 11LB (at least two of the microphones) are held by the fixed part 21L (second fixed part), and the microphone 11RF (at least one of the microphones) is held by the glasses-type device 22. When the fixed part 21R (first fixed part) is fixed to the right ear 110R (one ear), the fixed part 21L (second fixed part) is fixed to the left ear 110L (the other ear), and the eyeglass-type device 22 is held by both ears 110R, 110L (and nose), the microphones 11RF, 11RB held by the fixed part 21R (first fixed part) and the eyeglass-type device 22 are arranged on the right ear 110R (one ear) side, and the microphones 11LF, 11LB held by the fixed part 21L (second fixed part) are arranged on the left ear 110L (the other ear), so that the positions of the microphones 11RF, 11RB arranged on the right ear 110R (one ear) side and the positions of the microphones 11LF, 11LB arranged on the left ear 110L (the other ear) side are asymmetric. As a result, the acoustic signal R obtained by the microphones 11RF, 11RB, 11LF, 11LB is F ,R B ,L F ,L B By using these, it is possible to perform acoustic processing such as detecting acoustic events in the surrounding environment of the user 100, localizing the sound source, and detecting directional information based on machine learning or the like.

[0041] Furthermore, the microphones 11RF, 11RB, 11LF, and 11LB are attached to the user 100, and are compatible with wearable devices and the like, making them highly practical.

[0042] Furthermore, the microphones 11RF, 11RB, 11LF, and 11LB of this embodiment are placed in positions where the sounds observed by them are affected by the head of the user 100 but not by the pinna. This makes it possible to suppress individual differences in the acoustic signals obtained by the microphones 11RF, 11RB, 11LF, and 11LB due to the physical characteristics of the user 100.

[0043] In this embodiment, the glasses-type device 22 holds the microphone 11RF, the fixed part 21R holds the microphone 11RB, the base part 21LA holds the microphone 11LB, and the microphone boom 21LB holds the microphone 11LB. Therefore, the distance between the microphone 11RF and the microphone 11RB and the distance between the microphone 11LF and the microphone 11LB can be longer than in the configuration of the first embodiment. This configuration is suitable for localization on the low frequency side. In addition, compared to the configuration of the first embodiment, the microphone 11RF and the microphone 11LF on the front side are arranged further forward than the two ears 110R and 110L where the microphone 11RB and the microphone 11LB on the rear side are arranged, so that it is easier to capture the difference in sound between the front and rear of the user 100, and it is easier to judge the front and rear.

[0044] [Modification of the second embodiment] As in the modified example of the first embodiment, if θ1 is equal to or close to θ2, a straight line L passing through the microphones 11RF and 11RB is aligned with a reference plane P3′ (second reference plane) obtained by rotating a reference plane P3 including the axis L1 around the axis L1. RF-RB (first straight line) is inclined at an angle θ1° (first angle) in a rotation direction d3 (first rotation direction) centered on an axis L1, and a straight line L passing through the microphones 11LF and 11LB LF-LB The (second straight line) may be inclined at an angle θ2° (second angle) in a rotation direction d4 (second rotation direction) about the axis L1 with respect to a reference plane P3' (second reference plane).

[0045] In the second embodiment, an example in which the microphone 11LB is held at the end of the tip side of the microphone boom 21LB (extension part) has been shown. However, this does not limit the present invention, and the microphone 11LB may be held at the base side (base part 21LA side) of the microphone boom 21LB, or may be held in the middle of the microphone boom 21LB. Also, in the second embodiment, the fixed part 21L (second fixed part) has a base part 21LA fixed to the left ear 110L (the other ear) and a rod-shaped microphone boom 21LB (extension part) extending from the base part 21LA, and the microphone 11LB (at least one of the microphones) is held by the base part 21LA, and the microphone 11LF (at least one of the microphones) is held by the microphone boom 21LB (extension part). However, like the fixed part 11L in the first embodiment, the microphone 11LB and the microphone boom 21LB may be held by the base part 21LA. In this case, the microphone boom 21LB may be omitted.

[0046] In the second embodiment, an example has been shown in which the microphone 11RF is held on the frame 22FR on the right side of the eyeglass-type device 22. The microphone 11RF may be held anywhere on the frame 22FR, and may be held on an end of the frame 22FR (the end on the side where the lens is attached), may be held on the other end of the frame 22FR (the end on the side held by the ear 110R), or may be held in the middle of the frame 22FR. The microphone 11RF may also be held in other parts, such as near the lens of the eyeglass-type device 22.

[0047] [Hardware configuration] The signal conversion device 13 in each embodiment is a device configured by a general-purpose or dedicated computer having a processor (hardware processor) such as a CPU (central processing unit) and memories such as a RAM (random-access memory) and a ROM (read-only memory) executing a predetermined program. That is, the signal conversion device 13 in each embodiment has, for example, a processing circuit configured to implement each unit. This computer may have one processor and memory, or may have multiple processors and memories. This program may be installed in the computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units may be configured using electronic circuits that realize processing functions independently, rather than electronic circuits that realize functional configurations by reading programs like a CPU. In addition, an electronic circuit that configures one device may include multiple CPUs.

[0048] FIG. 6 is a block diagram illustrating a hardware configuration of the signal conversion device 13 in each embodiment. As illustrated in FIG. 6, the signal conversion device 13 in this example has a CPU (Central Processing Unit) 10a, an input unit 10b, an output unit 10c, a RAM (Random Access Memory) 10d, a ROM (Read Only Memory) 10e, an auxiliary storage device 10f, and a bus 10g. The CPU 10a in this example has a control unit 10aa, a calculation unit 10ab, and a register 10ac, and executes various calculation processes according to various programs loaded into the register 10ac. The input unit 10b is an input terminal to which data is input, a keyboard, a mouse, a touch panel, etc. The output unit 10c is an output terminal to which data is output, a display, a LAN card controlled by the CPU 10a that has loaded a predetermined program, etc. The RAM 10d is a static random access memory (SRAM), a dynamic random access memory (DRAM), or the like, and has a program area 10da in which a predetermined program is stored, and a data area 10db in which various data is stored. The auxiliary storage device 10f is, for example, a hard disk, a magneto-optical disc (MO), a semiconductor memory, or the like, and has a program area 10fa in which a predetermined program is stored, and a data area 10fb in which various data is stored. The bus 10g connects the CPU 10a, the input unit 10b, the output unit 10c, the RAM 10d, the ROM 10e, and the auxiliary storage device 10f so that information can be exchanged. The CPU 10a writes the program stored in the program area 10fa of the auxiliary storage device 10f to the program area 10da of the RAM 10d according to the loaded OS (Operating System) program. Similarly, the CPU 10a writes various data stored in the data area 10fb of the auxiliary storage device 10f to the data area 10db of the RAM 10d. The addresses on the RAM 10d where these programs and data are written are stored in the register 10ac of the CPU 10a.The control unit 10aa of the CPU 10a sequentially reads out these addresses stored in the register 10ac, reads out programs and data from the areas on the RAM 10d indicated by the read addresses, sequentially causes the calculation unit 10ab to execute the calculations indicated by the programs, and stores the results of the calculations in the register 10ac. With this configuration, the functional configuration of the signal conversion device 13 is realized.

[0049] The above-mentioned program can be recorded in a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

[0050] The distribution of this program is performed, for example, by selling, transferring, lending, etc., a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network. As described above, a computer that executes such a program, for example, first temporarily stores in its own storage device the program recorded in the portable recording medium or the program transferred from the server computer. Then, when executing the process, the computer reads the program stored in its own storage device and executes the process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute the process according to the program, and further, each time the program is transferred from the server computer to the computer, the computer may execute the process according to the received program. Furthermore, the server computer may not transfer the program to the computer, and the above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by issuing an execution instruction and obtaining the result. In addition, the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (such as data that is not a direct instruction to a computer but has the nature of defining computer processing).

[0051] In each embodiment, the present device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0052] The present invention is not limited to the above-mentioned embodiment. For example, in the above-mentioned embodiment, microphones are arranged near both ears 110R, 110L of the user 100, near the tip of the microphone boom 21LB, and on the eyeglass-type device 22. However, it is sufficient if another microphone included in the microphone array is installed at a position where a sound that is difficult to observe with a certain microphone included in the microphone array can be easily observed. For example, the microphone may be attached to another part of the user 100, such as the hair or nose. Also, the "one ear" may be the left ear, and the "other ear" may be the right ear.

[0053] The microphone array may include five or more microphones. Conversely, any one of the microphones 11RF, 11RB, 11LF, and 11LB included in the microphone array described above may be omitted. That is, when the microphone array is worn by the user 100, two microphones may be arranged on one ear side of the user 100, and one microphone may be arranged on the other ear side. In addition, at least some of the microphones included in the microphone array may be microphones having directionality such as unidirectional or bidirectional.

[0054] In the above-mentioned embodiment, the microphone arrays 11 and 21 are attached to the head of the user 100. However, a microphone array of a similar configuration may be attached to an object other than a human (a three-dimensional object having acoustic insulation). That is, the microphone array may have a plurality of attachment parts attached to a three-dimensional object having acoustic insulation and a plurality of microphones held by the attachment parts, and when the attachment parts are attached to the three-dimensional object, at least two of the microphones are arranged at a first attachment position on one side of the three-dimensional object, and at least two of the microphones are arranged at a second attachment position on the other side of the three-dimensional object, so that the positions of the microphones arranged on one side of the three-dimensional object and the positions of the microphones arranged on the other side of the three-dimensional object are asymmetric. For example, the above-mentioned microphone array may be attached to an object to which an existing Ambisonics microphone cannot be attached, such as an animal such as a dog, a drone, or a robot. In addition, the above-mentioned microphone array may be attached to a drone or a robot whose housing design cannot be changed.

[0055] In addition, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device that executes the processes or on the need of the device. Needless to say, other appropriate modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0056] 11,21 Microphone Array 13 Signal conversion device 133 Conversion section 133

Claims

1. A first fixing part that is fixed to one ear of a user; A second fixing portion fixed to the other ear of the user; A glasses-type device held by at least both ears of the user; A plurality of microphones; At least one of the microphones is held by the first fixed portion, at least two of the microphones are held by the second fixed portion, and at least one of the microphones is held by the glasses-type device; A microphone array configured such that, when the first fixed part is fixed to the one ear and the second fixed part is fixed to the other ear and the eyeglass-type device is held by both ears, the microphones held by the first fixed part and the eyeglass-type device are positioned on the one ear side and the microphone held by the second fixed part is positioned on the other ear side, and the position of the microphone positioned on the one ear side and the position of the microphone positioned on the other ear side are asymmetrical.

2. 2. The microphone array of claim 1, The second fixing portion is a base portion fixed to the other ear; an extension extending from the base, At least one of the microphones is supported on the base and at least one of the microphones is supported on the extension. Microphone array.

3. 3. The microphone array of claim 1 or 2, A microphone array configured such that the position of the microphone arranged on the one ear side and the position of the microphone arranged on the other ear side are asymmetrical with respect to a first reference plane located between the one ear and the other ear.

4. 4. The microphone array of claim 1, further comprising: a first straight line passing through the two microphones arranged on the side of the one ear is inclined at a first angle in a first rotation direction about an axis with respect to a second reference plane including an axis passing through the one ear and the other ear, a second straight line passing through the two microphones arranged on the other ear side is inclined at a second angle in a second rotation direction about the axis with respect to the second reference plane, the second rotation direction is a reverse rotation direction of the first rotation direction, The second angle is substantially the same as the first angle.

5. 5. The microphone array of claims 1 to 4, A microphone array, wherein the microphones are positioned in a position where sound observed by the microphones is influenced by the user's head but not by the user's pinna.

6. The microphone array according to any one of claims 1 to 5, wherein the acoustic signal L obtained by the microphone arranged on the first direction side of the two microphones arranged on the one ear side of the microphone array according to any one of claims 1 to 5 is F and an acoustic signal L obtained by a microphone arranged in a second direction opposite to the first direction. B , and an acoustic signal R obtained by the microphone arranged on the first direction side among the two microphones arranged on the other ear side. F and an acoustic signal R obtained by a microphone arranged on the second direction side. B Using X=L F -L B +R F -R B , Y=L F -R B +L B -R F , Z=L F -L B +R B -R F , W=L F +L B +R B +R F A signal conversion device having a conversion unit that calculates and outputs the above.

Images