Communication device and signal generation method

Abstract

The present invention improves the quality of signals of a virtual stereo microphone. This communication device comprises: a reception circuit that receives encoded information obtained by encoding signals of a stereo microphone; and a generation circuit that generates signals of a virtual stereo microphone configured from virtual microphones arranged on both sides of the stereo microphone through an extrapolation process performed on the phase spectrum of a decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference of signals between microphones constituting the stereo microphone is 2π or less.

Description

Communication device and signal generation method

[0001] This disclosure relates to a communication device and a signal generation method.

[0002] For example, there is a method that uses extrapolation to extend the distance between microphones to the distance between both ears and generates a binaural signal using a virtual microphone (hereinafter also referred to as "virtual microphone") (see, for example, Non-Patent Document 1).

[0003] Ryohei Jinzai et al., "Virtual Extension of Microphone Spacing by Extrapolation of Virtual Microphones," *Onkoron* 1-1-21, pp. 149-152, September 2018.

[0004] There is room for further investigation into methods for generating binaural signals using virtual microphones.

[0005] Non-limiting embodiments of this disclosure contribute to the provision of a communication terminal capable of improving the quality of binaural signals using a virtual microphone, and a signal generation method.

[0006] A communication device according to one embodiment of the present disclosure comprises a receiving circuit that receives encoded information obtained by encoding the signal of a stereo microphone, and a generating circuit that generates a signal of a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of a decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π.

[0007] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0008] According to one embodiment of this disclosure, the quality of binaural signals using a virtual microphone can be improved.

[0009] Further advantages and effects of one embodiment of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features.

[0010] Diagram illustrating the principle of generating a virtual microphone Diagram illustrating the principle of generating a pseudo-binaural signal Block diagram showing an example configuration of a communication terminal Flowchart of the pseudo-binaural signal generation process Diagram illustrating the principle of determining the azimuth angle from the inter-channel phase difference and the distance between microphones Block diagram showing an example of a transmitting terminal Block diagram showing an example of a receiving terminal Block diagram showing an example configuration of the inter-channel phase difference changing unit Block diagram showing an example configuration of the inter-channel amplitude difference changing unit Block diagram showing an example of a transmitting terminal Block diagram showing an example of a receiving terminal

[0011] Embodiments of this disclosure will be described in detail below with reference to the drawings.

[0012] The method described in Non-Patent Document 1 applies an approximation (for example, a plane wave assumption approximation) that assumes (or is assumed to be) that the sound source is sufficiently far away. Therefore, when applying this method to a nearby sound source (for example, a spherical sound wave), the error due to the plane wave assumption approximation tends to be large.

[0013] In contrast, in one non-limiting embodiment of the present disclosure, for example, by performing virtual microphone extrapolation (e.g., arrangement of virtual microphones) on both sides of the actual microphone (hereinafter also referred to as "real microphone") instead of on one side of the actual microphone, the communication device (e.g., also referred to as a communication terminal, binaural communication terminal, or binaural audio communication device) becomes capable of generating virtual binaural microphone signals having a binaural phase difference. This reduces errors due to the approximation of the plane wave assumption (e.g., plane wave sound waves), even in the case of a near-field sound source (e.g., spherical wave sound waves).

[0014] Figure 1 shows an example of a method for generating a virtual microphone as described in Non-Patent Document 1.

[0015] In Figure 1, "x1" and "x2" represent real microphones, and "v" represents a virtual microphone. In the example in Figure 1, if the distance between x1 and x2 (also called the "microphone distance") is 1, then the distance between x1 and v is α. The value of α represents the degree or direction of extrapolation and will be called the "extrapolation coefficient" below.

[0016] The observed signal v(ω,t,α) in the virtual microphone shown in Figure 1 is expressed by the following equation (where ω is each frequency, t is time, and α is the extrapolation coefficient): v(ω, t, α) = A v food(jφ v ) (1)

[0017] As shown in Figure 1, the virtual microphone v generated by extrapolation (or interpolation) exists outside the real microphones x1-x2. For example, if the virtual microphone v is outside x2, then α > 1, and if the virtual microphone v is outside x1, then α < 0.

[0018] Here, A v =A1 (if α>1) or A v =A2 (when α<0), φ v =φ1+α(φ2-φ1)=(1-α)φ1+αφ2. Here, A1 and φ1 represent the amplitude and phase of the observed signal at microphone x1, respectively, and A2 and φ2 represent the amplitude and phase of the observed signal at microphone x2, respectively.

[0019] Figure 2 shows an example of a method in which virtual microphones are placed on both sides of a real microphone (for example, symmetrically), rather than the method shown in Figure 1 where the virtual microphone is placed on one side of the real microphone. In Figure 2, "L" and "R" represent the real microphones, respectively, and "Lv" and "Rv" represent the virtual microphones, respectively.

[0020] For example, when the sound source is located far enough away and the sound wave can be regarded as a plane wave, there is no essential difference regardless of which virtual microphone arrangement in FIGS. 1 and 2 is used. On the other hand, when the sound source is located at a short distance and the sound wave is considered as a spherical wave, in the method of arranging virtual microphones on one side of the actual microphone shown in FIG. 1, the error of the plane wave approximation with respect to the actual sound wave can become non-uniform and large. In contrast, as shown in FIG. 2, by arranging the virtual microphones Lv-Rv symmetrically on both sides of the actual microphones L-R, even when considering the sound wave as a spherical wave as in the case of a short-distance sound source, compared with FIG. 1, the error of the plane wave approximation with respect to the actual sound wave (spherical wave) can be dispersed on both sides of the actual microphone, so that it is possible to make the error of the plane wave approximation uniform and small (for example, low-level equalization). Also, in FIG. 2, only the case where α > 1 needs to be considered, and the case where α ≤ 1 does not need to be considered.

[0021] In FIG. 2, when the distance between the actual microphones L-R is "r", the distance between the virtual microphones Lv-Rv is represented by "α × r". At this time, the phase components φ Rv (t, f) and φ Lv (t, f) are expressed as follows in the same manner as in the case of FIG. 1. φ Rv (t,f)=φ L (t,f)+0.5×(α+1)×φ d1 (t,f) φ d1 (t,f)=φ R (t,f)-φ L (t,f), φ d1 (t,f)=φ R (t,f)-φ L (t,f)-2π, or, φ d1 (t,f)=φ R (t,f)-φ L (t,f)+2π, -π<φ d1 (t,f)<π (2) φ Lv (t,f)=φ R (t,f)+0.5×(α+1)×φ d2 (t,f) φ d2 (t,f)=φ L (t,f)-φR (t,f), φ d2 (t,f)=φ L (t,f)-φ R (t,f)-2π, or φ d2 (t,f)=φ L (t,f)-φ R (t,f)+2π, -π<φ d2 (t,f)<π (3)

[0022] By the way, the method for generating binaural signals when virtual microphones are placed on either side of physical microphones has not been sufficiently studied.

[0023] For example, in the binaural signal generation method described above, the inter-channel phase difference of the actual microphone (e.g., (φ2-φ1), φ d1 or φ d2 A virtual microphone (or binaural signal) is generated by extrapolating the above. In this case, for example, the higher the frequency component, the more likely the inter-channel phase difference is to exceed 2π, making it difficult to uniquely identify the correct phase difference. Thus, because the phase wrap-around processing has not been sufficiently considered, the accuracy of estimating the inter-channel phase difference by extrapolation may decrease, and the quality of the binaural signal using the virtual microphone may deteriorate.

[0024] Furthermore, the binaural signal generation method described above does not modify the amplitude difference between channels (hereinafter also referred to as "inter-channel amplitude difference"). Therefore, for example, when a nearby sound source is located behind the head (or auricle), amplitude attenuation due to occlusion (or diffraction) is not taken into consideration. Consequently, the difference between the binaural signal from a virtual microphone and the binaural signal from an actual microphone tends to become large, and the quality of the binaural signal using a virtual microphone may deteriorate.

[0025] In one non-limiting embodiment of this disclosure, a method for improving the quality of a binaural signal when virtual microphones are placed on either side of a physical microphone is described.

[0026] For example, a communication terminal may set a frequency band that can uniquely identify the phase difference between signals (inter-channel phase difference) based on the distance between the actual microphones constituting a stereo microphone as the target for phase difference calculation, and perform a phase difference calculation in that frequency band such that the phase difference is reliably within the range of 2π, thereby generating a virtual binaural microphone signal having an inter-aural phase difference.

[0027] Furthermore, the communication terminal applies adjustments (e.g., modifications) to the interaural amplitude difference of the virtual binaural microphone signal, taking into account the effects of shielding (or diffraction) by the head, etc., based on the estimated direction of arrival (angle of arrival) of the sound wave in a frequency band in which the phase difference can be uniquely identified.

[0028] This allows communication terminals to improve the quality of binaural signals when virtual microphones are placed on either side of physical microphones.

[0029] [Overview of the transmission system] A transmission system for voice signals or acoustic signals (for example, referred to as voice-acoustic signals) according to one non-limiting embodiment of the present disclosure may include, for example, a plurality of communication terminals (for example, corresponding to communication devices).

[0030] Multiple communication terminals may include, for example, a transmitting device (hereinafter also referred to as a "transmitting terminal") that encodes an audio signal and transmits the encoded information, and a receiving device (hereinafter also referred to as a "receiving terminal") that receives the encoded information and decodes the audio signal. Furthermore, in one non-limiting embodiment of this disclosure, the process for improving the quality of the binaural signal described above (hereinafter also referred to as the "virtual binauralization process") may be performed in either the transmitting terminal or the receiving terminal.

[0031] Figure 3 is a block diagram showing an example configuration of a communication terminal. In the communication terminal (transmitting terminal or receiving terminal) shown in Figure 3, the communication unit (corresponding to, for example, a transmitting circuit or a receiving circuit) transmits or receives encoded audio signal information (for example, stereo encoded information). The generation unit (corresponding to, for example, a generation circuit) performs virtual binaural processing on the stereo signal (for example, the stereo signal to be encoded or the decoded stereo signal) to generate a virtual stereo signal (for example, a pseudo-binaural signal or a virtual binaural signal). An example configuration of a communication terminal in an audio signal transmission system will be described later.

[0032] Figure 4 is a diagram showing the process flow for generating signals observed by virtual microphones Lv and Rv in a communication terminal according to one embodiment of the present disclosure.

[0033] In Figure 4, the communication terminal calculates (generates) signals observed by virtual microphones Lv and Rv, taking as input the signals observed by real microphones L and R, the distance r between L and R, and the ratio α (extrapolation coefficient) of the distance between Lv and Rv and the distance between L and R.

[0034] <Step 1 (S1)> The communication terminal converts the L-channel signal (Lch signal) and R-channel signal (Rch signal) contained in the input stereo signal into a frequency spectrum using FFT (Fast Fourier Transform) or Short-Time Fourier Transform, and generates an amplitude spectrum a L (t, f), a R (t, f), and the phase spectrum φ L (t, f), φ R Calculate (t, f).

[0035] <Step 2 (S2)> The communication terminal calculates the phase spectrum φ using the frequency band where the frequency (f) is less than ss / r as the target of calculation. L (t, f) and φ R Phase difference with (t, f) (inter-channel phase difference) φ d Calculate (t, f). Here, ss represents the speed of sound. Phase difference φ d The method for calculating (t, f) is, for example, the method explained using Figure 2 (for example, φ d1(t, f) or φ d2 The calculation method of (t, f) may be applied.

[0036] Here, the phase difference φ d Setting (for example, limiting) the calculation target of (t, f) to a frequency band where the frequency is less than ss / r is to ensure that the phase difference φ d (t, f) is within 2π.

[0037] For example, the time required for the sound wave to travel the distance r between the microphones is r / ss. That is, the maximum value of the time difference between the signals observed by the two actual microphones L and R is r / ss. At this time, for a frequency component whose one period exceeds r / ss (that is, a frequency component whose frequency is less than ss / r), the phase difference between the signals observed by the two actual microphones L and R falls within the range of 2π (for example, ±π). On the other hand, for a frequency component whose one period is less than or equal to r / ss (that is, a frequency component whose frequency is greater than or equal to ss / r), the phase difference between the signals observed by the two actual microphones L and R may be 2π (for example, ±π) or more.

[0038] When the phase difference is 2π or more, it becomes difficult for the communication terminal to uniquely determine the phase difference, and there is a possibility that the true phase difference cannot be calculated. The extrapolation process (generation of virtual microphones) using an incorrect phase affects the quality of the signals observed by the virtual microphones. Therefore, the communication terminal performs the extrapolation process in a frequency component (for example, a frequency band where the frequency is less than ss / r) where the true phase difference can be specified.

[0039] <Step 3 (S3)> The communication terminal calculates (or estimates) the azimuth angle θ(t, f) of the sound source using, for example, the phase difference φ d (t, f) for a frequency band where the frequency is less than ss / r (f < ss / r).

[0040] FIG. 5 is a diagram for explaining an example of a method for calculating the azimuth angle (azimuth) θ of a sound wave arriving from a sound source from the phase difference φ d of the signals observed by the actual microphones L and R with the microphone - to - microphone distance r.

[0041] As shown in FIG. 5, when assuming that the sound wave is a plane wave, the difference d in the propagation distances of the sound wave arriving at the microphone L and the sound wave arriving at the microphone R is expressed as d = r×sinθ. Also, the phase difference φ d of the signals observed at the microphones L and R at the frequency f d is expressed as φ d = 2πf×d / ss. Therefore, from these two equations, the azimuth angle θ can be calculated as θ = arcsin((ss×φ

[0042] For calculating the azimuth angle θ, it is preferable to calculate the accurate phase difference φ d . Therefore, as described above, the frequency band (target band) for calculating the azimuth angle θ may be set (for example, limited) to a frequency component (for example, a frequency band with a frequency less than ss / r) that can identify the true phase difference.

[0043] <Step 4 (S4)> The communication terminal calculates, for example, the phase spectra φ Lv (t, f) and φ Rv (t, f) of the signals observed at the virtual microphones Lv and Rv for a frequency band (f < ss / r) where the frequency is less than ss / r. For example, the communication terminal uses the phase difference φ d calculated in Step 2 and the extrapolation coefficient α to calculate the phase spectra φ Lv and φ Rv by the extrapolation process described in FIG. 2.

[0044] <Step 5 (S5)> For a frequency band (off-target band, f≧ss / r) that is not the calculation target of the phase difference φ d , the communication terminal copies, for example, the corresponding time and frequency components of the phase spectra φ L and φ R to generate the phase spectra φ Lv and φ Rv of the signals (virtual Lch signal and virtual Rch signal) observed at the virtual microphones Lv and Rv.

[0045] Also, the communication terminal, for example, the amplitude spectra a L and a RCopy the following and view the amplitude spectrum of the signal observed at virtual microphone Lv and Rv. Lv and a Rv Generates.

[0046] <Step 6 (S6)> For frequency bands where the frequency is less than ss / r, the communication terminal uses the inter-channel amplitude difference (e.g., LR amplitude difference) of the actual microphone L and R, which is associated with the azimuth angle θ calculated in Step 3. d Determine the amplitude difference a d Based on this, the amplitude difference of the signals between virtual microphones is adjusted (or corrected, modified). For example, the communication terminal adjusts the amplitude spectrum a of the signal observed by the virtual microphone. Lv (t, f) and a Rv Of (t, f), the amplitude spectrum with the smallest amplitude is attenuated, and the amplitude difference is a d a Lv (t, f) and a Rv Adjust (t, f).

[0047] Azimuth angle θ and inter-channel amplitude difference a d In this correspondence, for example, amplitude attenuation due to shielding (or diffraction) may be considered when the sound source is located behind the head (or auricle) relative to a communication terminal (e.g., a smartphone).

[0048] Furthermore, for example, for each frequency (e.g., bin or subband) or for multiple frequencies, the azimuth angle θ and the inter-channel amplitude difference a d The two may be associated with each other. For example, the azimuth angle θ and the inter-channel amplitude difference a d A lookup table may be created with the data points associated with each other.

[0049] Alternatively, the azimuth angle θ may be associated with a uniform amplitude ratio (for example, the amplitude ratio of the actual microphone L and R) in part or all of the bandwidth targeted for amplitude difference adjustment. In this case, the relationship between the azimuth angle and the amplitude ratio may be approximated by a transformation function using a linear function, a quadratic function, or a combination thereof.

[0050] <Step 7 (S7)> The communication terminal generates the phase spectrum φ Lv and amplitude spectrum a LvBy performing an inverse Fourier transform such as IFFT (Inverse Fast Fourier Transform), it is converted into a time signal L(t) (frequency-time conversion), and the generated phase spectrum φ Rv and amplitude spectrum a Rv are subjected to an inverse Fourier transform to be converted into a time signal R(t) (frequency-time conversion), and signals observed by the virtual microphones Lv and Rv are generated. For example, when the distance (spacing) between the virtual microphones Lv and Rv constituting a virtual stereo microphone is the inter-aural distance, the signals observed by the virtual microphones Lv and Rv are called "pseudo-binaural signals" or "virtual binaural signals".

[0051] The example of the process for generating signals observed by the virtual microphones Lv and Rv in the communication terminal has been described above.

[0052] In step 4 (phase extrapolation process), the frequency band for calculating the phase of the signal observed by the virtual microphone is not limited to the frequency band where the frequency is less than ss / r (f < ss / r). For example, the frequency band for calculating the phase of the signal observed by the virtual microphone may be set to a frequency band lower than the frequency (for example, approximately 1500 Hz to 1600 Hz) specified (calculated) from the inter-aural distance (for example, approximately 22 cm to 23 cm). Also, the frequency band for calculating the phase of the signal observed by the virtual microphone may be set to, for example, a frequency band where the phase has a greater influence on auditory sound source localization (for example, a frequency band of approximately 1500 Hz or less). For example, the frequency band for calculating the phase of the signal observed by the virtual microphone may be set to a certain frequency band (for example, a frequency band of approximately 1500 Hz or less) regardless of the microphone distance r. Thereby, it becomes possible to narrow the frequency band to which the process of calculating the phase of the signal observed by the virtual microphone by extrapolation is applied, and the processing amount in the communication terminal can be reduced.

[0053] Alternatively, for example, by setting a wider distance r (e.g., 10cm to 20cm) between microphones in an area where physical microphones can be placed, the frequency band below ss / r can be narrowed. This makes it possible to further narrow the frequency band to which the process of extrapolating the phase of the signal observed by the virtual microphone is applied, thereby reducing the processing load on the communication terminal.

[0054] Furthermore, in step 2 (calculation of inter-channel phase difference of the actual microphone), the communication terminal may perform wrap processing by subtracting 2π from the calculated phase difference if the calculated phase difference is π or greater, and adding 2π to the calculated phase difference if the calculated phase difference is -π or less, in the frequency band where the frequency is less than ss / r. This allows the communication terminal to correct the calculated phase difference to be within the ±π range (i.e., within the range where the phase difference can be uniquely determined) even if an error occurs in the expected phase difference (for example, if the value falls outside the ±π range).

[0055] Furthermore, in step 6 (adjustment of amplitude difference), the frequency band to which the amplitude difference of the signal observed by the virtual microphone is adjusted may be set (or limited) to a frequency higher than the boundary frequency calculated from the interaural distance (e.g., approximately 22cm to 23cm) (e.g., approximately 1500Hz to 1600Hz). Alternatively, the frequency band to which the amplitude difference of the signal observed by the virtual microphone is adjusted may be set to a frequency band in which the amplitude has a greater influence on auditory sound source localization (e.g., a frequency higher than approximately 1500Hz to 1600Hz). The upper limit of the frequency band to which the amplitude difference of the signal observed by the virtual microphone is adjusted may be the ss / r frequency, as described above. These settings make it possible to narrow the frequency band to which the process of adjusting the amplitude of the signal observed by the virtual microphone is applied, thereby reducing the processing load on the communication terminal.

[0056] Furthermore, in step 6, the process of adjusting (correcting) the amplitude difference of the signal observed by the virtual microphone may be applied to frequency bands with frequencies greater than or equal to ss / r (for example, frequency bands with a phase difference greater than 2π). In this case, the phase difference φd For frequency components that are not the calculation target of (or the azimuth angle θ), for example, frequency components with a frequency of ss / r or higher, the azimuth angle θ may be determined as follows. For example, the communication terminal, in step 3 (for the frequency components that are the calculation target of the phase difference φ d ), classifies the azimuth angle θ calculated for the frequency components that are the calculation target into positive (for example, the azimuth is on the R channel side) and negative (for example, the azimuth is on the L channel side), and calculates the average values "θ+" and "θ-" of the respective azimuth angles. The communication terminal, for example, for the phase difference φ d may determine the azimuth angle θ(t, f) of frequency components that are not the calculation target as follows. The communication terminal, when a L (t, f) > a R (t, f), determines the azimuth angle θ(t, f) = θ-, and when a L (r, f) < a R (t, f), determines the azimuth angle θ(t, f) = θ+, and when a L (r, f) = a R (t, f), determines the azimuth angle θ(t, f) = 0. The communication terminal may apply the amplitude adjustment in step 6 based on the azimuth angle θ(t, f) determined in this way.

[0057] Also, in the phase difference adjustment of steps 4 and 5, the extrapolation coefficient α may be gradually reduced so as to approach 1.0 as the frequency increases. For example, in the phase extrapolation process of step 5, in the frequency band where the frequency is ss / r or higher, the extrapolation coefficient α is gradually reduced so as to approach 1.0 as the frequency increases (for example, corresponding to approaching the actual microphone for the extrapolation point), and α = 1 (for example, corresponding to no extrapolation) may be set at or above a certain frequency. Or, for example, α = 1 may be set in the frequency band where the frequency is ss / r or higher. Or, for example, in the phase difference adjustment of steps 4 and 5, the extrapolation coefficient α is gradually reduced so as to approach 1.0 as the frequency increases in the frequency band below the frequency of ss / r, and α = 1 is set at or above the frequency of ss / r (the boundary frequency of the extrapolation process).

[0058] Also, in the amplitude difference adjustment of step 6, the amplitude difference may be gradually reduced so as to approach 0 as the frequency increases.

[0059] Furthermore, in step 6, the amplitude difference of the signal observed by the virtual microphone is amplitude difference a d We have explained how to adjust the amplitude difference so that it becomes as follows, but the adjustment of the amplitude difference of the signal observed by the virtual microphone is not limited to this, and the amplitude difference a d An amplitude difference adjustment may be performed so that the value is calculated based on [the formula / condition].

[0060] Here, we will explain specific examples of the phase spectrum calculation process in Step 4 and Step 5.

[0061] As an example, we will explain the case where the distance between microphones is r = 4.25 cm, the speed of sound is ss = 340 m / s, the signal sampling frequency is 48 kHz, and the number of FFT points is 1024. The distance between microphones, sampling frequency, and number of FFT points are not limited to these values ​​and may be other values.

[0062] In this case, the frequency ss / r = 8000Hz corresponds to 1024 × 8000 / 48000 = 170.67 in the FFT frequency bins. Therefore, the communication terminal sets the frequency bins from the 170th onward, which correspond to frequency bands lower than 8000Hz, as targets for phase extrapolation (targets of step 4). Note that among the frequency bins from the 170th onward, the frequency bin corresponding to the DC component (the first frequency bin) does not need to be set as a target for phase extrapolation.

[0063] Furthermore, the communication terminal may gradually reduce α to 1.0 up to a certain frequency bin (for example, the 250th frequency bin) for the 171st frequency bin and above, which correspond to a frequency band of 8000 Hz or higher, and then set α=1 (no extrapolation or no phase manipulation) from the 251st frequency bin onward.

[0064] [Example of Communication Terminal Configuration] The following describes an example of the configuration of a communication terminal in a voice acoustic signal transmission system according to one non-limiting embodiment of the present disclosure.

[0065] As described above, the communication terminal performs pre-processing (e.g., processing at the transmitting terminal) or post-processing (e.g., processing at the receiving terminal) to generate an input signal (e.g., a pseudo-binaural signal) from a virtual binaural microphone (e.g., a virtual stereo microphone) based on signals input from two physical microphones (e.g., stereo microphones).

[0066] <Configuration Example 1> Configuration Example 1 describes a configuration example of a communication terminal when the pseudo-binaural signal generation process (virtual binauralization process) explained using Figure 4 is performed as processing (for example, post-processing) at the receiving terminal. In Configuration Example 1, the transmission system includes, for example, a transmitting terminal 10 and a receiving terminal 20.

[0067] <Example Configuration of Transmitting Terminal 10> Figure 6 is a block diagram showing an example configuration of the transmitting terminal 10 in Configuration Example 1. The transmitting terminal 10 transmits, for example, stereo encoded information obtained by encoding the signal from a stereo microphone to the receiving terminal 20.

[0068] In Figure 6, the transmitting terminal 10 comprises a plurality of microphones 11, an input signal selection unit 12, a microphone distance information transmission unit 13, and an encoding unit 14.

[0069] Multiple microphones 11 supply the audio signals observed by each microphone to the input signal selection unit 12.

[0070] The input signal selection unit 12 selects two signals from the audio signals supplied by the multiple microphones 11 to be used for generating a pseudo-binaural signal, and outputs the two selected signals as a stereo signal (L channel (Lch signal) and R channel (Rch signal)) to the encoding unit 14.

[0071] Furthermore, the input signal selection unit 12 outputs mode information to the encoding unit 14 indicating that the two selected signals are stereo signals and that encoding should be performed in stereo signal encoding mode. In addition to the stereo signal encoding mode, the information that can be set in the mode information may also include other encoding modes applicable to the transmitting terminal 10.

[0072] Furthermore, the input signal selection unit 12 outputs information (for example, referred to as "selected microphone information") regarding the microphones that supply the two selected signals (for example, the microphones that make up a stereo microphone) to the microphone distance information transmission unit 13. The selected microphone information may include, for example, at least microphone distance information indicating the distance between the two selected microphones.

[0073] The microphone distance information transmission unit 13 encodes the selected microphone information (including at least the microphone distance information) input from the input signal selection unit 12 and sends the selected microphone information to the receiving terminal 20.

[0074] The encoding unit 14 encodes the signal input from the input signal selection unit 12 based on the mode information input from the input signal selection unit 12, and sends the encoded information to the receiving terminal 20. For example, if the mode information indicates stereo encoding mode, the encoding unit 14 encodes the stereo signal input from the input signal selection unit 12 in stereo encoding mode and sends the stereo encoded information to the receiving terminal 20.

[0075] For example, the transmission frequency of selected microphone information may be lower than the transmission frequency of stereo coding information. Selected microphone information may also be transmitted as signaling information at the start of a session or as control information when the selected microphone is changed.

[0076] Furthermore, the encoded selected microphone information (including, for example, microphone distance information) may be transmitted separately from the stereo encoded information, as shown in Figure 6, or it may be included in the bitstream as part of the stereo encoded information.

[0077] <Example Configuration of Receiving Terminal 20> Figure 7 is a block diagram showing an example configuration of the receiving terminal 20 in Configuration Example 1. The receiving terminal 20, for example, receives stereo encoding information transmitted from the transmitting terminal 10 and performs virtual binaural processing using the received stereo encoding information.

[0078] In Figure 7, the receiving terminal 20 includes a microphone distance information receiving unit 21, a decoding unit 22, and a virtual binaural processing unit 23. The virtual binaural processing unit 23 also includes, for example, an inter-channel phase difference changing unit 24 and an inter-channel amplitude difference changing unit 25.

[0079] The microphone distance information receiving unit 21 and decoding unit 22 shown in Figure 7 may be included in the communication unit shown in Figure 3, for example. Also, the virtual binaural processing unit 23 shown in Figure 7 may be included in the generation unit shown in Figure 3, for example.

[0080] The microphone distance information receiving unit 21 receives selected microphone information (for example, including microphone distance information) transmitted from the transmitting terminal 10 and outputs the microphone distance information included in the selected microphone information to the channel phase difference changing unit 24.

[0081] The decoding unit 22 receives stereo coding information transmitted from the transmitting terminal 10, decodes the stereo coding information, and outputs the decoded stereo signal (L channel (Lch signal) and R channel (Rch signal)) to the inter-channel phase difference changing unit 24.

[0082] The virtual binaural processing unit 23 uses, for example, the inter-microphone distance information input from the inter-microphone distance information receiving unit 21 to perform virtual binaural processing (for example, the processing shown in Figure 4) on the decoded stereo signal input from the decoding unit 22, thereby generating a signal from a virtual stereo microphone composed of virtual microphones (for example, a pseudo-binaural signal).

[0083] In the virtual binaural processing unit 23, the inter-channel phase difference changing unit 24 uses the inter-microphone distance information input from the inter-microphone distance information receiving unit and the decoded stereo signal input from the decoding unit 22 to adjust (change) the phase difference of the phase spectrum in the virtual microphone, for example, by the method described using Figures 2 and 4 (steps 4 and 5) (phase extrapolation processing), and outputs the frequency spectrum (e.g., L and R) of the pseudo-binaural signal including the phase spectrum to the inter-channel amplitude difference changing unit 25. The inter-channel phase difference changing unit 24 also determines the azimuth angle θ of the sound source, for example, by the method described using Figures 4 (step 3) and 5 (estimation of the azimuth angle θ of the sound source), and outputs the information of the determined azimuth angle θ to the inter-channel amplitude difference changing unit 25.

[0084] In the virtual binaural processing unit 23, the inter-channel amplitude difference modification unit 25 determines (estimates) the inter-channel amplitude difference based on the azimuth angle θ input from the inter-channel phase difference modification unit 24. The inter-channel amplitude difference modification unit 25 adjusts (changes) the amplitude difference of the frequency spectrum L (amplitude and phase) and R (amplitude and phase) of the pseudo-binaural signal input from the inter-channel phase difference modification unit 24 using the determined inter-channel amplitude difference, for example, by the method (amplitude difference adjustment) described using Figure 4 (step 6). The inter-channel amplitude difference modification unit 25 outputs a pseudo-binaural signal with adjusted amplitude differences. The output pseudo-binaural signal is presented to the user's ears, for example, via headphones.

[0085] <Example of the configuration of the inter-channel phase difference changing unit 24> Figure 8 is a block diagram showing an example of the internal configuration of the inter-channel phase difference changing unit 24 shown in Figure 7.

[0086] The inter-channel phase difference changing unit 24 generates a signal from a virtual stereo microphone (e.g., a pseudo-binaural signal) composed of virtual microphones placed on either side of the stereo microphone by extrapolating the phase spectrum (or phase component) of the decoded stereo signal obtained by decoding the stereo coding information in a frequency band where the phase difference between the signals of the microphones constituting the stereo microphone (e.g., microphone 11) is within 2π.

[0087] The inter-channel phase difference changing unit 24 shown in FIG. 8 includes, for example, time / frequency conversion units 241-1, 241-2, a phase difference calculation unit 242, phase difference changing units 243-1, 243-2, and an azimuth estimation unit 244.

[0088] The inter-microphone distance information output from the inter-microphone distance information receiving unit 21 in the receiving terminal 20 is input to the phase difference changing units 243-1, the phase difference changing unit 243-2, and the azimuth estimation unit 244.

[0089] Also, the L-channel component (Lch signal) of the decoded stereo signal output from the decoding unit 22 in the receiving terminal 20 is input to the time / frequency conversion unit 241-1 for the L channel, and the R-channel component (Rch signal) of the decoded stereo signal is input to the time / frequency conversion unit 241-2 for the R channel.

[0090] The time / frequency conversion unit 241-1 converts the input L-channel signal into a frequency spectrum (amplitude component, phase component) using FFT or short-time Fourier transform, etc., and outputs the amplitude component (amplitude spectrum) (for example, a L ) to the inter-channel amplitude difference changing unit 25, and outputs the phase component (phase spectrum) (for example, φ L ) to the phase difference calculation unit 242 and the phase difference changing unit 243-1.

[0091] The time / frequency conversion unit 241-2 converts the input R-channel signal into a frequency spectrum (amplitude component, phase component) using FFT or short-time Fourier transform, etc., and outputs the amplitude component (amplitude spectrum) (for example, a R ) to the inter-channel amplitude difference changing unit 25, and outputs the phase component (phase spectrum) (for example, φ R ) to the phase difference calculation unit 242 and the phase difference changing unit 243-2.

[0092] The phase difference calculation unit 242 uses the phase spectrum of the L channel input from the time / frequency conversion unit 241-1 and the phase spectrum of the R channel input from the time / frequency conversion unit 241-2 to calculate the phase difference between the L-R channels (for example, φ dThe phase difference is calculated, and the calculated phase difference information is output to the phase difference changing unit 243-1 for the L channel, the phase difference changing unit 243-2 for the R channel, and the azimuth angle estimation unit 244.

[0093] The phase difference changing unit 243-1 receives microphone distance information (e.g., microphone distance r) and the phase spectrum of the L channel (e.g., φ). L ), and phase difference information (for example, φ d Based on this, for example, the phase spectrum (e.g., φ) of the signal observed by the virtual microphone (the L channel of the pseudo-binaural signal) is obtained by the method described using Figures 2 and 4 (Step 4) (e.g., extrapolation). Lv The phase difference changing unit 243-1 generates a phase difference spectrum. The phase difference changing unit 243-1 outputs the generated phase difference spectrum to the inter-channel amplitude difference changing unit 25.

[0094] The phase difference changing unit 243-2 (R channel) receives microphone distance information (e.g., microphone distance r) and the phase spectrum of the R channel (e.g., φ). R ), and phase difference information (for example, φ d Based on this, for example, the phase spectrum (e.g., φ) of the signal observed by the virtual microphone (the R channel of the pseudo-binaural signal) is obtained by the method described using Figures 2 and 4 (Step 4) (e.g., extrapolation). Rv The phase difference changing unit 243-2 generates a phase difference spectrum. The phase difference changing unit 243-2 outputs the generated phase difference spectrum to the inter-channel amplitude difference changing unit 25.

[0095] The azimuth angle estimation unit 244 receives microphone distance information (e.g., microphone distance r) and phase difference information (e.g., φ d Based on this, the azimuth angle θ of the sound source is estimated using the method described with reference to Figures 4 (Step 3) and 5. The azimuth angle estimation unit 244 outputs information regarding the estimated azimuth angle θ to the inter-channel amplitude difference changing unit 25.

[0096] The extrapolation coefficient α used in the phase extrapolation process in the phase difference changing units 243-1 and 243-2 may be determined, for example, by the ratio of the distance between the ears to the distance between the microphones r.

[0097] <Example of the configuration of the inter-channel amplitude difference changing unit 25> Figure 9 is a block diagram showing an example of the internal configuration of the inter-channel amplitude difference changing unit 25 shown in Figure 7.

[0098] The inter-channel amplitude difference changing unit 25 adjusts the distance r between the microphones constituting the stereo microphone and the phase difference φ between the microphones constituting the stereo microphone in a frequency band where the phase difference of the signals between the microphones (e.g., microphone 11) is within 2π. d Based on this, the azimuth angle θ of the sound source is estimated, and the amplitude difference a of the signals between the microphones constituting the stereo microphone, which is associated with the azimuth angle θ, is calculated. d Based on this, the amplitude difference between the signals from the virtual microphones that make up the virtual stereo microphone is adjusted. This generates a pseudo-binaural signal with adjusted amplitude.

[0099] The inter-channel amplitude difference changing unit 25 shown in Figure 9 comprises an amplitude difference estimation unit 251, an amplitude difference changing unit 252, and frequency / time conversion units 253-1 and 253-2.

[0100] The amplitude difference estimation unit 251 uses the azimuth angle θ input from the inter-channel phase difference changing unit 24 to determine the inter-channel amplitude difference (for example, a) using the method (azimuth angle θ estimation) described with reference to Figures 4 (step 6) and 5. d The system estimates the inter-channel amplitude difference and outputs the estimated inter-channel amplitude difference information to the amplitude difference modification unit 252.

[0101] The amplitude difference modification unit 252 uses the inter-channel amplitude difference input from the amplitude difference estimation unit 251 and the amplitude spectrum (L channel, R channel) of the pseudo-binaural signal input from the inter-channel phase difference modification unit 24 to attenuate the amplitude spectrum of the channel with the smaller amplitude among the L channel and R channel, as explained in Figure 4 (step 6), and adjusts (modifies) the amplitude of the amplitude spectrum so that there is an inter-channel amplitude difference. The amplitude difference modification unit 252 outputs the amplitude spectra of the pseudo-binaural signal (L channel and R channel) after amplitude adjustment to the frequency / time conversion units 253-1 and 253-2, respectively.

[0102] The frequency / time conversion unit 253-1 uses the phase spectrum of the phase-shifted L channel input from the inter-channel phase-shifting unit 24 and the amplitude spectrum of the amplitude-shifted L channel input from the amplitude-shifting unit 252 to convert the frequency spectrum into a time signal using IFFT or the like, and outputs it as the L channel signal (Lch signal) of the pseudo-binaural signal.

[0103] The frequency / time conversion unit 253-2 uses the phase spectrum of the R channel with phase difference changed input from the inter-channel phase difference changing unit 24 and the amplitude spectrum of the R channel with amplitude difference changed input from the amplitude difference changing unit 252 to convert the frequency spectrum into a time signal using IFFT or the like, and outputs it as the R channel signal (Rch signal) of the pseudo-binaural signal.

[0104] <Configuration Example 2> Configuration Example 2 describes a configuration example of a communication terminal when the pseudo-binaural signal generation process (virtual binauralization process) explained using Figure 4 is performed as processing (for example, pre-processing) at the transmitting terminal. In Configuration Example 2, the transmission system includes, for example, a transmitting terminal 30 and a receiving terminal 40.

[0105] <Example Configuration of Transmitting Terminal 30> Figure 10 is a block diagram showing an example configuration of the transmitting terminal 30 in Configuration Example 2. The transmitting terminal 30, for example, performs virtual binaural processing using a stereo signal and transmits encoded information that encodes the generated pseudo-binaural signal (virtual stereo signal).

[0106] In Figure 10, the transmitting terminal 30 comprises a plurality of microphones 31, an input signal selection unit 32, a virtual binaural processing unit 33, and an encoding unit 36. The virtual binaural processing unit 33 also includes, for example, an inter-channel phase difference changing unit 34 and an inter-channel amplitude difference changing unit 35.

[0107] The input signal selection unit 32 and the virtual binaural processing unit 33 shown in Figure 10 may be included in the generation unit shown in Figure 3, for example. The transmitting terminal 30 shown in Figure 10 may include, for example, a transmitting unit (corresponding to the communication unit shown in Figure 3) that transmits a signal (e.g., encoded information) to the receiving terminal 40, or a receiving unit (corresponding to the communication unit shown in Figure 3) that receives a signal (e.g., signaling information) from the receiving terminal 40 (not shown).

[0108] Multiple microphones 31 supply the audio signals observed by each microphone to the input signal selection unit 32.

[0109] The input signal selection unit 32 selects two signals from the audio signals supplied by multiple microphones 31 to be used for generating a pseudo-binaural signal, and outputs the two selected signals as a stereo signal (L channel (Lch signal) and R channel (Rch signal)) to the virtual binaural processing unit 33.

[0110] Furthermore, the input signal selection unit 32 outputs information (e.g., selected microphone information) regarding the microphones that supply the two selected signals (e.g., microphones constituting a stereo microphone) to the virtual binaural processing unit 33. The selected microphone information may include, for example, at least microphone distance information indicating the distance between the two selected microphones.

[0111] The virtual binaural processing unit 33 uses, for example, the microphone distance information and stereo signals (L channel, R channel) input from the input signal selection unit 32 to perform virtual binaural processing (for example, the processing shown in Figure 4) and generate a signal from a virtual stereo microphone composed of virtual microphones (for example, a pseudo-binaural signal).

[0112] Furthermore, the virtual binaural processing unit 33 outputs mode information to the encoding unit 36, for example, indicating that the signal output to the encoding unit 36 ​​is a binaural signal and that it should be encoded in binaural encoding mode.

[0113] In the virtual binaural processing unit 33, the inter-channel phase difference changing unit 34 generates a virtual stereo microphone signal (pseudo-binaural signal) composed of virtual microphones placed on either side of the stereo microphone by extrapolating the phase spectrum of the stereo microphone signal in a frequency band where the phase difference between the signals of the microphones constituting the stereo microphone (e.g., microphone 31) is within 2π.

[0114] For example, the inter-channel phase difference changing unit 34 uses the distance information between microphones and the stereo signal to adjust (change) the phase difference of the phase spectrum in the virtual microphone (e.g., the phase spectrum of the pseudo-binaural signal) using the method described using Figures 2 and 4 (steps 4 and 5) (phase extrapolation processing), and outputs the frequency spectrum (e.g., L and R) of the pseudo-binaural signal, including the phase spectrum, to the inter-channel amplitude difference changing unit 35. The inter-channel phase difference changing unit 34 also determines the azimuth angle θ of the sound source using the method described using Figures 4 (step 3) and 5 (estimation of the azimuth angle θ of the sound source), and outputs the information of the determined azimuth angle θ to the inter-channel amplitude difference changing unit 35.

[0115] The transmitting terminal 30 may, for example, receive interaural distance information from the receiving terminal 40 (for example, the dashed line shown in Figure 10). When interaural distance information is received from the receiving terminal 40, the inter-channel phase difference changing unit 34 may, for example, use the ratio of the interaural distance to the microphone distance as an extrapolation coefficient α. This allows the transmitting terminal 30 to generate a pseudo-binaural signal suitable for the receiving user.

[0116] Furthermore, the inter-channel phase difference changing unit 34 may determine the frequency band to be subject to inter-channel phase difference changing processing based on the interaural distance information.

[0117] Furthermore, the inter-channel phase difference conversion unit 34 outputs the amplitude spectra of the input stereo signals (L channel, R channel) to the inter-channel amplitude difference conversion unit 35 as the amplitude spectra of the pseudo-binaural signal.

[0118] The configuration of the inter-channel phase difference changing unit 34 may be the same as that of the inter-channel phase difference changing unit 24 shown in Figure 8, for example.

[0119] In the virtual binaural processing unit 33, the inter-channel amplitude difference changing unit 35 adjusts the distance r between the microphones constituting the stereo microphone and the phase difference φ between the microphones constituting the stereo microphone in a frequency band where the phase difference of the signals between the microphones constituting the stereo microphone is within 2π. d Based on this, the azimuth angle θ of the sound source is estimated, and the amplitude difference a of the signals between the microphones constituting the stereo microphone, which is associated with the azimuth angle θ, is calculated. d Based on this, the amplitude difference between the signals from the virtual microphones that make up the virtual stereo microphone is adjusted. This generates a pseudo-binaural signal with adjusted amplitude.

[0120] For example, the inter-channel amplitude difference changing unit 35 determines (estimates) the inter-channel amplitude difference based on the azimuth angle θ input from the inter-channel phase difference changing unit 34. Using the determined inter-channel amplitude difference, the inter-channel amplitude difference changing unit 35 adjusts (changes) the amplitude difference of the frequency spectrum L (amplitude and phase) and R (amplitude and phase) of the pseudo-binaural signal input from the inter-channel phase difference changing unit 34.

[0121] Furthermore, the inter-channel amplitude difference changing unit 35 uses the amplitude-adjusted amplitude spectra (L channel, R channel) and the phase spectra (L channel, R channel) whose phase difference has been changed by the inter-channel phase difference changing unit 34 to convert the frequency spectrum into a time signal using IFFT or the like to generate a pseudo-binaural signal (L channel, R channel), which is then output to the encoding unit 36.

[0122] Furthermore, the inter-channel amplitude difference changing unit 35 may, for example, determine the frequency band to be subjected to amplitude adjustment processing based on the inter-aural distance information when it receives inter-aural distance information from the receiving terminal 40 (for example, the dashed line shown in Figure 10).

[0123] The configuration of the inter-channel amplitude difference changing unit 35 may be the same as that of the inter-channel amplitude difference changing unit 25 shown in Figure 9.

[0124] In Figure 10, if the mode information input from the virtual binauralization processing unit 33 is in binaural encoding mode, the encoding unit 36 ​​encodes the pseudo-binaural signals (L channel, R channel) input from the virtual binauralization processing unit 33 in binaural encoding mode and sends the encoded binaural signals (encoded information) to the receiving terminal 40.

[0125] <Example Configuration of Receiving Terminal 40> Figure 11 is a block diagram showing an example configuration of the receiving terminal 40 in Configuration Example 2. The receiving terminal 40 receives, for example, encoded information transmitted from the transmitting terminal 30, decodes the received encoded information, and obtains a pseudo-binaural signal.

[0126] In Figure 11, the receiving terminal 40 includes a decoding unit 41. The receiving terminal 40 may also include, for example, a binaural distance information analysis and transmission unit 42.

[0127] The decoding unit 41 receives the encoded binaural signal transmitted from the transmitting terminal 30, decodes it, and outputs a decoded pseudo-binaural signal. The output pseudo-binaural signal is presented to the user's ears, for example, via headphones.

[0128] The binaural distance information analysis and transmission unit 42 extracts the user's binaural distance information using, for example, a position sensor provided on an output device such as headphones worn by the user, and sends it to the transmission terminal 30 (for example, the dashed line shown in Figure 11).

[0129] The above describes an example of the configuration of a communication terminal according to one non-limiting embodiment of the present disclosure.

[0130] Thus, in this embodiment, the communication terminal generates a virtual stereo microphone signal (e.g., a pseudo-binaural signal) composed of virtual microphones arranged on both sides of the stereo microphone by performing virtual binaural processing (e.g., phase difference change or amplitude difference change) on the signal of the actual microphone constituting the stereo microphone.

[0131] For example, in phase difference shifting, the communication terminal generates a pseudo-binaural signal by performing phase extrapolation in the frequency band where the phase difference between the signals from the actual microphones falls within a range of 2π, and by not performing phase extrapolation in frequency bands other than the frequency band where the phase difference falls within a range of 2π. In this way, since extrapolation is not performed in high-frequency components where the phase difference between the signals from the actual microphones tends to exceed 2π (for example, frequency components where it is difficult to accurately identify the phase difference), the communication terminal can improve the accuracy of estimating the inter-channel phase difference by extrapolation.

[0132] Furthermore, for example, in amplitude difference modification, the communication terminal estimates the azimuth angle of the direction of arrival of the sound source signal by phase difference analysis, and applies the interaural amplitude difference corresponding to the estimated azimuth angle to the pseudo-binaural signal. This allows the communication terminal to generate a pseudo-binaural signal that takes into account amplitude attenuation due to shielding (or diffraction) that may occur, for example, when a nearby sound source is located in the shadow of the head (or auricle).

[0133] This allows for the reduction of the difference between the binaural signal obtained using an actual binaural microphone and the binaural signal obtained using a virtual binaural microphone when, for example, extrapolation is used to extend the distance between microphones to the distance between both ears to generate a virtual binaural microphone, thereby improving the quality of the pseudo-binaural signal. Therefore, according to one embodiment of this disclosure, the quality of the binaural signal can be improved when virtual microphones are placed on both sides of the actual microphones.

[0134] For example, even when the distance between microphones is short compared to the distance between both ears, such as when a mobile device (e.g., a smartphone) is equipped with a stereo microphone, the user can obtain a realistic sense of three-dimensionality by listening to the pseudo-binaural signal generated using the stereo signal input from the stereo microphone through headphones. Therefore, according to one embodiment of this disclosure, binaural audio communication using a smartphone's built-in microphone becomes possible, for example.

[0135] An embodiment of the present disclosure has been described above.

[0136] The above example describes a case where an audio signal (e.g., a pseudo-binaural signal) is presented to the user's ears via headphones or the like in a communication terminal, but it is not limited to this. For example, the audio signal may be played back by a speaker instead of headphones. When playing back the audio signal by a speaker, the communication terminal may play back the output signal of the stereo encoding unit as is without performing virtual binaural processing, or it may apply other processing different from virtual binaural processing (e.g., conversion processing to an arbitrary virtual microphone distance different from the inter-ear distance).

[0137] Furthermore, while the above example described a case where the communication terminal performs both phase difference modification and amplitude difference modification during virtual binaural processing, it is not limited to this case. The communication terminal may perform either phase difference modification or amplitude difference modification, but not the other. Even in this case, the quality of the binaural signal can be improved.

[0138] Furthermore, the virtual microphone is not limited to being positioned symmetrically with respect to the physical microphone; it may also be positioned asymmetrically on both sides of the physical microphone.

[0139] Furthermore, while the above example described the generation of binaural signals using a virtual microphone, the signals virtually generated using a virtual microphone are not limited to binaural signals; other signals may also be used. For example, the distance between virtual microphones that constitute a virtual stereo microphone is not limited to the distance between the ears.

[0140] Furthermore, for operations related to parts of the communication terminal configuration described above that are not explicitly stated, operations based on other existing encoding schemes (e.g., stereo encoding schemes) may be performed.

[0141] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. Furthermore, the components in each of the embodiments described above may be combined in any way.

[0142] Furthermore, the notation "...part" in the above-described embodiment may be replaced with other notations such as "...circuit," "...device," "...unit," or "...module."

[0143] This disclosure can be implemented in software, hardware, or software in conjunction with hardware. Each functional block used in the description of the above embodiments may be implemented in part or in whole as an integrated circuit (LSI), and each process described in the above embodiments may be controlled in part or in whole by a single LSI or a combination of LSIs. An LSI may consist of individual chips, or it may consist of a single chip that includes some or all of the functional blocks. An LSI may have data inputs and outputs. Depending on the degree of integration, LSIs may be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

[0144] The integrated circuit implementation method is not limited to LSIs; it may also be implemented using dedicated circuits, general-purpose processors, or dedicated processors. Furthermore, a Field Programmable Gate Array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that allows for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used. This disclosure may be implemented as digital or analog processing.

[0145] Furthermore, if advancements in semiconductor technology or other derived technologies lead to the emergence of integrated circuit technologies that can replace LSIs, then naturally, it would be possible to use those technologies to integrate functional blocks. The application of biotechnology, for example, is a possibility.

[0146] This disclosure is applicable to all types of devices, systems, and equipment having communication capabilities (collectively referred to as communication equipment). Communication equipment may include a radio transceiver and a processing / control circuit. The radio transceiver may include a receiver and a transmitter, or both as functions. The radio transceiver (transmitter, receiver) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or similar. Non-exclusive examples of communication devices include telephones (mobile phones, smartphones, etc.), tablets, personal computers (PCs) (laptops, desktops, notebooks, etc.), cameras (digital still / video cameras, etc.), digital players (digital audio / video players, etc.), wearable devices (wearable cameras, smartwatches, tracking devices, etc.), game consoles, digital book readers, telehealth / telemedicine devices, vehicles or mobile transport with communication capabilities (cars, airplanes, ships, etc.), and combinations of the above-mentioned devices.

[0147] Communication devices are not limited to portable or movable devices, but also include all kinds of non-portable or fixed devices, devices, and systems, such as smart home devices (appliances, lighting fixtures, smart meters or measuring instruments, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0148] Communication includes data communication via cellular systems, wireless LAN (Local Area Network) systems, and communication satellite systems, as well as data communication using combinations of these.

[0149] Furthermore, the communication device also includes devices such as controllers and sensors that are connected to or linked to a communication device that performs the communication functions described in this disclosure. For example, this includes controllers and sensors that generate control signals and data signals used by the communication device that performs the communication functions of the communication device.

[0150] Furthermore, communication equipment includes infrastructure facilities such as base stations, access points, and any other devices, devices, and systems that communicate with or control the aforementioned non-limited types of equipment.

[0151] A communication device according to one embodiment of the present disclosure comprises a receiving circuit that receives encoded information obtained by encoding the signal of a stereo microphone, and a generating circuit that generates a signal of a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of a decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π.

[0152] In one embodiment of the present disclosure, the generation circuit estimates the azimuth angle of the sound source based on the distance between the microphones constituting the stereo microphone and the phase difference, and adjusts the amplitude difference of the signals between the virtual microphones constituting the virtual stereo microphone based on the amplitude difference of the signals between the microphones constituting the stereo microphone, which is associated with the azimuth angle.

[0153] In one embodiment of the present disclosure, given the speed of sound ss and the distance r between the microphones constituting the stereo microphone, the frequency band is a frequency band less than the frequency ss / r.

[0154] In one embodiment of the present disclosure, the frequency band is a frequency band below a frequency determined based on the interaural distance.

[0155] In one embodiment of the present disclosure, the generation circuit adjusts the amplitude difference of the signals between the virtual microphones in a frequency band where the phase difference is within 2π, a frequency band where the phase difference exceeds 2π, or a frequency band higher than the frequency determined based on the distance between the ears.

[0156] In one embodiment of the present disclosure, the extrapolation coefficient in the extrapolation process is determined by the ratio of the interaural distance to the distance between the microphones constituting the stereo microphone.

[0157] In one embodiment of the present disclosure, the distance between the virtual microphones constituting the virtual stereo microphone is the distance between the ears.

[0158] In a signal generation method according to one embodiment of the present disclosure, the communication device receives encoded information obtained by encoding the signal of a stereo microphone, and generates a signal of a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of the decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π.

[0159] A communication device according to one embodiment of the present disclosure comprises a generation circuit that generates signals for a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of the signals of the stereo microphone in a frequency band in which the phase difference of the signals between the microphones constituting the stereo microphone is within 2π, and a transmission circuit that transmits encoded information obtained by encoding the signals of the virtual stereo microphone.

[0160] In one embodiment of the present disclosure, the present invention further comprises a receiving circuit that receives information regarding the distance between the ears from a receiving device that receives the encoded information, wherein the generating circuit determines the extrapolation coefficient in the extrapolation process based on the ratio of the distance between the ears and the distance between the microphones constituting the stereo microphone.

[0161] In a signal generation method according to one embodiment of the present disclosure, the communication device generates a signal for a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of the signal of the stereo microphone in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π, and transmits encoded information obtained by encoding the signal of the virtual stereo microphone.

[0162] All disclosures in the specification, drawings, and abstract contained in the Japanese application No. 2024-196587, filed on November 11, 2024, are incorporated herein by reference.

[0163] One embodiment of this disclosure is useful for audio signal transmission systems and the like.

[0164] 10, 30 Transmitting terminal 11, 31 Microphone 12, 32 Input signal selection unit 13 Microphone distance information transmission unit 14, 36 Encoding unit 20, 40 Receiving terminal 21 Microphone distance information receiving unit 22, 41 Decoding unit 23, 33 Virtual binaural processing unit 24, 34 Inter-channel phase difference modification unit 25, 35 Inter-channel amplitude difference modification unit 42 Inter-ear distance information analysis and transmission unit 241 Time / frequency conversion unit 242 Phase difference calculation unit 243 Phase difference modification unit 244 Azimuth angle estimation unit 251 Amplitude difference estimation unit 252 Amplitude difference modification unit 253 Frequency / time conversion unit

Claims

1. A communication device comprising: a receiving circuit that receives encoded information obtained by encoding the signal of a stereo microphone; and a generating circuit that generates a signal of a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of a decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π.

2. The communication device according to claim 1, wherein the generation circuit estimates the azimuth angle of a sound source based on the distance between the microphones constituting the stereo microphone and the phase difference, and adjusts the amplitude difference of signals between virtual microphones constituting the virtual stereo microphone based on the amplitude difference of signals between the microphones constituting the stereo microphone which is associated with the azimuth angle.

3. The communication device according to claim 1, wherein, given the speed of sound ss and the distance r between the microphones constituting the stereo microphone, the frequency band is a frequency band less than the frequency ss / r.

4. The communication device according to claim 1, wherein the frequency band is a frequency band below a frequency determined based on the distance between the ears.

5. The communication device according to claim 2, wherein the generation circuit adjusts the amplitude difference of the signals between the virtual microphones in a frequency band where the phase difference is within 2π, a frequency band where the phase difference exceeds 2π, or a frequency band higher than the frequency determined based on the distance between the two ears.

6. The communication device according to claim 1, wherein the extrapolation coefficient in the extrapolation process is determined by the ratio of the distance between both ears to the distance between the microphones constituting the stereo microphone.

7. The communication device according to claim 1, wherein the distance between the virtual microphones constituting the virtual stereo microphone is the distance between both ears.

8. A signal generation method comprising: a communication device receiving encoded information obtained by encoding the signal of a stereo microphone; and generating a signal for a virtual stereo microphone composed of virtual microphones arranged on both sides of the stereo microphone by extrapolating the phase spectrum of the decoded stereo signal obtained by decoding the encoded information in a frequency band in which the phase difference between the signals of the microphones constituting the stereo microphone is within 2π.

Images