Signal processing device and signal processing method

Abstract

This signal processing device is provided with a detection circuit for detecting a temporal variation in a time difference between channels of a stereo signal, and a control circuit for controlling the degree of smoothing of an inter-channel correlation function on the basis of the temporal variation in the time difference between the channels.

Description

[Technical Field]

[0001] The present disclosure relates to a signal processing device and a signal processing method. [Background technology]

[0002] For example, there is a coding technique for stereo audio signals (hereinafter also referred to as stereo signals) (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-60788 Summary of the Invention

[0004] However, there is room for further study on how to encode a stereo signal when the sound source is moving.

[0005] Non-limiting embodiments of the present disclosure contribute to providing a signal processing device and a signal processing method that can improve the encoding performance of a stereo signal when a sound source is moving.

[0006] A signal processing device according to one embodiment of the present disclosure includes a detection circuit that detects a change over time in the inter-channel time difference of a stereo signal, and a control circuit that controls a degree of smoothing of an inter-channel correlation function based on the change over time in the inter-channel time difference.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0008] According to an embodiment of the present disclosure, it is possible to improve the coding performance of a stereo signal when a sound source is moving.

[0009] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a transmission system for audio and sound signals; [Figure 2] Block diagram showing an example of the configuration of an ITD (inter-channel time difference) estimation unit [Figure 3] Flowchart showing an example of ITD estimation processing [Figure 4] Block diagram showing an example of the configuration of an ITD estimation unit [Figure 5] Block diagram showing an example of the configuration of a single sound source movement detection unit [Figure 6] FIG. 10 is a diagram showing an example of an inter-channel phase difference spectrum. [Figure 7] Flowchart showing an example of ITD estimation processing [Figure 8] Block diagram showing an example of the configuration of an ITD estimation unit [Figure 9] Block diagram showing an example of the configuration of an ITD estimation unit DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0012] One example of stereo signal coding is binaural cue coding (BCC), in which a stereo signal including an L channel (left channel or L-ch) and an R channel (right channel or R-ch) is parameterized by binaural cues such as an inter-channel level difference (ILD), an inter-channel cross correlation (ICC), and an inter-channel time difference (ITD).

[0013] For example, the inter-channel time difference (ITD) of a stereo signal is a parameter related to the time difference between the arrival times of sound between the L channel and the R channel. For example, the ITD may be estimated based on the time lag relative to the peak position of the time-domain inter-channel correlation function (ICC) obtained by performing an inverse fast Fourier transform (IFFT) on the frequency-domain ICC determined based on the fast Fourier transform (FFT) spectra of a pair of channel signals included in the stereo signal.

[0014] To improve the accuracy of ITD estimation or achieve stable estimation, there is a method of applying inter-frame smoothing processing to ICC based on, for example, Spectral Flatness Measurement (SFM) (see, for example, Patent Document 1). For example, the stronger the tonality or periodicity of the input signal, the lower the SFM. For example, in Patent Document 1, when the input signal has stronger tonality (e.g., lower SFM), a stronger smoothing processing is applied to the ICC in the encoding device. In other words, when the input signal has stronger tonality, the ICC data of past frames is more likely to be reflected in the current frame. This improves the accuracy of determining the peak position of the ICC in the time domain corresponding to the time lag, and improves the accuracy of ITD estimation.

[0015] Here, for example, even if the actual ITD changes due to the motion (e.g., movement) of the sound source of the stereo signal, the stronger the smoothing process applied, the less likely the estimated ITD (e.g., the peak position of the ICC in the time domain) is to change due to the smoothing between frames. Therefore, for example, application of the smoothing process may reduce the accuracy of tracking the moving sound source (in other words, the accuracy of estimating the ITD).

[0016] In one embodiment of the present disclosure, a method for improving the ITD estimation accuracy and improving the coding performance when the sound source of a stereo signal moves will be described.

[0017] [Configuration example of a voice and audio signal transmission system] FIG. 1 is a diagram showing an example of the configuration of a transmission system for voice signals or acoustic signals (for example, referred to as voice and acoustic signals).

[0018] The transmission system shown in FIG. 1 may include, for example, an encoding device and a decoding device.

[0019] [Example of encoding device configuration] The encoding device may include, for example, an input device (not shown) such as a microphone, an A / D conversion device (not shown), and an encoder.

[0020] The input device, for example, outputs an input audio signal (analog signal) to an A / D conversion device. The A / D conversion device, for example, converts the input analog signal into a digital signal and outputs it to an encoder. Note that the encoding device may include multiple (for example, two) input devices and at least one A / D conversion device in order to handle stereo signals.

[0021] The encoder may include, for example, a transform unit (for example, an FFT unit) that transforms a signal from the time domain to a signal in the frequency domain, a stereo information extraction unit, a downmix unit, and a coding unit (not shown).

[0022] The conversion unit, for example, converts the stereo signals (e.g., L channel signal and R channel signal) input to the encoder from the time domain to frequency domain data (e.g., FFT spectrum) for each channel, and outputs the data to the stereo information extraction unit and the downmix unit.

[0023] The stereo information extraction unit may extract stereo information based on, for example, the FFT spectrum of each channel. As an example, the stereo information extraction unit may parameterize the stereo signal using binaural cues such as ILD, ICC, and ITD, and output the parameterized stereo signal to the downmixing unit and the encoding unit. For example, the stereo information extraction unit may include an ITD estimation unit 10 (e.g., corresponding to a signal processing device) that parameterizes the ITD. The ITD estimation unit 10 estimates, for example, an inter-channel time difference (ITD). An example of an ITD estimation method in the ITD estimation unit 10 will be described later.

[0024] The downmixing unit may perform downmixing processing based on, for example, the FFT spectrum of each channel output from the conversion unit and binaural cue parameters (including, for example, estimated ITD) output from the stereo information extraction unit, to generate a Mid signal (also referred to as an M signal) and a Side signal (also referred to as an S signal). For example, if data obtained by manipulating the FFT spectrum of the L channel is defined as "L'", the downmixing unit may perform downmixing such that M = (L' + R) / 2 and S = (L' - R) / 2, and output the M signal and S signal to the encoding unit. Here, M represents the Mid signal, S represents the Side signal, and R represents the FFT spectrum of the R channel.

[0025] Note that, in the processing of the downmixing section, an example has been described in which the FFT spectrum of the L channel is manipulated based on the R channel, but this is not limiting, and for example, the FFT spectrum of the R channel may be manipulated based on the L channel.

[0026] The encoding unit encodes, for example, the M signal and S signal output from the downmix unit and the binaural cue parameters (including, for example, the estimated ITD) output from the stereo information extraction unit, and outputs the encoded data. Note that the encoding unit is not limited to the codecs described above, and may include various standardized audio and video codecs such as those of the Moving Picture Experts Group (MPEG), 3rd Generation Partnership Project (3GPP), or International Telecommunication Union Telecommunication Standardization Sector (ITU-T).

[0027] The encoding device transmits the encoded data output from the encoding unit of the encoder to the decoding device via a communication network or a storage medium (not shown).

[0028] [Example of the configuration of a decoding device] The decoding device may include, for example, a decoder, a D / A conversion device (not shown), and an output device (not shown) such as a speaker. The decoding device receives encoded data via, for example, a communication network or a storage medium (not shown), and inputs the data to the decoder.

[0029] The decoder may comprise, for example, a decoding unit, an upmixing unit, a stereo information synthesis unit, and a transform unit (eg, an IFFT unit) for transforming signals from the frequency domain to the time domain (not shown).

[0030] For example, encoded data input to a decoder is input to a decoding unit. The decoding unit decodes the input encoded data using the codec used in the encoding device, and outputs, for example, M signals, S signals, and binaural cue parameters to an upmixing unit and a stereo information synthesis unit. The decoding unit may include various standardized audio and speech codecs, such as MPEG, 3GPP, or ITU-T.

[0031] The upmixing unit may perform upmixing processing based on the M signal and S signal output from the decoding unit. For example, the upmixing unit performs upmixing processing such that L'=M+S and R=MS, and outputs the L' signal and R signal of the FFT spectrum to the stereo information synthesis unit.

[0032] The stereo information synthesis unit may, for example, use the binaural cue parameters (including the estimated ITD) output from the decoding unit and the L' signal of the FFT spectrum output from the upmix unit to perform the inverse operation of the encoding device (e.g., the stereo information extraction unit), and output the L signal of the FFT spectrum to the conversion unit.

[0033] The conversion unit converts, for example, the L and R signals of the FFT spectrum into L and R channel digital signals in the time domain for each channel, and outputs the digital signals as output signals of the decoder.

[0034] The D / A conversion device converts, for example, the digital signal output from the decoder into an audio signal (analog signal), and outputs it to an output device.

[0035] The output device outputs the analog signal output from the D / A conversion device from, for example, a speaker. Note that the decoding device may include a plurality (for example, two) of at least one of the D / A conversion device and the output device device in order to handle stereo signals.

[0036] [Configuration example of ITD estimation unit] Next, a description will be given of an example configuration of the ITD estimation unit 10. Fig. 2 is a block diagram showing an example configuration of the ITD estimation unit 10. Fig. 3 is a flowchart showing an example operation of the ITD estimation unit 10 shown in Fig. 2.

[0037] The ITD estimation unit 10 shown in FIG. 2 may include, for example, an FFT unit 11, an ICC determination unit 12, an SFM determination unit 13, a smoothing processing unit 14, an IFFT unit 15, and an ITD detection unit 16.

[0038] For example, time-domain stereo signals (e.g., L channel and R channel) may be input to the FFT unit 11 independently for each channel. The FFT unit 11 converts, for example, the time-domain channel signals into frequency-domain signals (hereinafter referred to as FFT spectra) (e.g., S11 in FIG. 3). The FFT unit 11 outputs information about the FFT spectrum to the ICC determination unit 12 and the SFM determination unit 13. Note that the method of converting the time-domain signals into frequency-domain signals is not limited to FFT, and other methods may be used.

[0039] The ICC determination unit 12 determines (e.g., calculates) an inter-channel correlation function (ICC) (e.g., S12 in FIG. 3) based on the FFT spectrum of each channel output from the FFT unit 11. The ICC determination unit 12 outputs information about the determined ICC to the smoothing processing unit 14.

[0040] The SFM determination unit 13 determines (e.g., calculates) the spectral flatness (SFM) based on the FFT spectrum of each channel output from the FFT unit 11 (e.g., S13 in FIG. 3). The SFM determination unit 13 outputs information about the determined SFM to the smoothing processing unit 14.

[0041] The smoothing processing unit 14, for example, sets the SFM output from the SFM determination unit 13 as a smoothing coefficient and performs inter-frame smoothing processing on the ICC output from the ICC determination unit 12 (for example, S14 in FIG. 3). For example, the lower the SFM (or smoothing coefficient), the stronger the degree (or strength) of smoothing. The smoothing processing unit 14 outputs information related to the ICC after the smoothing processing to the IFFT unit 15.

[0042] For example, the IFFT unit 15 converts the ICC smoothed by the smoothing processing unit 14 from a frequency domain signal to a time domain signal. The IFFT unit 15 outputs information about the time domain ICC to the ITD detection unit 16. Note that the method for converting a frequency domain signal into a time domain signal is not limited to IFFT, and other methods may be used.

[0043] The ITD detection unit 16 (corresponding to, for example, an estimation circuit) detects (or estimates) the ITD based on, for example, the ICC in the time domain output from the IFFT unit 15 (for example, S15 in FIG. 3).

[0044] In the ITD estimation unit 10 shown in Fig. 2, for example, the degree of smoothing increases with a stereo signal with stronger tonality (for example, the lower the SFM), and as described above, there is a possibility that the accuracy of ITD estimation will decrease when the sound source is moving. Below, as an example, a method for improving the accuracy of ITD estimation even when a single sound source is moving will be described.

[0045] FIG. 4 is a block diagram showing an example of the configuration of the ITD estimation unit 10a according to this embodiment.

[0046] 4, for example, a single sound source movement detection unit 50 is added to the configuration of the ITD estimation unit 10 shown in Fig. 2. For example, in the ITD estimation unit 10a shown in Fig. 4, components different from the single sound source movement detection unit 50 may be the same as those in Fig. 2.

[0047] The single sound source movement detection unit 50 (corresponding to, for example, a detection circuit and a control circuit) may have, for example, a function to detect movement of a single sound source of a stereo signal (in other words, a time change in the ITD of a stereo signal) based on the FFT spectrum of each channel output from the FFT unit 11, and a function to control smoothing in the smoothing process of the smoothing processing unit 14 (for example, control of the degree of smoothing of the ICC).

[0048] The single sound source movement detection unit 50 may, for example, detect the movement of a single sound source, for example, a time change in the ITD of a stereo signal, and control smoothing based on the detection result of the movement of the single sound source.

[0049] FIG. 5 is a block diagram showing an example of the configuration of the single sound source movement detection unit 50. As shown in FIG.

[0050] The single sound source movement detection unit 50 shown in FIG. 5 may include an IPD determination unit 51, data selection units 52-1 and 52-2, first-order difference determination units 53-1 and 53-2, variance determination units 54-1 and 54-2, and a smoothing control unit 55, for example.

[0051] The IPD determination unit 51 determines (e.g., calculates) an inter-channel phase difference (IPD or IPD spectrum) based on, for example, the FFT spectrum (e.g., FFT phase spectrum) of each of the L channel and R channel output from the FFT unit 11. The IPD determination unit 51 may determine, for example, the IPD spectrum for each frequency bin. The IPD determination unit 51 outputs information related to the IPD to the data selection unit 52-1 and the data selection unit 52-2.

[0052] Here, IPD may be defined as the difference between the phase spectra of two channels of a stereo signal. For example, when a single sound source moves so that the inter-channel time difference (ITD) changes by about 1 sample per frame (e.g., when sampling at 32 kHz and one frame being 20 ms, the IPD spectrum tends to have a linear shape (including, for example, a sawtooth shape) appearing in the low frequency band but not in the high frequency band. In other words, when a linear shape appears in the low frequency band but not in the high frequency band in the IPD spectrum, it is highly likely that the sound source is moving alone.

[0053] Fig. 6 is a diagram showing an example of an IPD spectrum, in which the vertical axis represents the phase of the IPD spectrum and the horizontal axis represents the frequency bin of the IPD spectrum.

[0054] The IPD spectrum shown in FIG. 6 is normalized to, for example, the range of -π to +π, and wraps around (or folds back) in the range of -π to +π. Here, for example, the higher the frequency band, the greater the IPD (for example, the slope of the IPD in FIG. 6) tends to be. Also, for example, when the sound source is moving (for example, when the ITD changes over time), the IPD value is more likely to fluctuate. For this reason, as shown in FIG. 6, components with a clear linear shape (for example, a sawtooth shape) are likely to appear in the low frequency band of the IPD spectrum, while in the high frequency band of the IPD spectrum, the wraparound described above is likely to occur frequently and noise-like components are likely to appear. In other words, for example, as shown in FIG. 6, it can be seen that clear linear shapes are likely to appear in the low frequency band of the IPD spectrum and are less likely to appear in the high frequency band of the IPD spectrum (or this includes the appearance of partial linear shapes).

[0055] As described above, the single sound source movement detection unit 50 may detect the movement of the single sound source, in other words, the time change of the ITD, based on the shape of the IPD spectrum in each of the low frequency band and the high frequency band. For example, the single sound source movement detection unit 50 may determine whether or not the single sound source is moving (for example, a case where the single sound source is moving slowly) by detecting (or identifying) whether or not a linear shape (for example, the spectral shape shown in FIG. 6) appears in the phase spectrum for each of the low frequency band and the high frequency band.

[0056] For example, the single sound source movement detection unit 50 may detect the movement of a single sound source (e.g., a change in the ITD over time) based on the variance of the IPD (e.g., a first-order difference of the IPD spectrum) for the low frequency band of the stereo signal and the variance of the IPD (e.g., a first-order difference of the IPD spectrum) for the high frequency band of the stereo signal.

[0057] In Figure 5, for example, the data selection unit 52-1, first-order difference determination unit 53-1, and variance determination unit 54-1 are components that perform processing corresponding to the IPD spectrum (or IPD data) in the low frequency band, and the data selection unit 52-2, first-order difference determination unit 53-2, and variance determination unit 54-2 are components that perform processing corresponding to the IPD spectrum (or IPD data) in the high frequency band.

[0058] The data selection unit 52-1 selects data to be output to the subsequent first-order difference determination unit 53-1 from IPD data in the low frequency band of the signal output from the IPD determination unit 51, for example. For example, the single sound source movement detection unit 50 does not need to use IPD information corresponding to +π and -π when the IPD data (phase) is normalized in the range of -π to +π, for detecting the movement of the single sound source (or a time change in the ITD). For example, the data selection unit 52-1 may select IPD data in the range of -0.75π to +0.75π. In other words, the data selection unit 52-1 may remove IPD data corresponding to a wraparound portion in the IPD spectrum from the IPD data used for detecting the movement of the single sound source, as shown in FIG. 6. The data selection unit 52-1 outputs the selected data to the first-order difference determination unit 53-1.

[0059] The first-order difference determination unit 53-1 determines (e.g., calculates) the first-order difference of the IPD data in the low frequency band selected by the data selection unit 52-1 (e.g., the difference between IPD data of adjacent frequency bins), and outputs information about the first-order difference to the distribution determination unit 54-1. Note that the difference determined (or detected) by the first-order difference determination unit 53-1 is not limited to the first-order difference. For example, the gradient between the IPD data may be detected by differentiating the IPD data. This also applies to the first-order difference determination unit 53-2, which will be described later.

[0060] The variance determination unit 54-1 determines (eg, calculates) the variance of the first-order differences in the low frequency band output from the first-order difference determination unit 53-1, and outputs information about the variance of the first-order differences in the low frequency band to the smoothing control unit 55.

[0061] Similar to the data selection unit 52-1, the data selection unit 52-2 selects data to be output to the subsequent first-order difference determination unit 53-2 from, for example, the IPD data in the high frequency band of the signal output from the IPD determination unit 51. The data selection unit 52-2 outputs the selected data to the first-order difference determination unit 53-2.

[0062] Similar to the first-order difference determination unit 53-1, the first-order difference determination unit 53-2 determines (e.g., calculates) the first-order difference of the IPD data in the high frequency band selected by the data selection unit 52-2, and outputs information about the first-order difference to the distribution determination unit 54-2.

[0063] Similar to the variance determination unit 54-1, the variance determination unit 54-2 determines (e.g., calculates) the variance of the first-order differences in the high frequency band output from the first-order difference determination unit 53-2, and outputs information regarding the variance of the first-order differences in the high frequency band to the smoothing control unit 55.

[0064] Here, the distribution determination units 54-1 and 54-2 may, for example, thin out the IPD data output from the data selection units 52-1 and 52-2. For example, out of IPD[k] (k is a number assigned to the IPDs in the order they are output from the data selection units 52-1 and 52-2), every other IPD may be thinned out, such as k=1, 3, 5, . . . , 2m-1 or k=2, 4, 6, . . . , 2m, or every third IPD may be thinned out. The distribution determination units 54-1 and 54-2 may determine the distribution based on the thinned IPD data, for example. Thinning out the IPD data reduces the amount of calculation in the distribution determination units 54-1 and 54-2. Note that the method of thinning out the IPD data is not limited to thinning out every other IPD or every third IPD, as described above, and other methods may be used. Furthermore, for example, the dispersion determiner 54-1 and the dispersion determiner 54-2 may calculate dispersion in a specified band (for example, 100 Hz width or 200 Hz width) near the center of at least one of the low frequency band and the high frequency band.

[0065] The smoothing control unit 55 determines (e.g., calculates) a smoothing coefficient based on, for example, the variance of the first-order differences in the low frequency band output from the variance determination unit 54-1, the variance of the first-order differences in the high frequency band output from the variance determination unit 54-2, and the SFM output from the SFM determination unit 13. The smoothing control unit 55 outputs information on the determined smoothing coefficient to the smoothing processing unit 14.

[0066] For example, in Patent Document 1, SFM is set as a smoothing coefficient (for example, represented as "alpha"). In this embodiment, for example, the smoothing coefficient alpha may be calculated based on the following equation (1). alpha = Max(SFM, 1-VL / VH) (1)

[0067] Here, the function Max(A, B) is a function that outputs the larger value between A and B. Furthermore, VL represents the variance of the low frequency band determined by the variance determination unit 54-1, and VH represents the variance of the high frequency band determined by the variance determination unit 54-2.

[0068] For example, as shown in Figure 6, in the case of a sound source (e.g., a single moving sound source) that has a linear shape in the low frequency band but not in the high frequency band, the variance VL in the low frequency band tends to be low and the variance VH in the high frequency band tends to be high. In this case, in equation (1), the value of 1-VL / VH tends to be high (e.g., closer to 1), and the value of alpha tends to be close to 1.

[0069] Here, the case of alpha=1 corresponds to the case where smoothing processing is not applied. Therefore, the smoothing control unit 55 weakens the degree (or strength) of smoothing when, for example, movement of a single sound source (for example, the shape of the IPD spectrum as shown in FIG. 6) is detected, compared to the degree of smoothing when movement of a single sound source is not detected.

[0070] As a result, for example, when a single sound source moves, the smoothing process in the smoothing processor 14 is weakened; in other words, the influence of the ICC of the past frame is reduced. Therefore, the ITD detector 16 can estimate an ITD that reflects instantaneous changes in the ICC due to the movement of the single sound source. Therefore, the ITD estimation unit 10a can improve the accuracy of ITD estimation even when the single sound source moves.

[0071] Alternatively, the smoothing control unit 55 may determine the smoothing coefficient alpha based on, for example, a comparison between the variance of the first-order difference of the IPD spectrum and a threshold. In other words, the smoothing control unit 55 may detect the movement of a single sound source based on, for example, a comparison between the variance of the first-order difference of the IPD spectrum and a threshold, and determine the smoothing coefficient alpha based on the detection result of the movement of the single sound source.

[0072] For example, when the variance VL of the low frequency band and the variance VH of the high frequency band satisfy a predetermined condition, the smoothing control unit 55 may determine that the sound source of the stereo signal is moving as a single unit (or the ITD is changing over time), and may weaken the degree of smoothing compared to the degree of smoothing when the condition is not satisfied. Note that weakening the degree of smoothing may include, for example, not performing smoothing.

[0073] For example, the smoothing control unit 55 may set the smoothing coefficient alpha=1 when the conditions VL < Th1 and VH / VL > Th2 are satisfied in a specified interval (e.g., five consecutive frames), and may set the smoothing coefficient alpha=SFM when the conditions VL < Th1 and VH / VL > Th2 are not satisfied in the specified interval.

[0074] 6, for example, the variance VL of the first-order difference of the IPD spectrum (e.g., linear shape) in the low frequency band tends to be small, while the variance VH of the first-order difference of the IPD spectrum in the high frequency band tends to be large. This makes it easy to satisfy the condition that the variance VL is smaller than the threshold value Th1 (VL < Th1) and the ratio of the variance VH to the variance VL is greater than the threshold value Th2 (VH / VL > Th2). Therefore, if the conditions VL < Th1 and VH / VL > Th2 are satisfied in the specified section, the smoothing control unit 55 may determine that movement of a single sound source has been detected and set the smoothing coefficient alpha to 1, i.e., decide not to perform smoothing. This allows the ITD estimation unit 10a to improve the accuracy of ITD estimation even when a single sound source is moving.

[0075] On the other hand, in the specified interval, when the conditions VL < Th1 and VH / VL > Th2 are not satisfied, the smoothing control unit 55 may determine that the movement of a single sound source is not detected, and determine to execute smoothing with the smoothing coefficient alpha = SFM. Further, after alpha is once set to alpha = 1, when either one of the conditions VL < Th1 and VH / VL > Th2 is not satisfied in the specified interval (for example, consecutive 5 frames), for example, it is determined that the movement of a single sound source is completed, and alpha may be set (or reset, switched) to alpha = SFM. Thereby, for example, when the sound source does not move, the ITD estimation unit 10a can improve the ITD estimation accuracy by smoothing the ICC for a signal with a strong tonality.

[0076] Note that, among the above-described conditions, instead of VH / VL > Th2, VL / VH < Th3 (for example, Th3 = 1 / Th2) may be applied. Here, Th1 and Th2 are threshold values. For example, Th1 may be set to 2.25 and Th2 may be set to 1.50. Note that the set values of Th1 and Th2 are not limited to these, and other values may be used.

[0077] Also, here, as an example, it is assumed that 1 frame = 20 ms, and the case where the specified interval is 5 frames (for example, 100 ms) has been described. In this case, the switching of the smoothing coefficient alpha in the determination process using the above-described threshold values can be performed every 100 ms at the shortest. Thereby, since the smoothing control unit 55 determines the movement of a single sound source based on the shape of the IPD spectrum over the specified interval, for example, even when there is a possibility of misjudging the detection of the movement of a single sound source such as detecting the movement of the sound source due to VH becoming large by wraparound in some frames within the specified interval when a single sound source with a strong periodicity having a certain phase difference does not move, the occurrence of incorrect switching of the smoothing process (for example, the smoothing coefficient alpha) can be suppressed. Note that the specified interval is not limited to 100 ms (or 5 frames), and other values may be used. For example, the specified interval may be determined according to the mode switching period in the stereo encoding system.

[0078] The smoothing processing unit 14 may perform smoothing processing on the ICC output from the ICC determination unit 12, using the smoothing coefficient alpha output from the single sound source movement detection unit 50. For example, the smoothing processing may be performed based on the following equation (2). ICCsmooth(t)[n] = (1-alpha)*ICCsmooth(t-1)[n]+alpha*ICC[n] (2)

[0079] Here, ICCsmooth(t)[n] denotes the nth element of the ICC to be smoothed at time t (or the tth frame), alpha denotes the smoothing coefficient determined by the smoothing control unit 55, and ICC[n] denotes the nth element of the ICC at the current time (or the current frame).

[0080] Then, the ITD detection unit 16 may estimate the ITD based on, for example, the ICC with the degree of smoothing controlled.

[0081] 7 is a flowchart showing an example of the operation of the ITD estimation process according to the present embodiment. Note that the processes of S11 to S15 shown in FIG. 7 are the same as the processes of S11 to S15 shown in FIG.

[0082] In FIG. 7, the ITD estimation unit 10a calculates an IPD spectrum based on, for example, the FFT spectrum of each of the L channel and R channel of a stereo signal (S51).

[0083] The ITD estimation unit 10a calculates a first-order difference based on, for example, the IPD spectrum (S52). Furthermore, the ITD estimation unit 10a calculates a variance of the first-order difference in the low frequency band (for example, VL) and a variance of the first-order difference in the high frequency band (for example, VH) based on, for example, the first-order difference of the IPD spectrum (S53).

[0084] The ITD estimation unit 10a determines whether or not the conditions VL<Th1 and VH / VL>Th2 are satisfied in a specified section (for example, five consecutive frames) (S54).

[0085] If the condition is met (S54: Yes), the ITD estimation unit 10a does not smooth the ICC (for example, sets alpha=1), or performs weak smoothing on the ICC (for example, sets alpha based on equation (1)) (S55). On the other hand, if the condition is not met (S54: No), for example, if the possibility of a single sound source moving is low, the ITD estimation unit 10a smooths the ICC based on SFM (S14).

[0086] As described above, according to the present embodiment, the ITD estimation unit 10a includes the single sound source movement detection unit 50 and detects the movement of the single sound source of the stereo signal (time change in ITD). The ITD estimation unit 10a controls smoothing of the ICC over multiple frames (sections) based on, for example, information related to the movement of the single sound source of the stereo signal (for example, the detection result).

[0087] This allows the ITD estimation unit 10a to improve robustness against time-dependent changes in the ITD when, for example, a single sound source moves. In other words, the ITD estimation unit 10a can improve, for example, the accuracy of tracking a moving sound source (for example, the temporal tracking ability of the ITD). Therefore, according to this embodiment, even when a single sound source of a stereo signal moves, it is possible to improve the ITD estimation accuracy and improve coding performance.

[0088] (Embodiment 2) In the ITD estimation unit 10a according to the present embodiment, for example, the configuration of the single sound source movement detection unit 60 is different from that of the first embodiment, but the other configurations may be the same as those of the first embodiment.

[0089] Fig. 8 is a block diagram showing an example of the configuration of the single sound source movement detection unit 60 according to this embodiment. The single sound source movement detection unit 60 shown in Fig. 8 includes a data selection unit 61-1 and a data selection unit 61-2 in addition to the same configuration as the single sound source movement detection unit 50.

[0090] The data selection unit 61-1 may be provided, for example, between the first-order difference determination unit 53-1 and the variance determination unit 54-1. The data selection unit 61-1 may select data by removing outliers from the first-order differences in the low frequency band, for example.

[0091] The removal of outliers may be achieved, for example, by setting upper and lower limits (in other words, setting boundaries) for the data (e.g., first-order differences of IPD spectra) selected by the data selection unit 61-1. For example, the upper limit of the data may be set to Dmean+π / 2, and the lower limit of the data may be set to Dmean-π / 2, where Dmean represents the mean value of the first-order differences.

[0092] The data selection unit 61-2 may be provided, for example, between the first-order difference determination unit 53-2 and the variance determination unit 54-2, similar to the data selection unit 61-1. The data selection unit 61-2 may select data by removing outliers from the first-order differences in the high frequency band, for example.

[0093] In this way, the single sound source movement detection unit 60 selects first-order difference data to be used for detecting the movement of a single sound source (for example, a time change in ITD) based on, for example, the average value Dmean of the first-order differences of the IPD spectrum (for example, in the above example, in the range of -0.75π to +0.75π).

[0094] This data selection (or outlier removal) can improve the accuracy of, for example, the first-order difference of the IPD spectrum (for example, the slope component of the IPD in the frequency domain), and therefore the ITD estimation unit 10a can improve the accuracy of determining the shape of the IPD spectrum when a single sound source moves (for example, the accuracy of detecting the movement of the single sound source).As a result, according to the present embodiment, the ITD estimation accuracy can be improved compared to, for example, the first embodiment, and coding performance can be improved.

[0095] In this embodiment, the single sound source movement detection unit 60 may switch between applying and not applying first-order differential data selection in the data selection unit 61-1 and the data selection unit 61-2, for example.

[0096] The embodiments of the present disclosure have been described above.

[0097] [Modification of smoothing control] For example, smoothing control may be performed based on SFM (or information about tonality).

[0098] Fig. 9 is a block diagram showing an example configuration of an ITD estimation unit 10b according to a modified example. The ITD estimation unit 10b shown in Fig. 9 may include, for example, a determination unit 71 in addition to the configuration of the ITD estimation unit 10a according to the first embodiment. Note that the ITD estimation unit 10b may include the single sound source movement detection unit 50 according to the first embodiment, or the single sound source movement detection unit 60 according to the second embodiment.

[0099] In FIG. 9, the determining unit 71 may determine whether or not to perform smoothing control (for example, determination of the smoothing coefficient alpha) by the single sound source movement detecting unit 50 based on information about the SFM input from the SFM determining unit 13, for example.

[0100] Here, the weaker the tonality of a stereo signal, the higher the SFM tends to be, and the more difficult it is to smooth the ICC by SFM. Therefore, when the SFM is high (for example, when the SFM is equal to or greater than a threshold), as in the case of a stereo signal with weak tonality, the smoothing control of the single sound source movement detection unit 50 may not be as effective in improving the ITD estimation accuracy as when the SFM is low (for example, when the SFM is less than a threshold), as in the case of a stereo signal with strong tonality.

[0101] Therefore, for example, when the SFM is equal to or greater than a threshold, the determination unit 71 may decide not to execute smoothing control by the single sound source movement detection unit 50. In this case, the single sound source movement detection unit 50 may set the SFM output from the SFM determination unit 13 as a smoothing coefficient (for example, alpha=SMF).

[0102] On the other hand, for example, when the SFM is less than the threshold value, the determination unit 71 may decide to execute smoothing control by the single sound source movement detection unit 50. In this case, for example, the single sound source movement detection unit 50 may perform smoothing control of the ICC (for example, determine the smoothing coefficient alpha) based on the detection of the movement of the single sound source, similar to the first embodiment.

[0103] In this way, smoothing control based on SFM can switch whether or not to apply smoothing control based on the detection of the movement of a single sound source (in other words, whether or not to bypass smoothing control) depending on the tonality of a stereo signal, which can, for example, simplify or improve the efficiency of smoothing control.

[0104] [Low and high frequency band settings] For example, when a single sound source is moving, wraparound of the IPD spectrum (phase) tends to occur frequently at high frequencies.

[0105] For example, when detecting the movement of a single sound source, the low frequency band and the high frequency band may be set to frequencies that are lower than the frequency band in which wraparound is likely to occur for both the low frequency band and the high frequency band.

[0106] For example, the low frequency band may be set to 0 to 8 kHz, and the high frequency band may be set to 8 kHz to 16 kHz. Other examples of the low frequency band and the high frequency band may include 0 to 2 kHz and 2 kHz to 4 kHz, 0 to 3 kHz and 3 kHz to 6 kHz, or 0 to 4 kHz and 4 kHz to 8 kHz.

[0107] The settings of the low frequency band and the high frequency band are not limited to these examples, and other setting values ​​may be used.

[0108] Furthermore, for example, the low frequency band and the high frequency band may be set to frequency bands that are separate from each other, or may be set to frequency bands that partially overlap, and each band may have a different bandwidth.

[0109] Furthermore, for example, the setting of at least one of the low frequency band and the high frequency band (e.g., at least one of the frequency position and the bandwidth) may be variable. For example, the setting of the frequency band may be determined (or changed) based on analysis results such as the type of stereo signal (e.g., speech signal or acoustic signal), the position of the sound source, or the dominant frequency band in the signal. Alternatively, for example, the setting of the frequency band may be determined based on the average value of the first-order differences of the IPD spectrum.

[0110] An example of setting the low frequency band and the high frequency band has been described above.

[0111] Furthermore, in each of the above-described embodiments, the ITD estimation unit 10a has been described as detecting the movement of a single sound source of a stereo signal based on the inter-channel phase difference (IPD). However, the method for detecting the movement of a single sound source of a stereo signal is not limited to this, and the movement of a single sound source may be detected by other methods.

[0112] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. Furthermore, the components in the above-described embodiments may be combined in any manner.

[0113] Furthermore, the notation "··· part" in the above-described embodiments may be replaced with other notations such as "··· circuitry," "··· device," "··· unit," or "··· module."

[0114] The present disclosure can be realized by software, hardware, or software linked to hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0115] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0116] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology can be used to integrate functional blocks. The application of biotechnology is also a possibility.

[0117] The present disclosure may be implemented in any type of apparatus, device, or system (collectively referred to as a communications apparatus) that has a communications function. The communications apparatus may include a wireless transceiver and processing / control circuitry. The wireless transceiver may include a receiver and a transmitter, or both functions. The wireless transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communication devices include telephones (e.g., cell phones, smartphones), tablets, personal computers (PCs) (e.g., laptops, desktops, notebooks), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smartwatches, tracking devices), game consoles, digital book readers, telehealth / telemedicine devices, communication-enabled vehicles or mobile transportation (e.g., cars, airplanes, ships), and combinations of the above devices.

[0118] Communications equipment is not limited to portable or mobile equipment, but also includes non-portable or fixed equipment, devices, and systems of any kind, such as smart home devices (such as appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0119] Communications include data communications via cellular systems, wireless LAN (Local Area Network) systems, communication satellite systems, and the like, as well as data communications via combinations of these.

[0120] A communications apparatus also includes devices such as controllers and sensors connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0121] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these.

[0122] A signal processing device according to one embodiment of the present disclosure includes a detection circuit that detects a change over time in the inter-channel time difference of a stereo signal, and a control circuit that controls the degree of smoothing of an inter-channel correlation function based on the change over time in the inter-channel time difference.

[0123] In one embodiment of the present disclosure, the apparatus further comprises an estimation circuit that estimates the inter-channel time difference based on the inter-channel correlation function with the degree of smoothing controlled.

[0124] In one embodiment of the present disclosure, the detection circuit detects a change over time in the inter-channel time difference based on a first variance of the inter-channel phase difference for a first band of the stereo signal and a second variance of the inter-channel phase difference for a second band of the stereo signal.

[0125] In one embodiment of the present disclosure, the control circuit determines that the sound source of the stereo signal is moving as a single entity when the first variance and the second variance satisfy a predetermined condition, and weakens the degree of smoothing compared to the degree of smoothing when the condition is not satisfied.

[0126] In one embodiment of the present disclosure, weakening the degree of smoothing includes not performing the smoothing.

[0127] In one embodiment of the present disclosure, the second band is a band higher than the first band, and the condition is that the first variance is smaller than a first threshold and the ratio of the second variance to the first variance is greater than a second threshold.

[0128] In one embodiment of the present disclosure, the detection circuit does not use information on the inter-channel phase difference corresponding to +π and -π when the inter-channel phase difference of the stereo signal is normalized in the range of -π to +π, to detect the change over time in the inter-channel time difference.

[0129] In one embodiment of the present disclosure, the detection circuit selects a first-order difference of inter-channel phase differences to be used for detecting a time change in the inter-channel time difference based on an average value of first-order differences of inter-channel phase differences of the stereo signal.

[0130] In a signal processing method according to one embodiment of the present disclosure, a signal processing device detects a change over time in the inter-channel time difference of a stereo signal, and controls a degree of smoothing of an inter-channel correlation function based on the change over time in the inter-channel time difference.

[0131] The disclosures of U.S. Provisional Application No. 63 / 138,648 filed on January 18, 2021, the disclosures of U.S. Provisional Application No. 63 / 141,198 filed on January 25, 2021, and the disclosures of the specification, drawings, and abstract contained in Japanese Patent Application No. 2021-078567 filed on May 6, 2021 are all incorporated by reference into this application. [Industrial Applicability]

[0132] An embodiment of the present disclosure is useful for coding systems and the like. [Explanation of symbols]

[0133] 10,10a,10b ITD estimation part 11 FFT section 12 ICC Decision Department 13 SFM decision section 14 Smoothing processing section 15 IFFT section 16 ITD detection unit 50,60 Single sound source movement detection unit 51 IPD Determination Department 52,61 Data selection section 53 First-order difference determination unit 54 Variance determination part 55 Smoothing control section 71 Judgment section

Claims

1. a detection circuit for detecting a time change in the time difference between channels of a stereo signal; a control circuit that controls a degree of smoothing of the inter-channel correlation function based on a time change of the inter-channel time difference; Equipped with the detection circuit detects a time change in the inter-channel time difference based on a first variance of the inter-channel phase difference for a first band of the stereo signal and a second variance of the inter-channel phase difference for a second band of the stereo signal; the control circuit determines that the sound source of the stereo signal is moving as a single unit when the first variance and the second variance satisfy a predetermined condition, and weakens the degree of smoothing compared to the degree of smoothing when the condition is not satisfied. Signal processing device.

2. an estimation circuit for estimating the inter-channel time difference based on the inter-channel correlation function with the degree of smoothing controlled; The signal processing device according to claim 1 .

3. The weakening of the degree of smoothing includes not performing the smoothing. The signal processing device according to claim 1 .

4. the second band is a band higher than the first band, and the condition is that the first variance is smaller than a first threshold and a ratio of the second variance to the first variance is greater than a second threshold. The signal processing device according to claim 1 .

5. the detection circuit does not use information on the inter-channel phase difference corresponding to +π and −π when the inter-channel phase difference is normalized in the range of −π to +π, for detecting a change over time in the inter-channel time difference; The signal processing device according to claim 1 .

6. the detection circuit selects a first-order difference of the inter-channel phase differences to be used for detecting a time change in the inter-channel time difference based on an average value of the first-order differences of the inter-channel phase differences. The signal processing device according to claim 1 .

7. The signal processing device Detects the time difference between channels of a stereo signal over time, controlling a degree of smoothing of the inter-channel correlation function based on a time change of the inter-channel time difference; detecting a time change in the inter-channel time difference based on a first variance of the inter-channel phase difference for a first band of the stereo signal and a second variance of the inter-channel phase difference for a second band of the stereo signal; determining that the sound source of the stereo signal is moving as a single unit when the first variance and the second variance satisfy a predetermined condition, and weakening the degree of smoothing compared to the degree of smoothing when the condition is not satisfied; Signal processing methods.

Images