Processing device and processing method

Abstract

To provide a processing device and a processing method capable of generating a well-balanced filter.SOLUTION: In a measuring apparatus for measuring space acoustic transfer characteristics, a processing device 201 which performs arithmetic processing for properly generating a space acoustic filter comprises: a frequency characteristic acquisition unit 214 which acquires the frequency characteristics of a sound collection signal; a smoothing unit 215 which performs smoothing to generate second spectral data smoother than first spectral data based on the frequency characteristics; a first compression unit 217 which calculates a first difference value corresponding to the difference between the second spectral data and the first spectral data in a first band and compresses the second spectral data on the basis of the first difference value; and a filter generation unit 221 which generates a filter on the basis of the second spectral data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a processing device and a processing method. [Background technology]

[0002] One sound image localization technology is out-of-head localization, which uses headphones to localize a sound image outside the listener's head. Out-of-head localization technology localizes a sound image outside the listener's head by canceling the characteristics from the headphones to the ears (headphone characteristics) and providing two characteristics (spatial acoustic transfer characteristics) from one speaker (monaural speaker) to the ears.

[0003] In out-of-head localization playback using stereo speakers, a measurement signal (impulse sound, etc.) emitted from two-channel (hereafter referred to as ch) speakers is recorded by a microphone (hereafter referred to as mic) placed at the ear of the listener. Then, a processing device generates a filter based on the collected signal obtained by collecting the measurement signal. Out-of-head localization playback can be achieved by convolving the generated filter with the 2ch audio signal.

[0004] Furthermore, in order to generate a filter (also called an inverse filter) that cancels the characteristics from the headphones to the ears, the characteristics from the headphones to the ears and eardrum (also called the ear canal transfer function ECTF or ear canal transfer characteristics) are measured using a microphone placed at the listener's own ear.

[0005] Patent Document 1 discloses a device that performs out-of-head localization processing. Furthermore, in Patent Document 1, the out-of-head localization processing performs DRC (Dynamic Range Compression) processing on the playback signal, and the processing device smoothes the frequency characteristics in a stage prior to the DRC processing. Furthermore, the processing device performs band division based on the smoothed characteristics. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-62430 A Summary of the Invention [Problem to be solved by the invention]

[0007] Out-of-head localization processing uses a spatial audio filter obtained from the spatial audio transfer characteristics for the number of speakers, and an inverse filter calculated from the ECTF of the headphones. To maximize the out-of-head localization effect, it is ideal to use a spatial audio filter that is as measured as possible and an accurate inverse filter.

[0008] However, the frequency amplitude characteristics obtained by measuring with a microphone often have steep peaks (narrow band parts with very high levels) and dips (narrow band parts with very low levels), which often cause the processed signal to be clipped.

[0009] The levels and frequencies of the peaks and dips change due to various factors. For example, the level and frequency change depending on the characteristics of the speaker at the measurement position, the acoustic characteristics of the room, the characteristics of the headphones, etc. The level and frequency also change depending on the shape of an individual's head and ears. For this reason, it was necessary to check the characteristics of the equipment used at the time of measurement each time and make adjustments to suit that equipment while listening to them.

[0010] Therefore, if the correction amount (compression amount) in the compression process is too large, the balance of personal characteristics of each individual will be lost, which may result in the balance of localization being lost and the effect of out-of-head localization being lost.

[0011] Furthermore, to accurately measure individual characteristics in the low frequency range, it is necessary to extend the microphone pickup time. If the subject moves during measurement, the individual characteristics will change. Therefore, it is difficult to generate a well-balanced filter.

[0012] The present disclosure has been made in consideration of the above-mentioned points, and has an object to provide a processing device and a processing method capable of generating a well-balanced filter. [Means for solving the problem]

[0013] The processing device according to the present embodiment includes a frequency characteristic acquisition unit that acquires frequency characteristics of an input signal, a smoothing processing unit that performs a smoothing process so as to generate second spectral data that is smoother than first spectral data based on the frequency characteristics, a first compression unit that calculates a first difference value corresponding to a difference between the second spectral data and the first spectral data in a first band and compresses the second spectral data based on the first difference value, and a filter generation unit that generates a filter based on the second spectral data.

[0014] The processing method according to the present embodiment includes the steps of acquiring frequency characteristics of an input signal, performing a smoothing process to generate second spectral data that is smoother than first spectral data based on the frequency characteristics, calculating a first difference value corresponding to a difference between the second spectral data and the first spectral data in a first band, compressing the second spectral data based on the first difference value, and generating a filter based on the second spectral data. Effect of the Invention

[0015] According to the present disclosure, it is possible to provide a processing device and a processing method capable of generating a well-balanced filter. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an extra-head localization processing device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram illustrating a schematic configuration of a measuring device. [Diagram 3] FIG. 2 is a block diagram showing a configuration of a processing device. [Figure 4]11 is a graph for explaining the first compression process. [Diagram 5] 11 is a graph showing a spectrum obtained in the first compression process. [Figure 6] 11 is a graph for explaining a second compression process. [Figure 7] 13 is a graph showing a spectrum obtained in the second compression process. [Figure 8] 1 is a flowchart illustrating a processing method according to an embodiment. [Figure 9] 4 is a graph showing spectral data compressed in a first compression process. [Figure 10] 4 is a graph showing spectral data compressed in a first compression process. [Figure 11] 4 is a graph showing spectral data compressed in a first compression process. [Figure 12] 4 is a graph showing spectral data compressed in a first compression process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] An outline of the sound image localization process according to this embodiment will be described. The out-of-head localization process according to this embodiment is performed using the spatial acoustic transfer characteristic and the ear canal transfer characteristic. The spatial acoustic transfer characteristic is the transfer characteristic from a sound source such as a speaker to the ear canal. The ear canal transfer characteristic is the transfer characteristic from a speaker unit of a headphone or earphone to the eardrum. In this embodiment, the spatial acoustic transfer characteristic is measured when the headphone or earphone is not worn, and the ear canal transfer characteristic is measured when the headphone or earphone is worn, and the out-of-head localization process is realized using the measurement data. This embodiment is characterized by a microphone system for measuring the spatial acoustic transfer characteristic or the ear canal transfer characteristic.

[0018] The out-of-head localization processing according to this embodiment is executed by a user terminal such as a personal computer, a smartphone, or a tablet PC. The user terminal is an information processing device having a processing means such as a processor, a storage means such as a memory or a hard disk, a display means such as a liquid crystal monitor, and an input means such as a touch panel, a button, a keyboard, or a mouse. The user terminal may have a communication function for transmitting and receiving data. Furthermore, an output means (output unit) having headphones or earphones is connected to the user terminal. The connection between the user terminal and the output means may be a wired connection or a wireless connection.

[0019] Embodiment 1 (Extracranial Localization Processing Device) FIG. 1 shows a block diagram of an out-of-head localization processing device 100, which is an example of a sound field reproduction device according to this embodiment. The out-of-head localization processing device 100 reproduces a sound field for a user U wearing headphones 43. For this purpose, the out-of-head localization processing device 100 performs sound image localization processing on Lch and Rch stereo input signals XL and XR. The Lch and Rch stereo input signals XL and XR are analog audio reproduction signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). The audio reproduction signals or digital audio data are collectively referred to as reproduction signals. In other words, the Lch and Rch stereo input signals XL and XR are reproduction signals.

[0020] The extrahead localization processing device 100 is not limited to being a single physical device, and some of the processing may be performed by different devices. For example, some of the processing may be performed by a smartphone or the like, and the remaining processing may be performed by a DSP (Digital Signal Processor) built into the headphones 43.

[0021] The out-of-head localization processing device 100 includes an out-of-head localization processing unit 10, a filter unit 41 that stores an inverse filter Linv, a filter unit 42 that stores an inverse filter Rinv, and a headphone 43. Specifically, the out-of-head localization processing unit 10, the filter unit 41, and the filter unit 42 can be realized by a processor or the like.

[0022] The out-of-head localization processing unit 10 includes convolution calculation units 11-12, 21-22 that store spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs, and adders 24 and 25. The convolution calculation units 11-12, 21-22 perform convolution processing using the spatial acoustic transfer characteristics. Stereo input signals XL and XR from a CD player or the like are input to the out-of-head localization processing unit 10. The spatial acoustic transfer characteristics are set in the out-of-head localization processing unit 10. The out-of-head localization processing unit 10 convolves the stereo input signals XL and XR of each channel with a filter of the spatial acoustic transfer characteristics (hereinafter also referred to as a spatial acoustic filter). The spatial acoustic transfer characteristics may be head-related transfer functions HRTF measured on the head or auricle of the subject, or may be head-related transfer functions of a dummy head or a third party.

[0023] A set of four spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs is defined as a spatial acoustic transfer function. Data used for convolution in the convolution calculation units 11, 12, 21, and 22 becomes a spatial acoustic filter. The spatial acoustic filter is generated by cutting out the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length.

[0024] The spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are acquired in advance by impulse response measurement or the like. For example, a user U wears microphones on the left and right ears, respectively. Left and right speakers arranged in front of the user U output impulse sounds for performing impulse response measurement. Then, a measurement signal such as the impulse sound output from the speakers is picked up by a microphone. The spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are acquired based on the pickup signal by the microphone. The spatial acoustic transfer characteristic Hls between the left speaker and the left microphone, the spatial acoustic transfer characteristic Hlo between the left speaker and the right microphone, the spatial acoustic transfer characteristic Hro between the right speaker and the left microphone, and the spatial acoustic transfer characteristic Hrs between the right speaker and the right microphone are measured.

[0025] Then, the convolution calculation unit 11 convolves the Lch stereo input signal XL with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hls. The convolution calculation unit 11 outputs the convolution calculation data to the adder 24. The convolution calculation unit 21 convolves the Rch stereo input signal XR with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hro. The convolution calculation unit 21 outputs the convolution calculation data to the adder 24. The adder 24 adds the two convolution calculation data and outputs the result to the filter unit 41.

[0026] The convolution calculation unit 12 convolves the Lch stereo input signal XL with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hlo. The convolution calculation unit 12 outputs the convolution calculation data to the adder 25. The convolution calculation unit 22 convolves the Rch stereo input signal XR with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hrs. The convolution calculation unit 22 outputs the convolution calculation data to the adder 25. The adder 25 adds the two convolution calculation data and outputs the result to the filter unit 42.

[0027] Inverse filters Linv, Rinv that cancel headphone characteristics (characteristics between the playback unit of the headphones and the microphone) are set in the filter units 41, 42. The inverse filters Linv, Rinv are convolved with the playback signal (convolution operation signal) that has been processed by the out-of-head localization processing unit 10. The filter unit 41 convolves the inverse filter Linv of the Lch headphone characteristics with the Lch signal from the adder 24. Similarly, the filter unit 42 convolves the inverse filter Rinv of the Rch headphone characteristics with the Rch signal from the adder 25. The inverse filters Linv, Rinv cancel the characteristics from the headphone unit to the microphone when the headphones 43 are worn. The microphone may be placed anywhere between the entrance of the ear canal and the eardrum.

[0028] The filter unit 41 outputs the processed Lch signal YL to the left unit 43L of the headphones 43. The filter unit 42 outputs the processed Rch signal YR to the right unit 43R of the headphones 43. A user U wears the headphones 43. The headphones 43 output the Lch signal YL and the Rch signal YR (hereinafter, the Lch signal YL and the Rch signal YR are also collectively referred to as stereo signals) toward the user U. This makes it possible to reproduce a sound image localized outside the head of the user U.

[0029] In this way, the out-of-head localization processing device 100 performs out-of-head localization processing using the spatial audio filters according to the spatial audio transfer characteristics Hls, Hlo, Hro, and Hrs, and the inverse filters Linv and Rinv of the headphone characteristics. In the following description, the spatial audio filters according to the spatial audio transfer characteristics Hls, Hlo, Hro, and Hrs, and the inverse filters Linv and Rinv of the headphone characteristics are collectively referred to as the out-of-head localization processing filters. In the case of a 2ch stereo reproduction signal, the out-of-head localization filters are composed of four spatial audio filters and two inverse filters. The out-of-head localization processing device 100 then performs convolution calculation processing on the stereo reproduction signal using a total of six out-of-head localization filters, thereby executing the out-of-head localization processing. It is preferable that the out-of-head localization filters are based on the individual measurement of the user U. For example, the out-of-head localization filters are set based on a pickup signal picked up by a microphone attached to the ear of the user U.

[0030] In this way, the spatial audio filter and the headphone characteristic inverse filters Linv, Rinv are filters for audio signals. These filters are convoluted with the playback signal (stereo input signals XL, XR), so that the out-of-head localization processing device 100 executes out-of-head localization processing. In this embodiment, the process of generating the spatial audio filter is one of the technical features. Specifically, in the process of generating the spatial audio filter, a level range control process (Level Range Control, hereinafter, LRC process) is performed to range-compress the gain level of the spectrum data in the frequency characteristic. Here, the level width between the minimum gain level and the maximum gain level of the spectrum data in the frequency characteristic is called the level range.

[0031] (Spatial acoustic transmission characteristic measuring device) A measurement device 200 for measuring spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs will be described with reference to Fig. 2. Fig. 2 is a diagram showing a schematic measurement configuration for measuring a subject 1. Note that, in this example, the subject 1 is the same person as the user U in Fig. 1, but may be a different person.

[0032] As shown in Fig. 2, the measurement device 200 has a stereo speaker 5 and a microphone unit 2. The stereo speaker 5 is installed in a measurement environment. The measurement environment may be a room in the user U's home, an audio system sales store or showroom, or the like. The measurement environment is preferably a listening room equipped with speakers and good acoustics.

[0033] In this embodiment, the processing device 201 of the measuring device 200 performs arithmetic processing for appropriately generating a spatial acoustic filter. The processing device 201 has, for example, a music player such as a CD player. The processing device 201 may be a personal computer (PC), a tablet terminal, a smartphone, or the like. Moreover, the processing device 201 may be a server device itself.

[0034] The stereo speaker 5 includes a left speaker 5L and a right speaker 5R. For example, the left speaker 5L and the right speaker 5R are placed in front of the subject 1. The left speaker 5L and the right speaker 5R output impulse sounds and the like for performing impulse response measurement. In the following, in this embodiment, the number of speakers serving as sound sources is described as two (stereo speakers), but the number of sound sources used for measurement is not limited to two, and may be one or more. In other words, this embodiment can be similarly applied to a so-called multi-channel environment such as 1ch monaural, 5.1ch, 7.1ch, etc.

[0035] The microphone unit 2 is a stereo microphone having a left microphone 2L and a right microphone 2R. The left microphone 2L is installed at the left ear 9L of the subject 1, and the right microphone 2R is installed at the right ear 9R of the subject 1. Specifically, it is preferable to install the microphones 2L and 2R at a position from the entrance of the ear canal to the eardrum of the left ear 9L and the right ear 9R. The microphones 2L and 2R collect the measurement signal output from the stereo speaker 5 to obtain the collected sound signal. The microphones 2L and 2R output the collected sound signal to the processing device 201. The subject 1 may be a person or a dummy head. That is, in this embodiment, the subject 1 is a concept that includes not only a person but also a dummy head.

[0036] As described above, the impulse response is measured by measuring the impulse sound output from the left speaker 5L and the right speaker 5R with the microphones 2L and 2R. The processing device 201 stores the collected sound signal acquired by the impulse response measurement in a memory or the like. As a result, the spatial acoustic transfer characteristic Hls between the left speaker 5L and the left microphone 2L, the spatial acoustic transfer characteristic Hlo between the left speaker 5L and the right microphone 2R, the spatial acoustic transfer characteristic Hro between the right speaker 5R and the left microphone 2L, and the spatial acoustic transfer characteristic Hrs between the right speaker 5R and the right microphone 2R are measured. That is, the spatial acoustic transfer characteristic Hls is acquired by the left microphone 2L collecting the measurement signal output from the left speaker 5L. The spatial acoustic transfer characteristic Hlo is acquired by the right microphone 2R collecting the measurement signal output from the left speaker 5L. The spatial acoustic transfer characteristic Hro is acquired by the left microphone 2L collecting the measurement signal output from the right speaker 5R. The measurement signal output from the right speaker 5R is picked up by the right microphone 2R, and the spatial acoustic transfer characteristic Hrs is acquired.

[0037] Furthermore, the measuring device 200 may generate spatial acoustic filters according to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs from the left and right speakers 5L and 5R to the left and right microphones 2L and 2R based on the collected sound signals. For example, the processing device 201 cuts out the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length. The processing device 201 may correct the measured spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs.

[0038] In this way, the processing device 201 generates a spatial audio filter used in the convolution operation of the out-of-head localization processing device 100. As shown in Fig. 1, the out-of-head localization processing device 100 performs out-of-head localization processing using a spatial audio filter according to the spatial audio transfer characteristics Hls, Hlo, Hro, and Hrs between the left and right speakers 5L and 5R and the left and right microphones 2L and 2R. That is, the out-of-head localization processing is performed by convolving the spatial audio filter with the audio playback signal.

[0039] The processing device 201 performs the same processing on the pickup signals corresponding to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs. That is, the same processing is performed on the four pickup signals corresponding to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs. This makes it possible to generate spatial acoustic filters corresponding to the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs.

[0040] The processing device 201 of the measuring device 200 and its processing will be described in detail below. Fig. 3 is a control block diagram showing the processing device 201. The processing device 201 includes a measurement signal generating unit 211, a picked-up signal acquiring unit 212, a frequency characteristic acquiring unit 214, a smoothing processing unit 215, an axis converting unit 216, a first compression unit 217, a second compression unit 218, an axis converting unit 220, and a filter generating unit 221.

[0041] The measurement signal generating unit 211 includes a D / A converter, an amplifier, and the like, and generates a measurement signal for measuring ear canal transfer characteristics. The measurement signal is, for example, an impulse signal or a TSP (Time Stretched Pulse) signal. Here, the measuring device 200 performs impulse response measurement using an impulse sound as the measurement signal.

[0042] The left microphone 2L and right microphone 2R of the microphone unit 2 each pick up a measurement signal and output the picked-up signal to the processing device 201. The picked-up signals picked up by the left microphone 2L and right microphone 2R are input to the processing device 201 as input signals. The picked-up signal acquisition unit 212 acquires the picked-up signals picked up by the left microphone 2L and right microphone 2R. The picked-up signal acquisition unit 212 may include an A / D converter that A / D converts the picked-up signals from the microphones 2L and 2R. The picked-up signal acquisition unit 212 may synchronously add signals obtained by multiple measurements.

[0043] The frequency characteristic acquisition unit 214 acquires the frequency characteristic of the picked-up sound signal. The frequency characteristic acquisition unit 214 calculates the frequency characteristic of the picked-up sound signal by discrete Fourier transform or discrete cosine transform. The frequency characteristic acquisition unit 214 calculates the frequency characteristic by, for example, performing FFT (fast Fourier transform) on the picked-up sound signal in the time domain. The frequency characteristic includes an amplitude spectrum and a phase spectrum. Note that the frequency characteristic acquisition unit 214 may generate a power spectrum instead of the amplitude spectrum.

[0044] The smoothing processor 215 performs smoothing processing to generate second spectral data that is smoother than the first spectral data based on the frequency characteristics. That is, the smoothing processor 215 performs smoothing processing on the spectral data based on the frequency characteristics. The smoothing processor 215 smoothes the spectral data using a method such as a moving average, a Savitzky-Golay filter, a smoothing spline, a cepstrum transform, or a cepstrum envelope.

[0045] When smoothing is performed by cepstrum analysis, the smoothing processor 215 gives the order of the lifter as the order of smoothing. In this case, the smoothing processor 215 can change the degree of smoothing by giving different values ​​to the order of smoothing. When the order is large, the degree of smoothing is low, and when the order is small, the degree of smoothing is high. Therefore, the spectral data obtained by the smoothing process of a small order is smoother than the spectral data obtained by the smoothing process of a large order. The spectral data obtained by the smoothing process of a small order is smoother than the spectral data obtained by the smoothing process of a large order.

[0046] In this embodiment, the smoothing processing unit 215 performs smoothing processing of different orders on the frequency amplitude characteristics to generate first spectral data and second spectral data. The smoothing processing unit 215 performs smoothing processing of a relatively high order on the frequency amplitude characteristics (amplitude spectrum) to calculate the first spectral data. The smoothing processing unit 215 performs smoothing processing of a relatively low order on the spectral data of the frequency amplitude characteristics to calculate the second spectral data (also referred to as smoothed spectral data). The smoothing processing unit 215 generates the first spectral data and the second spectral data that is smoother than the first spectral data.

[0047] In the following embodiment, the first spectral data is the spectral data that has been smoothed with a large order. However, the first spectral data may be the spectral data whose frequency-amplitude characteristics have not been smoothed. In other words, the first spectral data may be the frequency-amplitude characteristics obtained by FFT.

[0048] Alternatively, the smoothing processor 215 generates the first spectral data and the second spectral data by performing the smoothing process multiple times. That is, the smoothing processor 215 generates the first spectral data by performing a first smoothing process on the frequency-amplitude characteristics. The smoothing processor 215 generates the second spectral data by performing a second smoothing process on the first spectral data that has already been smoothed. In this case, the smoothing processor 215 may use the same smoothing process or different smoothing processes in the first and second smoothing processes.

[0049] Figure 4 shows the first spectrum data A and the second spectrum data A. sm 4 is a graph showing the second spectrum data A. In FIG. 4, the horizontal axis is frequency [Hz] and the vertical axis is amplitude (gain) [dB]. sm is smoother than the first spectrum data A. In other words, the second spectrum data A smhas smoother gain data than the first spectrum data A.

[0050] The axis conversion unit 216 converts the first spectrum data A and the second spectrum data A by data interpolation. sm The axis conversion unit 216 converts the frequency axis of the gain data. The axis conversion unit 216 changes the scale of the frequency amplitude characteristic data so that the discrete spectrum data is equally spaced on the logarithmic axis. The first and second spectrum data (hereinafter collectively referred to as gain data) are equally spaced in frequency in the frequency characteristic acquisition unit 214. That is, since the gain data is equally spaced on the frequency linear axis, it is not equally spaced on the frequency logarithmic axis. Therefore, the axis conversion unit 216 performs an interpolation process on the gain data so that the gain data is equally spaced on the frequency logarithmic axis.

[0051] In gain data, on the logarithmic axis, the lower the frequency range, the coarser the interval between adjacent data, and the higher the frequency range, the closer the interval between adjacent data. Therefore, the axis conversion unit 216 interpolates data in the low frequency band where the data interval is coarse. Specifically, the axis conversion unit 216 performs an interpolation process such as three-dimensional spline interpolation to obtain discrete gain data that is equally spaced on the logarithmic axis. The gain data that has undergone axis conversion is called axis-converted data. The axis-converted data is a spectrum in which frequency and amplitude value (gain value) are associated with each other. The axis-converted data is smoothed spectrum data that has undergone axis conversion.

[0052] The reason for converting the frequency axis into a logarithmic scale will be explained. It is generally said that human sensory quantities are converted into logarithms. Therefore, it is important to consider the frequency of audible sounds on a logarithmic axis. By converting the scale, data is equally spaced in the above sensory quantities, so that data can be treated equally in all frequency bands. As a result, mathematical operations, frequency band division and weighting become easier, and stable results can be obtained. The axis conversion unit 216 may convert the envelope data into a scale close to human hearing (referred to as an auditory scale) instead of the logarithmic scale. The auditory scale may be a logarithmic scale (Log scale), a mel scale, a Bark scale, an Equivalent Rectangular Bandwidth (ERB) scale, or the like.

[0053] The axis conversion unit 216 performs scale conversion on the gain data in the auditory scale by data interpolation. For example, the axis conversion unit 216 interpolates data in a low frequency band with coarse data intervals in the auditory scale to make the data in the low frequency band dense. Data with equal intervals in the auditory scale becomes data with a dense low frequency band and a coarse high frequency band in a linear scale. In this way, the axis conversion unit 216 can generate axis converted data with equal intervals in the auditory scale. Of course, the axis converted data does not have to be data with completely equal intervals in the auditory scale.

[0054] The first compression unit 217 performs a first compression process on the second spectral data in the first band B1. The first compression unit 217 calculates a first difference value according to the difference between the second spectral data and the first spectral data in the first band B1. The first compression unit 217 compresses the second spectral data based on the first difference value. For example, the first compression unit 217 compresses the second spectral data A sm The value obtained by subtracting the first spectrum data A from sm -A) is calculated as a first difference value. The first difference value is calculated for each frequency.

[0055] The first compression unit 217 calculates the first difference value (A sm If -A) is a positive value, the first difference value (Asm The first compression value is calculated by multiplying the second spectrum data A by the first compression coefficient lrcRate1. sm The first compression value lrcRate1*(A sm -A) to perform compression processing. If the first difference value is a negative value, first compression section 217 does not perform compression. In other words, the gain of the second spectral data is used as is.

[0056] The first compression process in the first compression section 217 is expressed by the following equations (1) and (2). A is A sm If less than A lrc1 =lrcRate1*(A sm -A)+A sm (1) A is A sm In the above case A lrc1 =A sm (2)

[0057] The first compression unit 217 performs the above A lrc1 The first compression unit 217 does not add the first compression value to the second spectral data at frequencies where the gain of the first spectral data exceeds the gain of the second spectral data. The first compression unit 217 adds the first compression value to the second spectral data at frequencies where the gain of the first spectral data is lower than the gain of the second spectral data. The first compression unit 217 compresses the range so that the gain of the second spectral data approaches the gain of the first spectral data at frequencies where the gain of the first spectral data is lower than the gain of the second spectral data. The first compression unit 217 generates third spectral data by performing the first compression process on the second spectral data in the first band B1. That is, the second spectral data compressed by the first compression unit 217 becomes the third spectral data.

[0058] For example, the second spectrum data A at a certain frequency sm The first difference value (A smThe first compression coefficient lrcRate1=0.5. The first compression value is 0.5*(5-3)=1[dB], and the third spectrum data A lrc1 =5-1=4[dB].

[0059] In this way, the first compression unit 217 determines whether or not to perform compression based on the first difference value. That is, the first compression unit 217 determines the frequencies at which compression is performed and the frequencies at which compression is not performed according to the sign (positive or negative) of the first difference value. At the frequencies at which compression is performed, the gain after compression is a value between the first spectral data and the second spectral data.

[0060] The third spectral data A obtained by the first compression process in the first compression unit 217 lrc1 The third spectrum data A is shown in FIG. lrc1 In the bands other than the first band B1, the gain of the second spectrum data and the gain of the third spectrum data are equal. 1S The upper limit frequency is f 1E Let us assume that.

[0061] For example, the first band B1 may be 20 Hz to 1 kHz. 1S is 20Hz, and the upper limit frequency f 1E Of course, the first band B1 is not limited to this range.

[0062] The second compression unit 218 performs a second compression process on the third spectral data in the second band. The second compression unit 218 calculates a second difference value corresponding to the difference between the reference value and the third spectral data in the second band. The second compression unit 218 compresses the third spectral data based on the second difference value. Reference value A ref is a predetermined value for the gain of the spectrum data, which is a constant value of 0 [dB] here. Note that the reference value is a constant level in the second band, but may be different depending on the frequency.

[0063] The second compression section 218 is a reference value A ref From the third spectrum data A lrc1 The value obtained by subtracting (A ref -A lrc1 ) as the second differential value. The second differential value is calculated for each frequency. When the second differential value is a negative value, the second compression unit 218 multiplies the second differential value by the second compression coefficient lrcRate2 to calculate the second compressed value. lrc1 The second compression value lrcRate2*(A ref -A lrc1 The second compression unit 218 performs compression by adding the second difference value A lrc1 The gain is used as is.

[0064] The second compression process in the second compression section 218 is expressed by the following equations (3) and (4). A lrc1 A ref If less than A lrc2 =lrcRate2*(A ref -A lrc1 )+A lrc1 (3) A lrc1 A ref In the above case A lrc2 =A lrc1 (4)

[0065] 6 is a graph showing a second difference value between the third spectrum data and the reference value. lrc2The second compression unit 218 does not add the second compressed value to the third spectral data at frequencies where the gain of the third spectral data exceeds the reference value. The second compression unit 218 adds the second compressed value to the third spectral data at frequencies where the gain of the third spectral data falls below the reference value. The second compression unit 218 compresses the range so that the gain of the third spectral data approaches the reference value at frequencies where the gain of the third spectral data falls below the reference value. The second compression unit 218 generates fourth spectral data by performing the second compression process on the third spectral data in the second band B2. In other words, the third spectral data compressed by the second compression unit 218 becomes the fourth spectral data. The fourth spectral data A obtained by the second compression process in the second compression unit 218 lrc2 is shown in Figure 7.

[0066] For example, the third spectrum data A lrc1 is -2dB, reference value A ref is 0 dB. The difference value (A ref -A lrc1 ) is 2 dB. In addition, the second compression coefficient lrcRate2 = 0.5. The second compression value is 0.5 * 2 = 1 [dB], and the fourth spectrum data A lrc2 =1-2=-1[dB].

[0067] In this way, the second compression unit 218 determines whether or not to perform compression based on the second difference value. That is, the second compression unit 218 determines the frequencies at which compression is performed and the frequencies at which compression is not performed according to the sign (positive or negative) of the second difference value. At the frequencies at which compression is performed, the gain after compression becomes a value between the third spectral data and the reference value.

[0068] In the bands other than the second band B2, the gain of the third spectrum data is equal to the gain of the fourth spectrum data. 2S The upper limit frequency is f 2E Let us assume that.

[0069] The lower limit frequency f of the second band B2 2S is the lower limit frequency f of the first band B1 1SFor example, the lower limit frequency f 2S and the lower limit frequency f 1S The upper limit frequency f of the second band B2 is 20 Hz. 2E is the upper limit frequency f of the first band B1 1E For example, the upper limit frequency f 2E and upper frequency f 1E The frequency is 1kHz.

[0070] The first band B1 and the second band B2 are low frequency bands between 20 Hz and 1 kHz. 2S and the lower limit frequency f 1S is not limited to 20Hz. 2E and upper frequency f 1E is not limited to 1 kHz.

[0071] The axis conversion unit 220 performs axis conversion to convert the frequency axis of the fourth spectral data by data interpolation or the like. The process in the axis conversion unit 220 is the opposite process to that in the axis conversion unit 216. By performing axis conversion, the axis conversion unit 220 returns the frequency axis of the fourth spectral data to the frequency axis before the axis conversion in the axis conversion unit 216. For example, the axis conversion unit 220 performs a process to return the frequency axis that has been logarithmically scaled in the axis conversion unit 216 to a linear scale. The fourth spectral data is converted to data with equal intervals on the frequency linear axis. This makes it possible to obtain frequency amplitude characteristics on the same frequency axis as the frequency phase characteristics acquired in the frequency characteristic acquisition unit 214. In other words, the frequency axes (data intervals) of the spectral data of the frequency phase characteristics and the frequency amplitude characteristics match.

[0072] The filter generation unit 221 generates a filter using the fourth spectral data that has been axis-transformed by the axis transformation unit 220. The filter generation unit 221 generates a filter to be applied to a reproduction signal based on the fourth spectral data. For example, the filter generation unit 221 calculates a time domain signal from amplitude characteristics and phase characteristics by inverse discrete Fourier transform or inverse discrete cosine transform. The filter generation unit 221 generates a time signal by performing IFFT (inverse fast Fourier transform) on the amplitude characteristics and phase characteristics. The filter generation unit 221 calculates a spatial audio filter by cutting out the generated time signal with a predetermined filter length. The filter generation unit 221 may generate the spatial audio filter by performing windowing.

[0073] The filter generation unit 221 generates a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hls by performing the above-mentioned processing on a picked-up signal obtained by collecting a measurement signal from the left speaker 5L with the left microphone 2L. The filter generation unit 221 generates a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hlo by performing the above-mentioned processing on a picked-up signal obtained by collecting a measurement signal from the left speaker 5L with the right microphone 2L.

[0074] The filter generation unit 221 generates a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hro by performing the above-mentioned processing on a picked-up signal obtained by collecting a measurement signal from the right speaker 5R with the left microphone 2L. The filter generation unit 221 generates a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hrs by performing the above-mentioned processing on a picked-up signal obtained by collecting a measurement signal from the right speaker 5R with the right microphone 2R.

[0075] In this way, it is possible to compress the frequency characteristics in a well-balanced manner. Therefore, it is possible to generate a filter suitable for the localization of a sound image. It is possible to prevent the balance of the sound image localization from being lost. It is possible to localize a well-balanced sound image. It is possible to generate a filter adjusted to a well-balanced sound quality. It is possible to ensure a natural sound quality in terms of the auditory sensation.

[0076] In particular, since the low frequency band below the upper limit frequency can be compressed in a well-balanced manner, excellent sound quality can be achieved in the low frequency band. Even if the sound collection time of the measurement device 200 in FIG. 2 is short, a well-balanced filter can be generated.

[0077] The lower limit frequency f of the second band B2 2S is the lower limit frequency f of the first band B1 1S For example, the lower limit frequency f of the second band B2 may be different from 2S is the lower limit frequency f of the first band B1 1S The upper limit frequency f of the second band B2 is greater than 2E It is sufficient if the range is smaller.

[0078] The upper limit frequency f of the second band B2 2E is the upper limit frequency f of the first band B1 1E For example, the upper limit frequency f of the second band B2 may be different from 2E is the upper limit frequency f of the first band B1 1E The lower limit frequency f of the second band B2 is smaller than 2S A larger range would be better.

[0079] The first compression coefficient lrcRate1 and the second compression coefficient lrcRate2 may be the same value or different values. Here, the first compression coefficient lrcRate1 and the second compression coefficient lrcRate2 are 0.5. Of course, the values ​​of the first compression coefficient lrcRate1 and the second compression coefficient lrcRate2 are not limited to 0.5.

[0080] As shown in FIG. 2, a measurement signal from one speaker is collected by left and right microphones 2L and 2R. Therefore, two collected signals (also called left and right collected signals) are obtained in one measurement. The first compression coefficient lrcRate1 may have different values ​​for processing the collected signals from the left and right microphones 2L and 2R. Similarly, the second compression coefficient lrcRate2 may have different values ​​for the left and right microphones 2L and 2R.

[0081] As shown in FIG. 2, four collected sound signals are obtained because left and right speakers 5L, 5R and left and right microphones 2L, 2R are used. That is, collected sound signals indicating spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are respectively obtained. In this case, the first compression process and the second compression process can be performed on all of the four collected sound signals. Alternatively, the first compression process or the second compression process may not be performed on some of the four collected sound signals. In other words, the first compression process and the second compression process may be performed only on collected sound signals from a specific direction, and at least one of the first compression process and the second compression process may be omitted for the remaining directions.

[0082] Furthermore, the first difference value may be an average value of the left and right pickup signals. For example, the first spectrum data and the second spectrum data generated from the pickup signal of the left microphone 2L are L ,A smL The first and second spectrum data generated from the sound pickup signal of the right microphone 2R are represented as A R ,A smR In this case, the first difference value D1 can be set to an average value of the difference value obtained from the left picked-up sound signal and the difference value obtained from the right picked-up sound signal. The first difference value D1 is expressed by the following formula (5).

[0083] D1={(A smL -A L )+(A smR -A R )} / twenty five)

[0084] The first differential value D1 is common to the left and right pickup signals. smL -A L ) into D1 in equation (5), and the third spectrum data A lrc1 Then, the first compression section 217 performs the first compression process on the left and right spectral data by using the common first difference value D. This makes it possible to compress the left and right frequency characteristics in a well-balanced manner.

[0085] Furthermore, in the first compression process and the second compression process, the compression coefficient and the band in which the process is performed can be determined by adjusting the audible balance so as to follow the loudness curve.

[0086] The first compression process and the second compression process may be performed alternately. In other words, the first compression process may be performed again after the second compression process. The first compression process is performed multiple times, and the second compression process is performed multiple times. In each compression process, the band and compression coefficient may be the same or different. For example, the compression coefficient and band may be different or the same in the first first compression process and the second compression process.

[0087] 8 is a flowchart showing a processing method according to this embodiment. First, the frequency characteristic acquisition unit 214 acquires the frequency characteristic of the picked-up signal acquired by the picked-up signal acquisition unit 212 (S801). For example, the picked-up signal in the time domain is transformed into the frequency domain by FFT or the like. Next, the smoothing processing unit 215 performs smoothing processing on the spectral data (S802). As a result, second spectral data is obtained. Also, the smoothing processing unit 215 changes the order of the smoothing processing to obtain first spectral data.

[0088] The axis conversion unit 216 performs axis conversion on the second spectral data (S803). As a result, spectral data in which the frequency axis of the collected signal is converted to a logarithmic axis is obtained. Note that the axis conversion process by the axis conversion unit 216 can be omitted. In this case, the axis conversion process by the axis conversion unit 220 described later is also unnecessary.

[0089] Next, first compression unit 217 calculates a first difference value (S804). That is, first compression unit 217 calculates a first difference value corresponding to the difference between the second spectral data and the first spectral data. First compression unit 217 compresses the second spectral data using the first difference value (S805). In this way, third spectral data is calculated.

[0090] The second compression unit 218 calculates a second difference value (S806). That is, the second compression unit 218 calculates a second difference value corresponding to the difference between the reference value and the third spectral data. The second compression unit 218 compresses the third spectral data using the second difference value (S807). In this way, the fourth spectral data is calculated.

[0091] The axis conversion unit 220 performs axis conversion on the fourth spectral data (S808). The filter generation unit 221 generates a filter based on the axis-converted fourth spectral data (S809). A spatial acoustic filter corresponding to the spatial acoustic transfer characteristics Hls, Hlo or a spatial acoustic filter corresponding to the spatial acoustic transfer characteristics Hro, Hrs is generated. In this way, a well-balanced filter can be generated.

[0092] In the processing device and processing method according to the present embodiment, the second compression process may be omitted, that is, the processing device 201 may be configured to execute only the first compression process.

[0093] Although the axis conversion unit 220 performs the axis conversion process on the fourth spectral data, the axis conversion unit 220 may perform the axis conversion process on other spectral data. That is, the axis conversion unit 220 can perform the axis conversion on the spectral data after the first compression process by the first compression unit 217. In this case, when the filter generation unit 221 generates a filter, it is only necessary that the frequency axes of the phase measurement and the amplitude characteristics are aligned.

[0094] 9 to 12 are graphs showing the spectral data obtained by the processing of this embodiment. FIG. 9 shows the result of performing the first compression process on the spectral data of the picked-up signal showing the spatial acoustic transfer characteristic Hls. FIG. 10 shows the result of performing the first compression process on the spectral data of the picked-up signal showing the spatial acoustic transfer characteristic Hrs. In FIG. 9 and FIG. 10, the spectral data that has been subjected to the first compression process is shown as A. rdc1 is shown as:

[0095] Fig. 11 shows the result of performing the first compression process and the second compression process on the spectrum data of the sound pickup signal showing the spatial acoustic transfer characteristic Hls. Fig. 12 shows the result of performing the first compression process and the second compression process on the spectrum data of the sound pickup signal showing the spatial acoustic transfer characteristic Hrs. In Figs. 11 and 12, the spectrum data which has been subjected to the first compression process and the second compression process is shown as A. rdc1 For comparison, the spectrum data before compression is shown in Figures 9 to 12. Specifically, the spectrum data before smoothing is shown in Figures 9 to 12.

[0096] Furthermore, in the first and second embodiments, the processing device 201 processes the spectrum data of the collected sound signal indicating the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs, but may process the spectrum data of the collected sound signal indicating the ear canal transfer characteristics. Furthermore, the processing device 201 generates the outside-head low-level processing filter, but may generate other filters. By using the filter generated by the processing method according to the present embodiment, a balanced sound image can be localized.

[0097] The out-of-head localization processing device 100 is not limited to being a single physical device, but may be distributed among a plurality of devices connected via a network, etc. In other words, the out-of-head localization processing method according to this embodiment may be performed by a plurality of distributed devices.

[0098] A part or all of the above processes may be executed by a computer program. The above-mentioned program includes a set of instructions (or software code) for making the computer perform one or more functions described in the embodiments when the program is loaded into the computer. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or the tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or the communication medium includes electrical, optical, acoustic, or other forms of propagating signals.

[0099] The invention made by the inventor has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention. [Explanation of symbols]

[0100] U User 1 Person to be measured 2 Microphone unit 2L Left microphone 2R Right Microphone 5 Stereo speakers 5L Left speaker 5R Right speaker 10. Extra-head localization processing unit 11 Convolution Calculation Unit 12 Convolution Calculation Unit 21 Convolution Calculation Unit 22 Convolution Calculation Unit 24 Adder 25 Adder 41 Filter section 42 Filter section 43 Headphones 200 Measuring Equipment 201 Processing equipment 211 Measurement signal generator 212 Sound collection signal acquisition unit 214 Frequency characteristic acquisition section 215 Smoothing processing unit 216 Axis conversion unit 217 First Compression Section 218 Second Compression Section 220 Axis conversion unit 221 Filter Generation Unit B1 First band B2 Second band

Claims

1. a frequency characteristic acquisition unit for acquiring a frequency characteristic of an input signal; a smoothing processing unit that performs a smoothing process so as to generate second spectral data that is smoother than the first spectral data based on the frequency characteristics; a first compression unit that calculates a first difference value corresponding to a difference between the second spectral data and the first spectral data in a first band, and compresses the second spectral data based on the first difference value; a filter generating unit configured to generate a filter based on the second spectral data.

2. 2. The processing device according to claim 1, further comprising: a second compression unit that calculates a second difference value corresponding to a difference between third spectral data generated by the first compression process in the first compression unit and a predetermined reference value in a gain of the spectral data, and compresses the third spectral data based on the second difference value.

3. The processing device according to claim 2 , wherein the first compression process by the first compression section and the second compression process by the second compression section are alternately performed.

4. a first axis conversion unit that converts a frequency axis of the first spectrum data by data interpolation; a second axis conversion unit that converts a frequency axis of the spectrum data compressed by the first compression unit by data interpolation, 4. The processing device according to claim 1, wherein the filter generating section generates the filter based on the spectrum data axis-converted by the second axis converting section.

5. obtaining a frequency characteristic of an input signal; performing a smoothing process to generate second spectral data that is smoother than the first spectral data based on the frequency characteristics; calculating a first difference value corresponding to a difference between the second spectral data and the first spectral data in a first band, and compressing the second spectral data based on the first difference value; generating a filter based on the second spectral data.

Images