Processing device and processing method

Abstract

To provide a processing device and a processing method capable of appropriately compressing peaks or dips.SOLUTION: A processing device 201 includes a frequency characteristic acquisition unit 214 that acquires the frequency characteristic of an input signal, an extreme value extraction unit 216 that extracts an extreme value of spectrum data, a kurtosis calculation unit 217 that calculates an evaluation value for evaluating the peak or the dip corresponding to extreme value from the spectrum data, and calculates the kurtosis of the peak or the dip on the basis of a plurality of evaluation values calculated by changing the calculation width, a determination unit 218 that determines whether to compress the peak or dip according to the comparison result between the kurtosis and a threshold, and a compression unit 219 that compresses the peak or the dip determined to compress the extreme value.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a processing device and a processing method. [Background technology]

[0002] One sound image localization technology is out-of-head localization, which uses headphones to localize a sound image outside the listener's head. Out-of-head localization technology localizes a sound image outside the listener's head by canceling the characteristics from the headphones to the ears (headphone characteristics) and providing two characteristics (spatial acoustic transfer characteristics) from one speaker (monaural speaker) to the ears.

[0003] In out-of-head localization playback using stereo speakers, a measurement signal (impulse sound, etc.) emitted from two-channel (hereafter referred to as ch) speakers is recorded by a microphone (hereafter referred to as mic) placed at the ear of the listener. Then, a processing device generates a filter based on the collected signal obtained by collecting the measurement signal. Out-of-head localization playback can be achieved by convolving the generated filter with the 2ch audio signal.

[0004] Furthermore, in order to generate a filter (also called an inverse filter) that cancels the characteristics from the headphones to the ears, the characteristics from the headphones to the ears and eardrum (also called the ear canal transfer function ECTF or ear canal transfer characteristics) are measured using a microphone placed at the listener's own ear.

[0005] Patent Document 1 discloses an out-of-head localization processing device that performs out-of-head localization processing using a filter. In Patent Document 1, a measurement microphone placed in the ear canal of a user picks up an impulse sound. This allows the ear canal transfer characteristic from the speaker unit of the headphones to the microphone to be measured. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2020-136752 A Summary of the Invention [Problem to be solved by the invention]

[0007] When performing out-of-head localization processing, it is preferable to measure the characteristics with a microphone placed at the listener's own ear. When measuring ear canal transfer characteristics, impulse response measurement, etc. is performed with the listener wearing a microphone and headphones at their ear. By using the listener's own characteristics, a filter suitable for the listener can be generated. For such filter generation, etc., it is desirable to appropriately process the collected sound signal obtained by measurement.

[0008] Inverse filters are generated using algorithms such as the least squares method, but due to the nature of the algorithm, it is difficult to create perfect inverse characteristics at all frequencies. Also, if the ECTF itself has steep peaks or dips, the inverse characteristics will be generated, and the inverse filter will have steep dips or peaks at those points. Furthermore, if an inverse filter is generated at a position where the control point of the sound field for out-of-head localization playback is different, unintended peaks may occur.

[0009] In addition, when a user puts on and takes off the headphones, the frequency at which the peak occurs may differ from before and after putting them on. This may have a negative effect on the localization and balance of sound quality. Ideally, the ECTF would be measured every time the headphones are put on and taken off, but this would impose a burden on the user. For this reason, it is desirable to compress in advance any steep peaks or dips in the frequency characteristics obtained by user measurement.

[0010] The present disclosure has been made in consideration of the above-mentioned points, and has an object to provide a processing device and a processing method that can appropriately compress peaks or dips. [Means for solving the problem]

[0011] The processing device according to this embodiment includes a frequency characteristic acquisition unit that acquires frequency characteristics of an input signal, an extreme value extraction unit that extracts extreme values ​​of spectral data based on the frequency characteristics, a kurtosis calculation unit that calculates an evaluation value for evaluating a peak or dip corresponding to the extreme value from the spectral data in a calculation width including the extreme value and calculates the kurtosis of the peak or dip based on a plurality of evaluation values ​​calculated by changing the calculation width, a determination unit that determines whether or not to compress the peak or dip depending on a comparison result between the kurtosis and a threshold value, and a compression unit that compresses the peak or dip for which it has been determined that the extreme value should be compressed.

[0012] The processing method according to the present embodiment includes the steps of acquiring frequency characteristics of an input signal, extracting extreme values ​​of the frequency characteristics, calculating an evaluation value for evaluating the extreme value from data in a calculation range including the extreme value, and calculating the kurtosis of the extreme value based on a plurality of evaluation values ​​calculated by changing the calculation range, determining whether or not to compress the extreme value depending on a result of comparing the kurtosis with a threshold, and compressing the extreme value that is determined to be compressed. Effect of the Invention

[0013] According to the present disclosure, it is possible to provide a processing device and a processing method capable of appropriately compressing peaks or dips. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing an extra-head localization processing device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram illustrating a schematic configuration of a measuring device. [Diagram 3] FIG. 2 is a block diagram showing a configuration of a processing device. [Figure 4] 1 is a graph showing an example of frequency amplitude characteristics. [Diagram 5] 1 is a graph showing peaks extracted from an axis transformed spectrum; [Figure 6]FIG. 13 is a diagram illustrating an example of a process for calculating kurtosis. [Figure 7] FIG. 11 is a diagram for explaining a process of merging adjacent peaks in the second embodiment. [Figure 8] 1 is a flowchart illustrating a processing method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] An outline of the sound image localization process according to this embodiment will be described. The out-of-head localization process according to this embodiment is performed using the spatial acoustic transfer characteristic and the ear canal transfer characteristic. The spatial acoustic transfer characteristic is the transfer characteristic from a sound source such as a speaker to the ear canal. The ear canal transfer characteristic is the transfer characteristic from a speaker unit of a headphone or earphone to the eardrum. In this embodiment, the spatial acoustic transfer characteristic is measured when the headphone or earphone is not worn, and the ear canal transfer characteristic is measured when the headphone or earphone is worn, and the out-of-head localization process is realized using the measurement data. This embodiment is characterized by a microphone system for measuring the spatial acoustic transfer characteristic or the ear canal transfer characteristic.

[0016] The out-of-head localization processing according to this embodiment is executed by a user terminal such as a personal computer, a smartphone, or a tablet PC. The user terminal is an information processing device having a processing means such as a processor, a storage means such as a memory or a hard disk, a display means such as a liquid crystal monitor, and an input means such as a touch panel, a button, a keyboard, or a mouse. The user terminal may have a communication function for transmitting and receiving data. Furthermore, an output means (output unit) having headphones or earphones is connected to the user terminal. The connection between the user terminal and the output means may be a wired connection or a wireless connection.

[0017] Embodiment 1 (Extracranial Localization Processing Device) FIG. 1 shows a block diagram of an out-of-head localization processing device 100, which is an example of a sound field reproduction device according to this embodiment. The out-of-head localization processing device 100 reproduces a sound field for a user U wearing headphones 43. For this purpose, the out-of-head localization processing device 100 performs sound image localization processing on Lch and Rch stereo input signals XL and XR. The Lch and Rch stereo input signals XL and XR are analog audio reproduction signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). The audio reproduction signals or digital audio data are collectively referred to as reproduction signals. In other words, the Lch and Rch stereo input signals XL and XR are reproduction signals.

[0018] The extrahead localization processing device 100 is not limited to being a single physical device, and some of the processing may be performed by different devices. For example, some of the processing may be performed by a smartphone or the like, and the remaining processing may be performed by a DSP (Digital Signal Processor) built into the headphones 43.

[0019] The out-of-head localization processing device 100 includes an out-of-head localization processing unit 10, a filter unit 41 that stores an inverse filter Linv, a filter unit 42 that stores an inverse filter Rinv, and a headphone 43. Specifically, the out-of-head localization processing unit 10, the filter unit 41, and the filter unit 42 can be realized by a processor or the like.

[0020] The out-of-head localization processing unit 10 includes convolution calculation units 11-12, 21-22 that store spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs, and adders 24 and 25. The convolution calculation units 11-12, 21-22 perform convolution processing using the spatial acoustic transfer characteristics. Stereo input signals XL and XR from a CD player or the like are input to the out-of-head localization processing unit 10. The spatial acoustic transfer characteristics are set in the out-of-head localization processing unit 10. The out-of-head localization processing unit 10 convolves the stereo input signals XL and XR of each channel with a filter of the spatial acoustic transfer characteristics (hereinafter also referred to as a spatial acoustic filter). The spatial acoustic transfer characteristics may be head-related transfer functions HRTF measured on the head or auricle of the subject, or may be head-related transfer functions of a dummy head or a third party.

[0021] A set of four spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs is defined as a spatial acoustic transfer function. Data used for convolution in the convolution calculation units 11, 12, 21, and 22 becomes a spatial acoustic filter. The spatial acoustic filter is generated by cutting out the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length.

[0022] The spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are acquired in advance by impulse response measurement or the like. For example, a user U wears microphones on the left and right ears, respectively. Left and right speakers arranged in front of the user U output impulse sounds for performing impulse response measurement. Then, a measurement signal such as the impulse sound output from the speakers is picked up by a microphone. The spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs are acquired based on the pickup signal by the microphone. The spatial acoustic transfer characteristic Hls between the left speaker and the left microphone, the spatial acoustic transfer characteristic Hlo between the left speaker and the right microphone, the spatial acoustic transfer characteristic Hro between the right speaker and the left microphone, and the spatial acoustic transfer characteristic Hrs between the right speaker and the right microphone are measured.

[0023] Then, the convolution calculation unit 11 convolves the Lch stereo input signal XL with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hls. The convolution calculation unit 11 outputs the convolution calculation data to the adder 24. The convolution calculation unit 21 convolves the Rch stereo input signal XR with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hro. The convolution calculation unit 21 outputs the convolution calculation data to the adder 24. The adder 24 adds the two convolution calculation data and outputs the result to the filter unit 41.

[0024] The convolution calculation unit 12 convolves the Lch stereo input signal XL with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hlo. The convolution calculation unit 12 outputs the convolution calculation data to the adder 25. The convolution calculation unit 22 convolves the Rch stereo input signal XR with a spatial acoustic filter corresponding to the spatial acoustic transfer characteristic Hrs. The convolution calculation unit 22 outputs the convolution calculation data to the adder 25. The adder 25 adds the two convolution calculation data and outputs the result to the filter unit 42.

[0025] Inverse filters Linv, Rinv that cancel headphone characteristics (characteristics between the playback unit of the headphones and the microphone) are set in the filter units 41, 42. The inverse filters Linv, Rinv are convolved with the playback signal (convolution operation signal) that has been processed by the out-of-head localization processing unit 10. The filter unit 41 convolves the inverse filter Linv of the Lch headphone characteristics with the Lch signal from the adder 24. Similarly, the filter unit 42 convolves the inverse filter Rinv of the Rch headphone characteristics with the Rch signal from the adder 25. The inverse filters Linv, Rinv cancel the characteristics from the headphone unit to the microphone when the headphones 43 are worn. The microphone may be placed anywhere between the entrance of the ear canal and the eardrum.

[0026] The filter unit 41 outputs the processed Lch signal YL to the left unit 43L of the headphones 43. The filter unit 42 outputs the processed Rch signal YR to the right unit 43R of the headphones 43. A user U wears the headphones 43. The headphones 43 output the Lch signal YL and the Rch signal YR (hereinafter, the Lch signal YL and the Rch signal YR are also collectively referred to as stereo signals) toward the user U. This makes it possible to reproduce a sound image localized outside the head of the user U.

[0027] In this way, the out-of-head localization processing device 100 performs out-of-head localization processing using the spatial audio filters according to the spatial audio transfer characteristics Hls, Hlo, Hro, and Hrs, and the inverse filters Linv and Rinv of the headphone characteristics. In the following description, the spatial audio filters according to the spatial audio transfer characteristics Hls, Hlo, Hro, and Hrs, and the inverse filters Linv and Rinv of the headphone characteristics are collectively referred to as the out-of-head localization processing filters. In the case of a 2ch stereo reproduction signal, the out-of-head localization filters are composed of four spatial audio filters and two inverse filters. The out-of-head localization processing device 100 then performs convolution calculation processing on the stereo reproduction signal using a total of six out-of-head localization filters, thereby executing the out-of-head localization processing. It is preferable that the out-of-head localization filters are based on the individual measurement of the user U. For example, the out-of-head localization filters are set based on a pickup signal picked up by a microphone attached to the ear of the user U.

[0028] In this way, the spatial acoustic filter and the headphone characteristic inverse filters Linv, Rinv are filters for audio signals. These filters are convoluted with the playback signal (stereo input signals XL, XR), so that the out-of-head localization processing device 100 executes out-of-head localization processing. In this embodiment, the processing for generating the inverse filters Linv, Rinv is one of the technical features. The processing for generating the inverse filters will be described below.

[0029] (Measuring Equipment) The measuring device 200 will be described with reference to Fig. 2. Fig. 2 shows a configuration for measuring the transfer characteristic of a user U. The measuring device 200 measures the ear canal transfer characteristic in order to generate an inverse filter. The measuring device 200 includes a microphone unit 2, a headphone 43, and a processing device 201. Note that the person 1 to be measured is the same person as the user U in Fig. 1, but may be a different person.

[0030] In this embodiment, the processing device 201 of the measurement device 200 performs arithmetic processing for appropriately generating a filter according to the measurement result. The processing device 201 is a personal computer (PC), a tablet terminal, a smartphone, or the like, and includes a memory and a processor. The memory stores a processing program, various parameters, measurement data, and the like. The processor executes the processing program stored in the memory. Each process is performed by the processor executing the processing program. The processor may be, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a GPU (Graphics Processing Unit), etc.

[0031] The processing device 201 is connected to a microphone unit 2 and headphones 43. The microphone unit 2 may be built into the headphones 43. The microphone unit 2 includes a left microphone 2L and a right microphone 2R. The left microphone 2L is attached to the left ear 9L of the user U. The right microphone 2R is attached to the right ear 9R of the user U. The processing device 201 may be the same processing device as the out-of-head localization processing device 100, or may be a different processing device. Also, earphones can be used instead of the headphones 43.

[0032] The headphones 43 have a headphone band 43B, a left unit 43L, and a right unit 43R. The headphone band 43B connects the left unit 43L and the right unit 43R. The left unit 43L outputs sound toward the left ear 9L of the user U. The right unit 43R outputs sound toward the right ear 9R of the user U. The headphones 43 may be of any type, such as a closed type, an open type, a semi-open type, or a semi-closed type. The user U wears the headphones 43 with the microphone unit 2 attached to the user U. That is, the left unit 43L and the right unit 43R of the headphones 43 are attached to the left ear 9L and the right ear 9R to which the left microphone 2L and the right microphone 2R are attached, respectively. The headphone band 43B generates a biasing force that presses the left unit 43L and the right unit 43R against the left ear 9L and the right ear 9R, respectively.

[0033] The left microphone 2L picks up the sound output from the left unit 43L of the headphones 43. The right microphone 2R picks up the sound output from the right unit 43R of the headphones 43. The microphone portions of the left microphone 2L and the right microphone 2R are disposed at sound collection positions near the external ear canals. The left microphone 2L and the right microphone 2R are configured so as not to interfere with the headphones 43. In other words, the user U can wear the headphones 43 with the left microphone 2L and the right microphone 2R disposed at appropriate positions on the left ear 9L and right ear 9R.

[0034] The processing device 201 outputs a measurement signal to the headphones 43. This causes the headphones 43 to generate an impulse sound or the like. Specifically, the impulse sound output from the left unit 43L is measured by the left microphone 2L. The impulse sound output from the right unit 43R is measured by the right microphone 2R. When the measurement signal is output, the microphones 2L and 2R acquire the picked-up signal, thereby performing impulse response measurement.

[0035] The processing device 201 performs similar processing on the collected sound signals from the microphones 2L and 2R to generate the inverse filters Linv and Rinv. Specifically, the processing device 201 performs processing to compress the peaks of the frequency characteristics of the collected sound signals.

[0036] The processing device 201 of the measurement device 200 and its processing will be described in detail below. Fig. 3 is a control block diagram showing the processing device 201. The processing device 201 includes a measurement signal generating unit 211, a picked-up signal acquiring unit 212, an inverse filter generating unit 213, a frequency characteristic acquiring unit 214, an axis conversion unit 215, an extreme value extracting unit 216, a kurtosis calculating unit 217, and a filter generating unit 221.

[0037] The measurement signal generating unit 211 includes a D / A converter, an amplifier, and the like, and generates a measurement signal for measuring ear canal transfer characteristics. The measurement signal is, for example, an impulse signal or a TSP (Time Stretched Pulse) signal. Here, the measuring device 200 performs impulse response measurement using an impulse sound as the measurement signal.

[0038] The left microphone 2L and the right microphone 2R of the microphone unit 2 each pick up a measurement signal and output the picked-up signal to the processing device 201. The picked-up signal acquisition unit 212 acquires the picked-up signals picked up by the left microphone 2L and the right microphone 2R. The picked-up signal acquisition unit 212 may include an A / D converter that performs A / D conversion of the picked-up signals from the microphones 2L and 2R. The picked-up signal acquisition unit 212 may synchronously add signals obtained by multiple measurements. The picked-up signal in the time domain is called an ECTF.

[0039] The inverse filter generation unit 213 generates an inverse filter that cancels the ear canal transfer characteristic based on the collected sound signal as an input signal. The inverse filter generation unit 213 calculates the frequency characteristic of the collected sound signal by discrete Fourier transform or discrete cosine transform. For example, the inverse filter generation unit 213 calculates the frequency characteristic by performing FFT (fast Fourier transform) on the input signal in the time domain. The frequency characteristic includes an amplitude spectrum and a phase spectrum. Note that the inverse filter generation unit 213 may generate a power spectrum instead of the amplitude spectrum.

[0040] The inverse filter generation unit 213 obtains an inverse characteristic that cancels the amplitude spectrum. The inverse characteristic becomes an amplitude spectrum having a filter coefficient that cancels the amplitude spectrum. The inverse filter generation unit 213 calculates a time domain signal from the inverse characteristic and the phase characteristic by inverse discrete Fourier transform or inverse discrete cosine transform. The inverse filter generation unit 213 generates a time signal by performing IFFT (inverse fast Fourier transform) on the inverse characteristic and the phase characteristic. The inverse filter generation unit 213 calculates an inverse filter by cutting out the generated time signal with a predetermined filter length. The inverse filter generation unit 213 may perform windowing to generate the inverse filter. The inverse filter generation unit 213 outputs the generated inverse filter to the frequency characteristic acquisition unit 214 as an input signal.

[0041] The frequency characteristic acquisition unit 214 acquires the frequency characteristic of the input signal. The frequency characteristic acquisition unit 214 calculates the frequency characteristic of the input signal by discrete Fourier transform or discrete cosine transform. The frequency characteristic acquisition unit 214 calculates the frequency characteristic by, for example, performing FFT (Fast Fourier Transform) on the input signal in the time domain. The frequency characteristic includes an amplitude spectrum and a phase spectrum. Note that the frequency characteristic acquisition unit 214 may generate a power spectrum instead of the amplitude spectrum. In this way, the frequency characteristic acquisition unit 214 acquires the frequency characteristic of the inverse filter, which is the input signal.

[0042] The axis conversion unit 215 converts the frequency axis of the frequency characteristics acquired by the frequency characteristic acquisition unit 214 by data interpolation. The axis conversion unit 215 changes the scale of the data of the frequency amplitude characteristics so that the discrete spectrum data is equally spaced on the logarithmic axis. The data of the frequency amplitude characteristics (also called gain data) obtained by the frequency characteristic acquisition unit 214 is equally spaced in frequency. In other words, since the gain data is equally spaced on the frequency linear axis, it is not equally spaced on the frequency logarithmic axis. Therefore, the axis conversion unit 215 performs an interpolation process on the gain data so that the gain data is equally spaced on the frequency logarithmic axis.

[0043] In gain data, on the logarithmic axis, the lower the frequency range, the coarser the interval between adjacent data, and the higher the frequency range, the closer the interval between adjacent data. Therefore, the axis conversion unit 215 interpolates data in the low frequency band where the data intervals are coarse. Specifically, the axis conversion unit 215 performs an interpolation process such as three-dimensional spline interpolation to obtain discrete gain data that are equally spaced on the logarithmic axis. The gain data that has undergone axis conversion is called axis converted data. The axis converted data is a spectrum in which frequencies and amplitude values ​​(gain values) are associated with each other.

[0044] The reason for converting the frequency axis into a logarithmic scale will be explained. It is generally said that human sensory quantities are converted into logarithms. Therefore, it is important to consider the frequency of audible sounds on a logarithmic axis. By converting the scale, data is equally spaced in the above sensory quantities, so that data can be treated equally in all frequency bands. As a result, mathematical operations, frequency band division and weighting become easier, and stable results can be obtained. The axis conversion unit 215 may convert the envelope data into a scale close to human hearing (referred to as a hearing scale) instead of a logarithmic scale. The hearing scale may be a logarithmic scale (Log scale), a mel scale, a Bark scale, an Equivalent Rectangular Bandwidth (ERB) scale, or the like.

[0045] The axis conversion unit 215 performs scale conversion of the gain data on the auditory scale by data interpolation. For example, the axis conversion unit 215 makes the data in the low frequency band, which has a coarse data interval on the auditory scale, denser by interpolating the data in the low frequency band. Data that is equally spaced on the auditory scale becomes data that is dense in the low frequency band and coarse in the high frequency band on a linear scale. In this way, the axis conversion unit 215 can generate axis-converted data that is equally spaced on the auditory scale. Of course, the axis-converted data does not have to be data that is completely equally spaced on the auditory scale.

[0046] In this way, the axis transformation data has spectrum data based on the frequency characteristics of the input signal. Fig. 4 is a graph showing an example of the spectrum data of the axis transformation data. In Fig. 4, the horizontal axis is frequency [Hz], and the vertical axis is amplitude value (gain) [dB]. Fig. 4 shows the waveforms of the spectrum before and after the peak compression process (before and after the peak suppression process) according to this embodiment.

[0047] The extreme value extraction unit 216 calculates the extreme values ​​of the spectrum data based on the frequency characteristics. That is, the extreme value extraction unit 216 calculates the extreme values ​​of the axis transformed data. The extreme values ​​of the spectrum data correspond to peaks or dips of the spectrum data. That is, a maximum value corresponds to a peak, and a minimum value corresponds to a dip.

[0048] An example in which the extreme value extraction unit 216 calculates maximum values ​​(peaks) will be described with reference to Fig. 5, but minimum values ​​(dip) may also be extracted. As shown in Fig. 5, the extreme value extraction unit 216 extracts six maximum values ​​as peaks P1 to P6. Each of the peaks P1 to P6 has data on the peak frequency (the center frequency of the peak, i.e., the frequency at the maximum value) and the gain at the peak frequency.

[0049] Furthermore, the extreme value extraction unit 216 may extract extreme values ​​from all bands of the frequency spectrum, or from only some bands. Here, the extreme value extraction unit 216 extracts only maximum values ​​from some bands of the frequency amplitude characteristics. As shown in FIG. 5, an extreme value search band to be subjected to compression processing is set in advance. The extreme value extraction unit 216 searches for peaks only in the extreme value search range. In other words, the extreme value extraction unit 216 does not extract extreme values ​​outside the extreme value search range. Therefore, peak compression, which will be described later, is not performed outside the extreme value search range.

[0050] In the above description, the extreme value extraction unit 216 extracts extreme values ​​of the amplitude spectrum of the axis-transformed data, but the extreme value extraction unit 216 may extract extreme values ​​of the frequency-amplitude characteristics before axis transformation by the axis transformation unit 215. The spectral data processed by the extreme value extraction unit 216 is not limited to axis-transformed data as long as it is spectral data based on frequency characteristics. For example, the extreme value extraction unit 216 may extract extreme values ​​of spectral data obtained by performing smoothing processing on the frequency characteristics or axis-transformed data of the input signal.

[0051] The kurtosis calculation unit 217 calculates the kurtosis of each of the peaks P1 to P6. The kurtosis of a peak is an index indicating how steep the peak is. For example, the higher the kurtosis of a peak, the steeper the peak is, and the lower the kurtosis of a peak, the broader the peak is. An example of a method for calculating kurtosis will be described below.

[0052] In this embodiment, the kurtosis calculation unit 217 calculates the kurtosis using a kurtosis function. The kurtosis function is a function that calculates an evaluation value for evaluating a peak. Specifically, the kurtosis function calculates an evaluation value using a frequency function and a gain function. The evaluation value is a value that evaluates a peak corresponding to an extreme value. Specifically, the extreme value is a value that evaluates the shape and kurtosis of a peak.

[0053] The evaluation value is represented by the product of the gain function and the frequency function (gain function*frequency function). Then, the kurtosis calculation unit 217 calculates the kurtosis function based on the evaluation value. Specifically, the kurtosis calculation unit 217 obtains the kurtosis function by the following formula (1). Kurtosis function = max{gain function * frequency function} (1)

[0054] The gain function and the frequency function are calculated for each peak. An example of calculating the gain function and the frequency function will be described below. In the following description, it is assumed that the position on the frequency axis (frequency position) is indicated by the order of data (integer). For example, in the spectrum of discrete axis-transformed data, the order of data counted from the lowest frequency indicates the frequency position.

[0055] The gain function and frequency function values ​​of one peak are calculated over the range W n The calculation width W n is a value that indicates how far away the frequency is from the peak frequency (extremum). Since the frequency position is expressed as an integer, W n is 1 or more W std (W std is an integer in the range of 2 or more). std is the calculation width W n W is an integer that indicates the maximum width (maximum value) of std can be set by the user. std is a parameter related to the frequency width of the peak to be detected. The user can set W based on the maximum width of the peak to be detected. std Just set it as:

[0056] The frequency at the maximum value is the peak frequency f p With the peak frequency f p From calculation width W n The frequency separated by f n Since the position of the discrete spectrum data on the frequency axis is expressed by the order of the data, f n =(f p +W n ), or f n =(f p -W n For example, in discrete spectrum data, the peak frequency f p is the 1000th data from the low frequency side, and W n If we set =100, then f n =900 or f n =1100.

[0057] The kurtosis calculation unit 217 calculates the kurtosis n In other words, the kurtosis calculation unit 217 calculates the values ​​of the gain function and the frequency function while gradually changing the calculation width W n 1, 2, . . . , W std The gain function and frequency function are calculated for each calculation width.

[0058] An example of the gain function is expressed by the following equation (2). Gain function = {(G p -G n ) / G std}^G m (2)

[0059] G p is the peak frequency f p Gain [dB] at the peak (maximum value) n is the frequency f n The gain [dB] at the calculation width W n By changing the gain G n Generally, the calculation width W n As the gain G n becomes smaller.

[0060] G std is a parameter (gain reference value) related to the gain intensity to be detected. std When G is large, the low peaks are not detected, and when G is small, the low peaks are detected. std As G approaches 0, the gain function approaches infinity. This means that the balance between the gain function and the frequency function is lost, so care must be taken. m is a parameter (gain multiplier) that determines the gain sensitivity of the detected peak. G m Increasing this will make the gain more sensitive to areas with large gain gradients.

[0061] G n is the calculation width W n It is a variable that changes by changing G p is a constant value determined for each peak. G m is a constant value set by the user. For example, G std is the maximum width W std When G is determined, it becomes a constant value determined for each peak. std does not have to be a constant value determined for each peak. For example, G stdmay be a value set by the user.

[0062] An example of the calculation of the gain function will be described with reference to FIG. 6. n 6 shows a calculation example for the case of f = 100. FIG. 6 shows the spectrum around the peak P2 shown in FIG. 5. As described above, the peak frequency f p The gain at is G p The peak frequency f p to 100 (=W n ) apart frequencies are called f n(n=100) The frequency f n(n=100) The gain at G n(n=100) Let us assume that.

[0063] In Figure 6, f n(n=100) =(f p +W n ) Therefore, (f p +W n ) is the gain at G n(n=100) In other words, (f p -W n The gain at f is not used. n(n=100) (f p +W n ) or (f p -W n ) can be determined by comparing the gains of each. Specifically, (f p +W n ) and (f p -W n ) the frequency with the larger gain is f n(n=100) For example, in Figure 6, (f p +W n The gain at (f p -W n ) is larger than the gain at f n(n=100) =(f p +W n ), and (f p +W n ) to set the gain to Gain G n Let us assume that.

[0064] Similarly, the maximum width Wstd Let the frequency at f n(n=Wstd) The frequency f n(n=Wstd) The gain at G n(n=Wstd) In Figure 6, f n(n=Wstd) =(f p +W std ) In other words, (f p -W std The gain at (f p +W std ) is the gain at G n(n=Wstd) It is as follows.

[0065] f n(n=Wstd) (f p +W std ) or (f p -W std ) can be determined by comparing the gains of each. Specifically, (f p +W std ) and (f p -W std ) the frequency with the larger gain is f n(n=Wstd) For example, in Figure 6, (f p +W std The gain at (f p -W std ) is larger than the gain at f n(n=Wstd) =(f p +W std ), and (f p +W std ) to set the gain to Gain G n(n=Wstd) Let us assume that. std =G p -G n(n=Wstd) It becomes.

[0066] An example of the frequency function is shown in the following equation (3). Frequency function = {(W std -W n ) / W std}^W m (3)

[0067] W m is the calculation width W of the peak to be detected nis a parameter (frequency multiplier) that determines the sensitivity of W m If W is increased, it will only respond to narrow peaks. m When W becomes smaller, it becomes possible to respond to broad peaks. m can be assumed to be constant. std is a parameter related to the frequency width of the peak to be detected. The user can set W based on the maximum width of the peak to be detected. std Just set it as:

[0068] Depending on the peak shape you want to compress, you can adjust the W m , G m , W std , G std In other words, the user can set parameters such as W depending on how steep the peaks are to be compressed. m , G m , W std , G std Adjust.

[0069] The kurtosis calculation unit 217 calculates the width W n By changing the frequency function and the gain function, G m and W m The shape of the peak that is judged to have high kurtosis is determined by the balance between these two.

[0070] The kurtosis calculation unit 217 calculates the kurtosis n In other words, the kurtosis calculation unit 217 calculates the frequency function and the gain function by changing the calculation width W n The frequency f n and its gain G n Substitute the values ​​of into equations (2) and (3). Here, W m , G m , W std is a constant value set by the user. std When you set std is determined for each peak. Of course, the user can std You may set the value of

[0071] Therefore, the kurtosis calculation unit 217 calculates the value of the frequency function and the value of the gain function for one calculation width. n W1~W std By incrementing by 1 in the range up to std The values ​​of the frequency functions and W std Calculate the gain function values. In general, the calculation width W n Increasing decreases the frequency function and increases the gain function.

[0072] The kurtosis calculation unit 217 calculates the product of the frequency function and the gain function as an evaluation value. std The kurtosis calculation unit 217 calculates evaluation values ​​W std The maximum value of these evaluation values ​​is calculated as the kurtosis.

[0073] In this way, the kurtosis calculation unit 217 calculates the frequency f n Gain G n Then, the kurtosis calculation unit 217 calculates an evaluation value for evaluating the peak from the calculation width W n The kurtosis of the peaks is calculated based on a plurality of evaluation values ​​calculated by changing the kurtosis calculation unit 217. Furthermore, the kurtosis calculation unit 217 calculates the kurtosis for each of the peaks P1 to P6. Here, since six peaks P1 to P6 are extracted, the kurtosis calculation unit 217 calculates six kurtosis values.

[0074] The determination unit 218 determines whether or not to compress each peak based on the kurtosis. The determination unit 218 compares the kurtosis of the peak with a threshold, and determines whether or not to compress the peak based on the comparison result. If the kurtosis of the peak is equal to or greater than the threshold, the determination unit 218 determines that the peak is to be compressed. If the kurtosis of the peak is less than the threshold, the determination unit 218 determines that the peak is not to be compressed.

[0075] The compression unit 219 compresses the peaks that are determined to be compressed. That is, the compression unit 219 performs compression processing on peaks whose kurtosis is equal to or greater than a threshold value. For example, the compression unit 219 replaces the peaks with attenuated characteristics using a polynomial curve. This makes it possible to suppress steep peaks.

[0076] For example, the compression unit 219 suppresses the peak by replacing it with a Bezier curve calculated using three points obtained by multiplying both ends of the peak and the maximum point of the peak by a predetermined damping coefficient. This reduces the gain of the peak. This method is an example of replacement, and the characteristics to be replaced are not limited to the calculation result using a Bezier curve. The both ends of the peak can be set, for example, by the calculation width when the evaluation value becomes maximum.

[0077] In the above description, the process of compressing the gain of the peaks is performed, but the process of compressing the gain of the dips (minimum values) may be performed. In this case, the extreme value extraction unit 216 extracts the minimum values ​​as dips. The kurtosis calculation unit 217 calculates the kurtosis for each of the extracted dips. The kurtosis calculation unit 217 can calculate the kurtosis by processing the peaks with positive and negative inversion. This increases the gain of the dips.

[0078] The axis conversion unit 220 performs axis conversion so as to convert the frequency axis of the spectrum data having compressed peaks by data interpolation or the like. The process in the axis conversion unit 220 is the opposite process to the process in the axis conversion unit 215. By performing axis conversion, the axis conversion unit 220 returns the frequency axis of the spectrum data to the frequency axis before the axis conversion in the axis conversion unit 215. For example, the axis conversion unit 220 performs a process to return the frequency axis that has been logarithmically scaled in the axis conversion unit 215 to a linear scale. The spectrum data with compressed peaks is converted to data with equal intervals on the frequency linear axis. This makes it possible to obtain frequency amplitude characteristics on the same frequency axis as the frequency phase characteristics acquired by the frequency characteristic acquisition unit 214. In other words, the frequency axes (data intervals) of the spectrum data of the frequency phase characteristics and the frequency amplitude characteristics match.

[0079] The filter generation unit 221 generates a filter using the spectrum data axis-converted by the axis conversion unit 220. The filter generation unit 221 converts the frequency characteristic indicated by the compressed amplitude spectrum into a time domain signal. Here, the frequency characteristic has a frequency amplitude characteristic and a frequency phase characteristic. The compressed amplitude spectrum can be used as the frequency amplitude characteristic. The frequency phase characteristic can be the frequency phase characteristic obtained by the frequency characteristic acquisition unit 214.

[0080] The filter generation unit 221 generates a filter to be applied to the reproduction signal based on the spectrum data having the peaks compressed by the compression unit 219. For example, the filter generation unit 221 calculates a time domain signal from the frequency amplitude characteristic and the phase characteristic by inverse discrete Fourier transform or inverse discrete cosine transform. The filter generation unit 221 generates a time signal by performing IFFT (inverse fast Fourier transform) on the frequency amplitude characteristic and the phase characteristic. The filter generation unit 221 calculates a filter by cutting out the generated time signal with a predetermined filter length. The filter generation unit 221 may generate a filter by performing windowing.

[0081] The filter generated by the filter generation unit 221 is set as an inverse filter in the filter unit 41 and the filter unit 42 in Fig. 1. The processing device 201 generates the inverse filter Linv by performing the above-mentioned processing on the sound signal collected by the left microphone 2L. The processing device 201 generates the inverse filter Rinv by performing the above-mentioned processing on the sound signal collected by the right microphone 2R. The inverse filters Linv and Rinv are set in the filter units 41 and 42 in Fig. 1, respectively.

[0082] In this manner, in the present embodiment, the processing device 201 has the kurtosis calculation unit 217 set to the calculation width W n By changing the calculation width W nThe kurtosis is calculated based on a plurality of evaluation values ​​obtained by changing the kurtosis calculation unit 217. For example, the kurtosis calculation unit 217 calculates the maximum value of the plurality of evaluation values ​​as the kurtosis. In this way, the peaks can be appropriately compressed. Since peaks of various shapes can be appropriately evaluated, steep peaks can be removed. Stable sound quality and sound field can be provided. A robust filter that does not become unstable even when the headphones are put back on can be generated.

[0083] The measurement of the ear canal transfer characteristic of the subject 1 is performed using the microphone unit 2 and the headphones 43. Furthermore, the processing device 201 can be a smartphone or the like. For this reason, the measurement settings may differ for each measurement. In addition, there is a possibility that the wearing of the headphones 43 and the microphone unit 2 may vary. For example, the wearing position of the headphones 43 during measurement may differ from the wearing position of the headphones 43 during out-of-head localization listening. The processing device 201 suppresses the peaks or dips as described above. This makes it possible to generate an inverse filter of the ear canal transfer characteristic while suppressing the variation due to the measurement or the like.

[0084] In the compression unit 219, the filter generation unit 221 generates a filter using the spectrum corrected to suppress the peaks. This makes it possible to effectively suppress peaks that occur in the inverse filters Linv and Rinv. Therefore, more appropriate inverse filters Linv and Rinv can be generated.

[0085] Embodiment 2 The processing device and processing method according to this embodiment will be described with reference to Fig. 7. Fig. 7 shows spectrum data for explaining the processing of this embodiment. Specifically, Fig. 7 is a graph showing an enlarged view of the periphery of two adjacent peaks P4 and P5. The horizontal axis is frequency [Hz], and the vertical axis is amplitude (gain) [dB].

[0086] In the second embodiment, a process of merging two adjacent peaks is added to the process of the first embodiment. Since the process other than the merging process is the same as the first embodiment, the description will be omitted.

[0087] The processing device 201 calculates the peak-to-peak frequency distance between the two maximum values ​​extracted by the extreme value extraction unit. If the peak-to-peak frequency distance is equal to or less than the frequency threshold, the processing device 201 merges the two peaks. Specifically, in FIG. 7, the frequency at which the peak P4 has a maximum value is f4, and the frequency at which the peak P5 has a maximum value is f5. The frequency distance is (f5-f4). f5-f4 is expressed by an integer indicating the order of the data. The frequency distance is the distance on the frequency axis converted by the axis conversion unit 215.

[0088] The inter-peak frequency distance (f5-f4) between the adjacent peaks P4 and P5 is equal to or less than a threshold. Therefore, the processing device 201 merges the peaks P4 and P5 into one peak. The peak frequency of the merged peak may be a frequency between the peak frequencies f4 and f5, or may be equal to the peak frequency f4 or the peak frequency f5.

[0089] The interpolation method here may be linear interpolation or polynomial interpolation. Of course, an interpolation method other than linear interpolation or polynomial interpolation may be used. In this way, the peaks can be appropriately compressed.

[0090] 8 is a flowchart showing a processing method according to this embodiment. First, the frequency characteristic acquisition unit 214 acquires the frequency characteristic of the input signal (S801). For example, the input signal in the time domain is converted to the frequency domain by FFT or the like. The input signal is, for example, an inverse filter that cancels the ear canal transfer characteristic. The axis conversion unit 215 performs axis conversion on the frequency characteristic (S802). As a result, spectrum data in which the frequency axis of the collected sound signal is converted to a logarithmic axis is obtained.

[0091] The extreme value extraction unit 216 extracts extreme values ​​(S803). As a result, peaks corresponding to maximum values ​​are extracted. Next, the kurtosis calculation unit 217 calculates the kurtosis of the peaks (S804). The kurtosis calculation unit 217 calculates the kurtosis of the peaks using the calculation width W n The kurtosis calculation unit 217 calculates the kurtosis of the peak.

[0092] The determination unit 218 determines whether the kurtosis is 0.5 or more (S805). Here, the threshold value of the kurtosis is 0.5, but the threshold value is not limited to 0.5, and a value of 0.5 or more is preferable, and a value between 0.5 and 0.8 is more preferable. If the kurtosis is not 0.5 or more (NO in S805), the processing device 201 determines whether the processing for all peaks has been completed (S809). If the processing for all peaks has not been completed (NO in S809), the process returns to S804 and the process is repeated. That is, the kurtosis calculation unit 217 calculates the kurtosis of the next peak.

[0093] If the kurtosis of the peak is 0.5 or more (YES in S805), it is determined whether the peak-to-peak frequency distance is 100 or less (S806). Here, the frequency threshold of the peak-to-peak frequency distance is 100, but the frequency threshold may be a value other than 100.

[0094] If the peak-to-peak frequency distance is less than or equal to 100 (YES in S806), the compression unit 219 merges the peaks (S807). As a result, the two peaks are merged into one peak. The compression unit 219 then compresses the merged peak (S808). If the peak-to-peak frequency distance is not less than or equal to 100 (NO in S806), the compression unit 219 compresses the peaks (S808).

[0095] When the compression unit 219 compresses the peak in S808, the processing device 201 judges whether or not the processing for all peaks has been completed (S809). When the processing for all peaks has not been completed (NO in S809), the processing device 201 returns to S804 and repeats the processing. When the processing for all peaks has been completed (YES in S809), the processing device 201 ends the processing.

[0096] In this way, the peaks can be appropriately compressed. In the above description, the compression unit 219 merges peaks when the kurtosis is equal to or greater than the threshold and the peak-to-peak frequency distance is equal to or less than the frequency threshold. However, the compression unit 219 may also merge peaks when the kurtosis is less than the threshold. In other words, when the peak-to-peak frequency distance is equal to or less than the frequency threshold, the compression unit 219 may merge peaks regardless of the kurtosis. Furthermore, when the process of merging two peaks is not performed as in the first embodiment, steps S806 and S807 are omitted.

[0097] Other embodiments In the first and second embodiments, the processing device 201 compresses the peaks of the spectrum data based on the collected signal, but it may also compress the dips corresponding to the minimum values. In this case, the processing device 201 extracts the minimum values ​​and calculates the same kurtosis for the dips corresponding to the minimum values. At this time, the processing device 201 performs a process in which the positive and negative are inverted.

[0098] Furthermore, in the first and second embodiments, the processing device 201 processes the spectrum data of the input signal corresponding to the inverse filter of the ear canal transfer characteristic, but the processing device 201 may process the spectrum data based on the input signal indicating the spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs. Furthermore, the processing device 201 generates the outside-head low-level processing filter, but the processing device 201 may generate other filters. For example, the processing device 201 can also generate a noise suppression filter that suppresses peaks or dips. Furthermore, the processing of compressing peaks or dips can be applied to other purposes besides filter generation. For example, the processing device 201 can also perform noise reduction processing without using a filter.

[0099] The out-of-head localization processing device 100 or the processing device 201 is not limited to being a single physical device, and may be distributed among a plurality of devices connected via a network, etc. In other words, the out-of-head localization processing method or the processing method according to the present embodiment may be implemented by a plurality of distributed devices.

[0100] A part or all of the above processes may be executed by a computer program. The above-mentioned program includes a set of instructions (or software code) for making the computer perform one or more functions described in the embodiments when the program is loaded into the computer. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or the tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or the communication medium includes electrical, optical, acoustic, or other forms of propagating signals.

[0101] The invention made by the inventor has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention. [Explanation of symbols]

[0102] U User 1 Person to be measured 2 Microphone Unit 2L Left microphone 2R Right Microphone 5 Stereo speakers 5L Left speaker 5R Right speaker 10. Extra-head localization processing unit 11 Convolution Calculation Unit 12 Convolution Calculation Unit 21 Convolution Calculation Unit 22 Convolution Calculation Unit 24 Adder 25 Adder 41 Filter section 42 Filter section 43 Headphones 200 Measuring Equipment 201 Processing equipment 211 Measurement signal generator 212 Sound collection signal acquisition unit 213 Inverse filter generation unit 214 Frequency characteristic acquisition section 215 Axis conversion unit 216 Extreme Value Extraction Unit 217 Kurtosis Calculation Unit 218 Judgment section 219 Compression section 220 Axis conversion unit 221 Filter Generation Unit

Claims

1. A frequency characteristic acquisition unit that acquires a frequency characteristic of an input signal; an extreme value extracting unit that extracts extreme values ​​of the spectrum data based on the frequency characteristics; a kurtosis calculation unit that calculates an evaluation value for evaluating a peak or a dip corresponding to the extreme value from the spectrum data in a calculation range including the extreme value, and calculates a kurtosis of the peak or the dip based on a plurality of evaluation values ​​calculated by changing the calculation range; a determination unit that determines whether or not to compress the peak or dip depending on a result of comparing the kurtosis with a threshold value; a compression unit that compresses the peak or dip that is determined to be compressed as an extreme value.

2. The processing device according to claim 1 , wherein when a frequency distance between two maximum values ​​or a frequency distance between two minimum values ​​extracted by the extreme value extraction unit is equal to or less than a frequency threshold, the two peaks or two dips are merged.

3. an inverse filter generating unit that generates an inverse filter that cancels the ear canal transfer characteristic based on a sound signal collected by a microphone attached to the ear of the subject as the input signal; 3. The processing device according to claim 1, further comprising a filter generating unit that generates a filter to be applied to a reproduced signal based on the spectrum data having the peak or dip compressed by the compression unit.

4. a first axis conversion unit that converts a frequency axis of the frequency characteristic acquired by the frequency characteristic acquisition unit through data interpolation; a second axis conversion unit that converts a frequency axis of the spectrum data having the compressed peak or dip by data interpolation, The processing device according to claim 3 , wherein the filter generating section generates the filter based on the spectrum data axis-converted by the second axis conversion section.

5. obtaining a frequency characteristic of an input signal; extracting extreme values ​​of the frequency characteristics; calculating an evaluation value for evaluating the extreme value from data in a calculation range including the extreme value, and calculating the kurtosis of the extreme value based on a plurality of evaluation values ​​calculated by changing the calculation range; determining whether to compress the extreme values ​​according to a comparison result between the kurtosis and a threshold; compressing the extreme values ​​determined to be compressed.

Images