Filter information generation device, method, and program

Abstract

Provided is a filter information generation device in which a plurality of impulse responses respectively corresponding to a plurality of different conditions from the input of a wearable device that reproduces sound without completely blocking the external auditory opening to the external auditory canal entrance of a wearer wearing the wearable device are measured in advance, and the plurality of impulse responses are clustered into a plurality of clusters, the filter information generation device comprising: a filter information generation impulse response determination unit 11 that selects any one from among the plurality of clusters, and determines a filter information generation impulse response from at least one impulse response included in the selected cluster; and a filter information generation unit 12 that generates, from the filter information generation impulse response, filter information which is information relating to a filter for weakening an acoustic characteristic of the wearable device that is determined from the filter information generation impulse response and for performing filter processing on a signal to be reproduced by the wearable device.

Description

Filter information generation device, method, and program

[0001] The disclosed technology relates to a technology for generating information about a filter.

[0002] Wearable devices that reproduce sound without completely blocking the ear canal include open-ear earphones, shoulder speakers, audio glasses, and smart glasses.

[0003] It is known that sounds reproduced from such wearable devices do not localize to the expected position due to the influence of the headphone transfer function (HpTF) from the speaker of the wearable device to the entrance of the ear canal.

[0004] Scharer, Zora, and Alexander Lindau, "Evaluation of equalization methods for binaural signals", Audio Engineering Society Convention 126, Audio Engineering Society, 2009

[0005] One possible method to reduce the above-mentioned effects is to generate one correction filter from multiple impulse responses. However, if the average of the time-domain transfer function calculated from the multiple impulse responses and the average of the frequency-domain transfer function calculated from the multiple impulse responses have different characteristics, the generated correction filter may have characteristics different from those expected.

[0006] The disclosed technology aims to provide a filter information generating device, method, and program for generating filter information, which is information about an appropriate filter, from a plurality of impulse responses.

[0007] A filter information generation device, which is one aspect of the disclosed technology, measures in advance a plurality of impulse responses corresponding to a plurality of different conditions from the input of a wearable device that reproduces sound without completely blocking the external auditory canal to the entrance of the ear canal of a wearer wearing the wearable device, and the plurality of impulse responses are clustered into a plurality of clusters. The filter information generation device includes: an impulse response determination unit for filter information generation that selects one of the plurality of clusters and determines an impulse response for filter information generation from at least one impulse response included in the selected cluster; and a filter information generation unit that generates, from the impulse response for filter information generation, filter information that is information about a filter for performing filtering on a signal that the wearable device is to reproduce, and that is used to weaken the acoustic characteristics of the wearable device as determined from the impulse response for filter information generation.

[0008] According to the disclosed technique, information about an appropriate filter can be generated from multiple impulse responses.

[0009] Fig. 1 is a diagram showing an example of the functional configuration of a filter information determination device. Fig. 2 is a diagram showing an example of a processing procedure of a filter information determination method. Fig. 3 is a diagram for explaining the processing of a filter information generation impulse response determination unit. Fig. 4 is a diagram for explaining the processing of a filter information generation impulse response determination unit. Fig. 5 is a diagram showing an example of the functional configuration of a computer. Fig. 6 is a diagram showing an example of experimental results. Fig. 7 is a diagram showing an example of experimental results.

[0010] Hereinafter, embodiments of the disclosed technology will be described with reference to the drawings. Note that components having the same functions in the drawings are given the same reference numerals, and redundant description will be omitted.

[0011] [Filter Information Generating Device and Method] As shown in Fig. 1, the filter information generating device includes, for example, a filter information generating impulse response determining unit 11 and a filter information generating unit 12. The filter information generating device may further include a filter processing unit 13 indicated by a dashed line in Fig. 1.

[0012] The filter information determination method is realized, for example, by each unit of the filter information determination device performing the processes of steps S11 and S12 shown in FIG.

[0013] First, a plurality of impulse responses input to the filter information generating impulse response determination unit 11 will be described.

[0014] Assume that multiple impulse responses corresponding to different conditions from the input of a wearable device that reproduces sound without completely blocking the ear canal to the entrance of the ear canal of a wearer wearing the wearable device have been measured in advance.

[0015] Examples of wearable devices that reproduce sound without completely blocking the ear canal include open-ear earphones, shoulder speakers, audio glasses, and smart glasses. Hereinafter, wearable devices that reproduce sound without completely blocking the ear canal may be abbreviated as "wearable devices."

[0016] For example, a microphone is placed at the entrance to the ear canal of a person wearing a wearable device, and the TSP signal is played back from the wearable device to measure the impulse response of that person. By performing this measurement for each of the left and right ears of multiple people, multiple impulse responses can be obtained.

[0017] The plurality of pre-measured impulse responses are input to the filter information generation impulse response determination unit 11 .

[0018] It is assumed that the impulse responses are clustered into a plurality of clusters, and that this clustering has also been performed in advance.

[0019] An example of a method for clustering multiple impulse responses will be described below.

[0020] An example of this is clustering multiple impulse responses in the frequency domain.

[0021] In other words, in this example, the impulse responses are clustered into a plurality of clusters based on the value of the amplitude characteristic of the transfer function obtained by Fourier transforming the impulse responses, such as the value of the logarithmic power spectrum indicating the amplitude characteristic of the transfer function.

[0022] For example, the amplitude characteristics of a transfer function obtained by Fourier transforming an impulse response are compressed to a number of dimensions with a total contribution rate exceeding 95% by principal component analysis. Clustering is performed based on the values ​​of the amplitude characteristics of this dimensionally compressed transfer function. Clustering is performed using an existing method such as the k-means method.

[0023] <Filter Information Generating Impulse Response Determining Unit 11> The filter information generating impulse response determining unit 11 receives a plurality of impulse responses measured in advance.

[0024] The filter information generation impulse response determination unit 11 selects one of the plurality of clusters.

[0025] When the wearer is associated with any one of the multiple clusters, the filter information generation impulse response determination unit 11 selects the cluster corresponding to the wearer of the wearable device that reproduces sound without completely blocking the external auditory canal.

[0026] When information specifying a cluster is input to the filter information generation impulse response determination unit 11, the filter information generation impulse response determination unit 11 selects a cluster specified by the input cluster specifying information.

[0027] For example, a wearer of a wearable device that reproduces sound without completely blocking the external ear canal may be able to input information that specifies a cluster to the filter information generation impulse response determiner 11. This allows a wearer of a wearable device that reproduces sound without completely blocking the external ear canal to select a cluster of their choice, in other words, a sound image localization of their choice.

[0028] Furthermore, a third party other than the wearer of the wearable device that reproduces sound without completely blocking the external ear canal, such as an acoustic engineer, or a device other than the filter information generating device may be able to input information that identifies a cluster to the filter information generation impulse response determination unit 11.

[0029] When pinna shape information, which is information about the pinna shape of a wearer of a wearable device that reproduces sound without completely blocking the external auditory canal, can be acquired, and when a cluster corresponding to the pinna shape information is predetermined, the filter information generation impulse response determination unit 11 may select a cluster corresponding to the acquired pinna shape information.

[0030] The pinna shape information can be acquired by, for example, capturing an image using a camera provided on an information processing device such as a smartphone, tablet terminal, or PC held by the wearer. The pinna shape information may be pinna shape information when a wearable device that reproduces sound without completely blocking the external ear canal is worn, or may be pinna shape information when a wearable device that reproduces sound without completely blocking the external ear canal is not worn.

[0031] A model that receives pinna shape information as input and outputs an appropriate cluster may be trained in advance. In this case, the filter information generation impulse response determiner 11 may select a cluster by obtaining an output when the pinna shape information of the wearer is input to this trained model.

[0032] The filter information generation impulse response determination unit 11 determines an impulse response for generating filter information from at least one impulse response included in the selected cluster. The determined impulse response for generating filter information is output to the filter information generation unit 12.

[0033] Below, <<Method (1)>> and <<Method (2)>>, which are examples of processing by the filter information generating impulse response determination unit 11, will be described.

[0034] <<Method (1)>> In method (1), each of a plurality of clusters formed by a plurality of impulse responses is associated with k impulse responses, each of which corresponds to k transfer functions having k amplitude characteristics that are close to the average value of the amplitude characteristics of the transfer functions corresponding to at least one impulse response included in each of the plurality of clusters, where k is a predetermined positive integer.

[0035] Then, the filter information generating impulse response determination unit 11 determines the average value of the k impulse responses associated with the selected cluster as the filter information generating impulse response.

[0036] For example, suppose multiple impulse responses are divided into three clusters as shown in Figure 3. As a result, the amplitude characteristic values ​​of the transfer functions corresponding to the multiple impulse responses are divided into three groups as shown in Figure 3. The selected cluster is cluster C1 in Figure 3. The average value A1 of the amplitude characteristic values ​​of the transfer functions obtained by Fourier transforming the impulse responses included in cluster C1 is shown as a white circle in Figure 3. In this example, k=3. Among the impulse responses included in cluster C1, there are three transfer functions that have amplitude characteristics H1, H2, and H3 that are close to the average value A1.

[0037] In this case, the filter information generating impulse response determination unit 11 determines the average value of three impulse responses corresponding to three transfer functions having three amplitude characteristics H1, H2, and H3, respectively, as the filter information generating impulse response.

[0038] Impulse responses other than the k impulse responses corresponding to the k transfer functions each having the k amplitude characteristics closest to the average value can be said to be impulse responses with large errors. Therefore, method (1) can be said to be a method of obtaining impulse responses for generating filter information from impulse responses in the selected cluster, excluding impulse responses with large errors.

[0039] The average value of the k impulse responses associated with each cluster may be calculated in advance. In this case, the filter information generation impulse response determination unit 11 selects the average value of the k impulse responses associated with the selected cluster from the pre-calculated average values ​​and determines it as the filter information generation impulse response.

[0040] <<Method (2)>> In method (2), each of the plurality of clusters is associated with a representative transfer function, which is a transfer function having amplitude characteristics closest to the average value of the amplitude characteristics of transfer functions corresponding to at least one impulse response included in each of the plurality of clusters.

[0041] Then, the filter information generation impulse response determination unit 11 determines the impulse response corresponding to the representative transfer function associated with the selected cluster as the filter information generation impulse response.

[0042] For example, suppose multiple impulse responses are divided into three clusters as shown in Figure 4. As a result, the amplitude characteristic values ​​of the transfer functions corresponding to the multiple impulse responses are divided into three groups as shown in Figure 3. The selected cluster is cluster C1 in Figure 3. The average value A1 of the amplitude characteristic values ​​of the transfer functions obtained by Fourier transforming the impulse responses included in cluster C1 is shown by a white circle in Figure 4. Among the impulse responses included in cluster C1, the amplitude characteristic closest to the average value A1 is assumed to be amplitude characteristic H1. Therefore, in this case, the transfer function having amplitude characteristic H1 is associated with cluster C1 as the representative transfer function.

[0043] In this case, the filter information generation impulse response determination unit 11 determines the impulse response corresponding to the transfer function (representative transfer function) having the amplitude characteristic H1 associated with the selected cluster as the filter information generation impulse response.

[0044] The representative transfer function, which is the transfer function with the amplitude characteristic closest to the average value, can be said to be a transfer function with high similarity. Therefore, method (2) can be said to be a method in which the impulse response corresponding to the transfer function with high similarity in the selected cluster is used as the impulse response for generating filter information.

[0045] <Filter Information Generating Unit 12 > The filter information generating unit 12 receives the filter information generating impulse response determined by the filter information generating impulse response determining unit 11 .

[0046] The filter information generation unit 12 is a filter for weakening the acoustic characteristics of the wearable device determined from the impulse response for generating filter information, and generates filter information, which is information about the filter for performing filtering on the signal that the wearable device is to reproduce, from the input impulse response for generating filter information.

[0047] Examples of filters for reducing the acoustic characteristics of a wearable device that reproduces sound without completely blocking the ear canal include an inverse filter of the HpTF and an approximation filter of the inverse filter of the HpTF.

[0048] The filter information generation unit 12 generates a filter for weakening the acoustic characteristics of a wearable device that reproduces sound without completely blocking the external auditory canal from the impulse response for generating filter information, for example, using the method described in non-patent document 1.

[0049] In this method, for example, a filter H is generated based on the following equation: c (x) is generated. x is the frequency. H c (x) is a filter generated by the filter information generating unit 12. D * (x) is a specific bandpass filter. * means a complex conjugate number. A bandpass filter is a filter that passes signals with frequencies between 100 Hz and 20,000 Hz, for example. D *(x) is determined appropriately to obtain a desired result. H(x) is the frequency characteristic of a transfer function that can be obtained by Fourier transforming the impulse response for generating filter information. B(x) is a predetermined high-pass filter for regularization. B(x) is determined appropriately to obtain a desired result. β is a predetermined weight for regularization. β is determined appropriately to obtain a desired result.

[0050] Of course, the filter information generation unit 12 may generate a filter for weakening the acoustic characteristics of a wearable device that reproduces sound without completely blocking the external auditory canal from the impulse response for generating filter information using a method other than the method described in Non-Patent Document 1.

[0051] The generated filter information is output from the filter information generation unit 12. For example, the filter information is output to the filter processing unit 13 shown by the dashed line in FIG. 1 . The filter processing unit 13 performs filter processing determined by the input filter information on a playback signal, which is a signal to be played back by a wearable device that plays back sound without completely blocking the external ear canal, and outputs a filter-processed playback signal. Examples of filter processing include FIR filter processing using a simple convolution, overlap-add method, or overlap-save method. After that, sound corresponding to the filter-processed playback signal is output from the wearable device that plays back sound without completely blocking the external ear canal.

[0052] The above process allows information about an appropriate filter to be generated from multiple impulse responses, thereby generating a correction filter with the desired frequency characteristics, and providing the wearer with a highly accurate stereophonic effect, for example.

[0053] [Other Modifications] The specific configurations of the embodiments of the disclosed technology are not limited to the configurations described above. The specific configurations of the embodiments of the disclosed technology can be appropriately modified in design, etc., within the scope of the spirit of the embodiments of the disclosed technology.

[0054] The various processes described in the embodiments of the disclosed technology may not only be performed chronologically in the order described, but may also be performed in parallel or individually depending on the processing capacity of the device performing the processes or as needed.

[0055] For example, data may be exchanged directly between the components of the filter information determination device, or may be exchanged via a storage unit (not shown).

[0056] [Program, Recording Medium] The processing of each unit of the above-described filter information determination device may be realized by a computer. In this case, the processing contents of the functions that the filter information determination device should have are described by a program. Then, by loading this program into storage unit 1020 of computer 1000 shown in Fig. 5 and causing arithmetic processing unit 1010, input unit 1030, output unit 1040, display unit 1060, etc. to operate, the various processing functions of the above-described filter information determination device are realized on the computer.

[0057] The program describing the processing contents can be recorded on a computer-readable recording medium, such as a non-transitory recording medium, specifically a magnetic recording device, an optical disk, or the like.

[0058] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0059] A computer executing such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own non-transitory storage device, auxiliary storage unit 1050. Then, when executing a process, the computer loads the program stored in auxiliary storage unit 1050, its own non-transitory storage device, into storage unit 1020 and executes processing in accordance with the loaded program. Alternatively, as another form of execution of this program, the computer may load the program directly from a portable recording medium into storage unit 1020 and execute processing in accordance with the program. Furthermore, the computer may execute processing in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the processing function by issuing an execution instruction and obtaining the results, thereby executing the above-described processing through a so-called ASP (Application Service Provider) type service. Note that, in this embodiment, the program includes information used for processing by a computer that is equivalent to a program (e.g., data that is not a direct instruction to the computer but has properties that define computer processing).

[0060] In this embodiment, the present device is configured by executing a predetermined program on a computer, but at least a part of the processing may be realized by hardware. For example, the filter information generation impulse response determiner 11 and the filter information generator 12 may be configured by a processing circuit.

[0061] It goes without saying that other modifications are possible without departing from the spirit of the present invention.

[0062] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0063] [Experimental Results] A microphone was placed at the entrance to the ear canal of a person wearing a wearable device, and the impulse response was measured by playing a TSP signal from the wearable device. This was done five times for each ear of 27 people, and 270 pieces of impulse response data were obtained.

[0064] The logarithmic power spectrum of the HpTF amplitude characteristics was calculated by Fourier transforming each of the 270 impulse responses. Based on the average of the logarithmic power spectra corresponding to the impulse response data measured five times in the same ear (54 in total), clustering was performed using the k-means method into three clusters. The range of 187.5–10031.25 Hz was used, and normalization was performed based on the value of 187.5–2015.625 Hz. Furthermore, principal component analysis was used to reduce the number of dimensions to a total contribution rate of more than 95%.

[0065] For the 22 × 5 data corresponding to the 22 data included in cluster A before taking the average of the five measurements, we calculated the average of the amplitude characteristics in the frequency domain (hereafter referred to as (e-1)), which is the expected characteristic, and the amplitude characteristics of the HpTF obtained by Fourier transforming the average impulse response taken in the time domain (hereafter referred to as (e-2)).

[0066] Additionally, three impulse responses were selected corresponding to the three HpTFs closest to the average HpTF of the HpTFs in cluster A. In this way, in this example, three impulse responses were selected using <<Method (1)>>. The three selected impulse responses were then averaged in the time domain, and the averaged impulse response was subjected to a Fourier transform to determine the amplitude characteristics of the HpTF (hereafter referred to as (e-3)).

[0067] In Figure 6, (e-1) is shown by a solid line, (e-2) by a dashed line, and (e-3) by a dot-dash line. Figure 6 shows that the amplitude characteristic (e-3) obtained from the impulse response selected using <<Method (1)>> is closer to the expected characteristic (e-1) than the amplitude characteristic (e-2) corresponding to the impulse response simply averaged in the time domain. The mean square error between (e-1) and (e-2) is 12.6, while the mean square error between (e-1) and (e-3) is 7.4.

[0068] Furthermore, one impulse response corresponding to the HpTF in cluster A that is closest to the average HpTF obtained in the frequency domain was selected. In this way, in this example, one impulse response was selected using <<Method (2)>>. Then, the amplitude characteristic (hereinafter referred to as (e-4)) corresponding to the selected one impulse response was selected.

[0069] In Figure 7, (e-1) is shown by a solid line, (e-2) by a dashed line, and (e-4) by a dashed line. Figure 7 shows that the amplitude characteristic (e-4) corresponding to the impulse response selected by <<Method (2)>> is closer to the expected characteristic (e-1) than the amplitude characteristic (e-2) corresponding to the impulse response simply averaged in the time domain. The mean square error between (e-1) and (e-2) is 12.6, while the mean square error between (e-1) and (e-4) is 6.3.

Claims

1. A filter information generating device comprising: a plurality of impulse responses corresponding to different conditions from an input of a wearable device that reproduces sound without completely blocking the external ear canal to the entrance of the ear canal of a wearer wearing the wearable device, the plurality of impulse responses being measured in advance; the plurality of impulse responses being clustered into a plurality of clusters; an impulse response determination unit for generating filter information that selects one of the plurality of clusters and determines an impulse response for generating filter information from at least one impulse response included in the selected cluster; and a filter information generation unit that generates, from the impulse response for generating filter information, filter information that is information about a filter for performing filtering on a signal to be reproduced by the wearable device, which is a filter for weakening the acoustic characteristics of the wearable device determined from the impulse response for generating filter information.

2. The filter information generating device of claim 1, wherein k is a predetermined positive integer, each of the plurality of clusters is associated with k impulse responses corresponding to k transfer functions each having k amplitude characteristics close to an average value of the amplitude characteristics of a transfer function corresponding to at least one impulse response included in each of the plurality of clusters, and the filter information generation impulse response determination unit determines an average value of the k impulse responses associated with the selected cluster as the filter information generation impulse response.

3. A filter information generating device according to claim 1, wherein each of said plurality of clusters is associated with a representative transfer function, which is a transfer function having amplitude characteristics closest to an average value of amplitude characteristics of transfer functions corresponding to at least one impulse response included in each of said plurality of clusters, and said filter information generation impulse response determination unit determines an impulse response corresponding to the representative transfer function associated with said selected cluster as said filter information generation impulse response.

4. A filter information generating device according to any one of claims 1 to 3, wherein the plurality of impulse responses are clustered into a plurality of clusters based on values ​​of amplitude characteristics of a transfer function obtained by Fourier transforming the impulse responses.

5. A filter information generating method comprising: a step of measuring in advance a plurality of impulse responses corresponding to a plurality of different conditions from the input of a wearable device that reproduces sound without completely blocking the external ear canal to the entrance of the ear canal of a wearer wearing the wearable device, the plurality of impulse responses being clustered into a plurality of clusters; and a step of generating filter information generating impulse response determining unit selecting one of the plurality of clusters and determining an impulse response for generating filter information from at least one impulse response included in the selected cluster; and a step of generating filter information from the impulse response for generating filter information generated by the filter information generating unit, the filter information being information about a filter for weakening the acoustic characteristics of the wearable device determined from the impulse response for generating filter information, the filter information being information about a filter for performing filtering on a signal to be reproduced by the wearable device.

6. A program for causing a computer to function as each part of the filter information generating device according to any one of claims 1 to 3.

Images