Acoustic device, acoustic method, and acoustic program

Abstract

An acoustic device according to one embodiment of the present disclosure comprises: an acquisition unit that acquires, from a user who is a measurement subject of an open ear canal head-related transfer function, which is a head-related transfer function in a state where the ear canals are open, user information which is information relating to the user; and a calculation unit that calculates, on the basis of the user information, an open ear canal head-related transfer function to be applied to the user by executing an acoustic simulation for obtaining an open ear canal head-related transfer function of the user or correction of a closed ear canal head-related transfer function, which is a head-related transfer function in a state where the ear canals are closed.

Description

Acoustic device, acoustic method, and acoustic program

[0001] The present disclosure relates to an acoustic device, an acoustic method, and an acoustic program related to a process for calculating a head-related transfer function.

[0002] A technology is being used to reproduce a three-dimensional sound image in headphones or the like by using HRTF (Head-Related Transfer Function), which mathematically represents how sound reaches the ears from a sound source.

[0003] For example, a technique is known in which the HRTF of a user is calculated based on an image obtained by photographing the user's ear, and the calculated HRTF is applied to the user.

[0004] International Publication No. 2020 / 075622

[0005] According to the conventional technology, it is possible to easily calculate the HRTF to be applied to the user without the need for laborious measurements or the like.

[0006] It is known that the reproducibility of an acoustic space can be improved by accurately measuring the HRTF of each individual. Therefore, in order to reproduce stereophonic sound with high accuracy, it is desirable to obtain a highly accurate HRTF that corresponds to each individual.

[0007] Therefore, the present disclosure proposes an audio device, an audio method, and an audio program that are capable of calculating HRTFs that can reproduce stereophonic sound with higher accuracy.

[0008] In order to solve the above problems, an acoustic device according to one embodiment of the present disclosure includes an acquisition unit that acquires user information, which is information about a user, from a user who is to be measured for an open-ear ear canal head-related transfer function, which is a head-related transfer function in an open ear canal state, and a calculation unit that calculates the open-ear ear canal head-related transfer function to be applied to the user by performing an acoustic simulation to obtain the open-ear ear canal head-related transfer function of the user based on the user information, or by performing correction of the closed-ear ear canal head-related transfer function, which is a head-related transfer function in a closed ear canal state.

[0009] 1 is a flowchart illustrating an overview of acoustic processing according to an embodiment; FIG. 2 is a block diagram illustrating acoustic processing according to an embodiment; FIG. 3 is a diagram illustrating an example configuration of an acoustic system according to an embodiment; FIG. 4 is a diagram illustrating an example configuration of a measurement system according to an embodiment; FIG. 5 is a diagram illustrating ear canal transfer characteristics of actual measurement data and simulation; FIG. 6 is a diagram illustrating the relationship between the energy reflectance of the eardrum and the ear canal transfer characteristics; FIG. 7 is a diagram illustrating the relationship between the phase of the eardrum reflectance and the ear canal transfer characteristics; FIG. 8 is a diagram illustrating the relationship between the energy reflectance of the eardrum and the resonance level; FIG. 9 is a diagram illustrating the relationship between the phase of the eardrum reflectance and the resonance frequency; FIG. 10 is a flowchart illustrating the procedure of adjustment processing according to an embodiment; FIG. 11 is a diagram illustrating an example of reflectance characteristics obtained from a subject by estimation processing; FIG. 12 is a diagram illustrating an example of phase characteristics obtained from a subject by estimation processing; FIG. 13 is a diagram illustrating ear canal transfer characteristics; FIG. 14 is a diagram illustrating impedance characteristics measured at the entrance of the ear canal using a second method; FIG. 15 is a diagram illustrating impedance characteristics measured at the entrance of the ear canal using a second method; FIG. 16 is a diagram illustrating an example of HRTF reflecting the impedance characteristics measured at the entrance of the ear canal; FIG. 17 is a diagram illustrating an average of PDs calculated from HRTFs obtained by simulation for multiple people; FIG. 18 is a diagram illustrating an example of an HRTF using a correction filter. 1 is a diagram showing an example of the relationship between PD and HpTF, and FIG. 2 is a diagram showing an example of characteristics obtained by correcting HpTF with a difference in resonance characteristics of HRTF, and FIG. 3 is a hardware configuration diagram showing an example of a computer that realizes the functions of an audio device.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0011] The present disclosure will be described in the following order: 1. Embodiments 1-1. Overview of acoustic processing according to embodiments 1-2. First method according to embodiments 1-3. Second method according to embodiments 1-4. Third method according to embodiments 1-5. Fourth method according to embodiments 1-6. Configuration of acoustic device according to embodiments 1-7. Modified examples according to embodiments 2. Other embodiments 3. Effects of acoustic device according to the present disclosure 4. Hardware configuration

[0012] (1. Embodiment) (1-1. Overview of Acoustic Processing According to Embodiment) First, an overview of acoustic processing according to an embodiment will be described using FIG. 1. FIG. 1 is a flowchart showing an overview of acoustic processing according to an embodiment. The acoustic processing according to the embodiment is executed by an acoustic device 100 not shown in FIG. 1. The acoustic device 100 is, for example, an information processing terminal such as a server, a PC (Personal Computer), a tablet terminal, or a smartphone.

[0013] The acoustic device 100 calculates a head-related transfer function (HRTF), which indicates characteristics that describe the effects of reflection, diffraction, and other factors caused by a person's torso, head, and pinna when sound waves propagate from a sound source to the ear. HRTFs contain information used by people to perceive the direction of sound, and are therefore widely used in the field of stereophonic technology. HRTFs vary significantly among individuals due to factors such as head dimensions and pinna shape, and therefore, in order to reproduce a realistic experience when reproducing audio signals, it is necessary to use an HRTF optimized for each individual.

[0014] Methods for obtaining an HRTF optimized for an individual include a method of actually measuring sound characteristics by having a user who is the subject of measurement wear a microphone, a method of calculating the HRTF by acoustic numerical simulation, etc. In actual measurements, audible sound waves can be considered to propagate one-dimensionally within the ear canal, so even if the measurement point for measuring the HRTF is the entrance of the ear canal, it is said that sufficient information about the direction of the sound source is included.

[0015] When a user listens to HRTF-processed sound through headphones, it is necessary to correct for the acoustic characteristics of the headphones themselves, the effects of the headphone housing when the headphones are worn on the ear, the user's pinna, and the ear canal. Specifically, the transfer characteristics from the headphones to the ear, i.e., the HpTF (Headphone Transfer Function), can be measured and the playback signal from the headphones can be corrected by processing the inverse characteristics in advance. In other words, if the measurement points for the HRTF and HpTF match at any position from the entrance of the ear canal to the eardrum, corrective action can be taken during playback, allowing sound pressure characteristics equivalent to the sound field intended by the manufacturer to be reproduced all the way to the eardrum.

[0016] Incidentally, when measuring HRTFs and the like, in order to reduce individual variations due to the influence of the ear canal, a microphone is generally placed on the user's auricle with the ear canal entrance blocked (hereinafter referred to as the "occluded state"). However, when measurements are made in an occluded state, the acoustic impedance at the entrance of the ear canal changes due to the occlusion of the ear canal, resulting in a difference in sound pressure between the HRTFs and HpTFs obtained in an occluded state and those obtained in an open state. For example, if the acoustic impedance of the boundary as seen from the entrance of the ear canal changes significantly depending on the headphones worn by the user during listening, even if the HRTFs obtained in the occluded state are corrected for headphone characteristics measured under the same conditions and positions, an error will occur in the sound pressure transmitted to the eardrum.

[0017] Therefore, considering that individual users use a variety of headphones, in order to reproduce stereophonic technology with higher accuracy, it is desirable to obtain HRTFs (hereinafter referred to as "open HRTFs") in a state where the ear canal is not blocked (hereinafter referred to as "open state").

[0018] On the other hand, when HRTF is calculated by numerical simulation, it is usually performed under the condition that the entrance of the ear canal is blocked and the measurement point is the seal position at the bottom of the cavity of the concha.

[0019] For these reasons, one of the challenges in acoustic simulations involving open HRTFs is that much is unknown about the acoustic characteristics of the middle ear, including the eardrum, making it difficult to accurately reflect the acoustic conditions when the ear canal is open during simulation. For example, in acoustic simulations in an occluded state, a 3D shape model (hereinafter referred to as a "3D model") used in the acoustic simulation is often assumed to be a rigid body, with the surface reflectance set to 100%. Since the reflectance of skin is said to be close to 100%, the rigid body assumption is not a major problem when simulating an occluded ear canal. However, in the case of an open ear canal, it is necessary to additionally consider the influence of the ear canal wall and the area beyond the eardrum. Because the ear canal wall is mainly composed of skin and cartilage, it may be reasonable to consider the surface as a rigid body. However, when sound is transmitted to the eardrum and then propagated to the middle ear via the vibration of the eardrum, the assumption of a rigid body deviates from this.

[0020] In order to measure the reflectivity of the eardrum and ear canal, previous studies have employed a method in which a probe microphone and earphones are placed in the ear canal to measure the sound pressure response, and then the ear canal impedance is calculated by calibrating with a tube of known impedance. However, since the impedance measured using this method is the impedance at the probe tip, it is difficult to obtain the impedance at the eardrum because the length and shape of the ear canal to the eardrum are unknown. Furthermore, although high-precision 3D model data of an individual's head for simulation can be obtained using a 3D scanner, this requires specialized equipment and skills.

[0021] On the other hand, to enable users to easily obtain HRTFs optimized for themselves, services have been developed that acquire anthropometric data, such as the shape of the head and auricle, from images of the user's head and ears taken with a smartphone or the like, and then use machine learning or simulation to provide HRTFs optimized for the user. However, it is difficult to acquire data related to the user's ear canal using methods that involve taking images from the outside. For this reason, the personal HRTFs calculated using such methods do not include the characteristics of the ear canal, i.e., they are limited to HRTFs when the ear canal is occluded.

[0022] As described above, calculating open HRTFs is ideal for reproducing stereophonic sound with high accuracy, and considering the recent widespread use of in-ear headphones, it is desirable to be able to apply open HRTFs to individuals. To achieve this, a technology is required that adds information to individual occlusion HRTFs obtained from, for example, photographs, videos, or body measurement data so that the ear canal is in an open state.

[0023] The acoustic device 100 implementing the technology of the present disclosure solves the above-described problems by the following configuration. Specifically, the acoustic device 100 acquires user information, which is information about the user, from the user whose open HRTFs are to be measured. Based on the user information, the acoustic device 100 then performs an acoustic simulation to obtain the open HRTFs of the user, or performs correction of the occluded HRTFs, to calculate the open HRTFs to be applied to the user. This allows the acoustic device 100 to provide HRTFs that can reproduce stereophonic sound with higher accuracy, i.e., personalized open HRTFs to be applied to the user.

[0024] Acoustic device 100 calculates the open HRTF using four methods according to the user information that can be acquired, following the procedure shown in Fig. 1. In this way, although user environments (such as whether ear canal 3D data can be acquired) vary, acoustic device 100 can flexibly select a method to calculate the open HRTF according to the information that the user can prepare, thereby providing an open HRTF that is optimized for the individual user, regardless of the environment.

[0025] First, the acoustic device 100 determines whether the user can acquire 3D data of the eardrum and ear canal (step S10). Specifically, the acoustic device 100 determines whether 3D data of the inside of the user's ear canal, obtained using a magnetic resonance imaging (MRI) device or the like, can be acquired. If acquisition is possible (step S10; Yes), the acoustic device 100 estimates the impedance of the eardrum portion of the acquired 3D data, reflects the estimated impedance in the user's 3D data, and then calculates the open-circuit HRTF (step S11). This method is the first method.

[0026] On the other hand, if 3D data cannot be acquired from the user (step S10; No), the acoustic device 100 determines whether ear canal impedance can be measured by, for example, inserting and placing a microphone inside the user's ear canal (step S20). If ear canal impedance can be measured (step S20; Yes), the acoustic device 100 reflects the ear canal impedance obtained by measurement in a 3D model that simulates the ear canal. Then, the acoustic device 100 calculates open HRTFs to be applied to the user based on the 3D model reflecting the ear canal impedance (step S21). This method is the second method. In this way, the first and second methods are methods of calculating open HRTFs based on acoustic simulation.

[0027] On the other hand, if the user's ear canal impedance cannot be measured (step S20; No), the acoustic device 100 determines whether the user's personal HpTF (headphone characteristics) can be acquired (step S30). If the personal HpTF can be acquired (step S30; Yes), the acoustic device 100 calculates the open HRTF by correcting the user's closed HRTF based on the personal ear canal characteristics obtained from the personal HpTF (step S31). This method is the third method.

[0028] On the other hand, if the personal HpTF cannot be obtained (step S30; No), the acoustic device 100 calculates the open-end HRTF by correcting the occlusion HRTF using general-purpose ear canal characteristics (for example, average ear canal characteristics obtained from an unspecified number of users) (step S40). This method is the fourth method. In this way, the third and fourth methods are methods for calculating the open-end HRTF based on a correction process for the occlusion HRTF.

[0029] Next, the four techniques will be described in detail with reference to Fig. 2. Fig. 2 is a block diagram for explaining acoustic processing according to the embodiment.

[0030] (1-2. First Method According to the Embodiment) First, the first method (step 11 in FIGS. 1 and 2) will be described. The effect of sound absorption on the propagation of sound to the eardrum can be reflected by setting impedance characteristics in the eardrum portion on the 3D simulation model. However, it is difficult to directly measure the impedance of the eardrum portion in a living body (hereinafter referred to as "eardrum impedance"). In other words, since it is difficult to obtain the sound absorption characteristics of the eardrum with conventional methods, it is necessary to set the eardrum impedance as accurately as possible in order to reflect the sound absorption characteristics in the acoustic simulation.

[0031] Therefore, the acoustic device 100 estimates the tympanic membrane impedance using the following methods: In a first method, the acoustic device 100 estimates the tympanic membrane impedance by an approach that determines the optimal tympanic membrane impedance so that the ear canal transfer characteristics (transfer characteristics from the entrance of the ear canal to the eardrum) match as closely as possible between the simulation results and actual measurements by the user.

[0032] Transmission within the ear canal can be considered one-dimensional, and since it is constant and independent of the direction of the sound source, noise due to deviations in the sound source position, the influence of structures other than the ear canal, and differences in acoustic characteristics between simulations and actual measurements is considered to be relatively small. Furthermore, the ear canal has characteristics similar to canal resonance. Therefore, the ear canal transfer characteristics have a resonance peak at a specific frequency. Because these resonance characteristics vary significantly depending on the impedance of the closed end (i.e., the eardrum), eardrum impedance can be appropriately estimated by determining the eardrum impedance so that the eardrum transfer characteristics match the actual measurements.

[0033] First, the acoustic device 100 obtains actual measurement data of the user's ear canal transfer characteristics (step S12). The actual measurement data of the ear canal transfer characteristics is calculated, for example, by measuring the difference between frequency characteristics captured by microphones placed at the entrance of the ear canal and the eardrum. The actual measurement data is obtained, for example, by measuring the ear canal transfer characteristics using measuring equipment such as a dedicated microphone and speaker in a dedicated facility and processing the data using an information processing device. Note that the actual measurement data may be, for example, the ear canal transfer characteristics calculated by playing back a measurement sound from a user's device (such as a smartphone), recording signals captured by microphones placed at the entrance of the ear canal and the eardrum on the device, transmitting the recorded signals to a cloud server, etc., and processing the recorded signals on the cloud server.

[0034] The acoustic device 100 also performs an acoustic simulation of the ear canal transfer characteristics (step S13). The ear canal transfer characteristics based on the acoustic simulation are calculated, for example, from the difference between the HRTF obtained by the simulation, in which the entrance of the ear canal is used as the sound receiving point, and the HRTF in which the eardrum is used as the sound receiving point. The simulation may be performed on a PC or the like, or may be performed by sending 3D data of the eardrum shape from the user's terminal to a cloud server and combining it with head and ear data on the cloud.

[0035] As described above, the ear canal transfer characteristics are considered to be constant regardless of the direction of the sound source at frequencies below 10 kHz. Therefore, the acoustic device 100 can reduce noise due to measurement errors, etc. by averaging the levels of the ear canal transfer characteristics calculated from the HRTFs of sound source positions in multiple directions.

[0036] The acoustic device 100 adjusts the eardrum impedance applied to the user based on the actual measurement data and the data obtained by the simulation (step S14). Here, Fig. 4 shows an example of data obtained when a simulation was performed assuming the eardrum to be a rigid body. Fig. 4 shows the ear canal transfer characteristics of the actual measurement data and the simulation.

[0037] In graph 300 shown in Figure 4, the frequency characteristics indicated by the solid line are the results obtained by simulation. The frequency characteristics indicated by the dotted line are actual measurement data. As shown in graph 300, when a simulation is performed assuming the eardrum to be a rigid body, the resonance peak level of the simulation is higher than the actual measurement data, and the resonance frequency also differs from the actual measurement. This graph 300 was obtained by analyzing the changes in resonance peak level and frequency due to the eardrum's sound absorption characteristics in order to determine eardrum sound absorption characteristics that would match the resonance characteristics in the simulation with the actual measurement data.

[0038] The acoustic device 100 adjusts the characteristics of the eardrum (e.g., the energy reflectance of the eardrum and the phase of the reflection) so that the simulation matches the actual measurement data. Figures 5 and 6 show the relationship between the energy reflectance and the phase characteristics. Figure 5 is a diagram showing the relationship between the energy reflectance of the eardrum and the ear canal transfer characteristics. Figure 6 is a diagram showing the relationship between the phase of the eardrum reflectance and the ear canal transfer characteristics.

[0039] Graph 310 shown in Fig. 5 is a diagram illustrating the relationship between the energy reflectance of the eardrum and the ear canal transfer characteristics. "R" in graph 310 represents reflectance. That is, graph 310 in Fig. 5 shows that as the energy reflectance increases, the resonance level increases, and as the energy reflectance decreases, the resonance level also decreases.

[0040] 6 is a graph showing the relationship between the phase of the eardrum reflectance and the ear canal transfer characteristics. The graph 320 shows that the resonant frequency in the ear canal transfer characteristics changes in response to changes in the phase characteristics.

[0041] The acoustic device 100 estimates the characteristics of the eardrum portion based on these relationships. For example, the acoustic device 100 calculates the frequency and level of each resonance peak from the local maximum value of the ear canal transfer characteristics, and determines the relationship between the energy reflectance and the resonance level, and the relationship between the phase of the reflectance and the resonance frequency. These relationships are shown in Figures 7 and 8. Figure 7 is a diagram showing the relationship between the energy reflectance of the eardrum and the resonance level. Figure 8 is a diagram showing the relationship between the phase of the reflectance of the eardrum and the resonance frequency.

[0042] Graph 330 shown in FIG. 7 indicates that the resonance level increases as the energy reflectance increases, and that the lower the frequency of the resonance peak, the more strongly it is affected by the energy reflectance.

[0043] Graph 340 shown in FIG. 8 shows that the value of each resonant frequency is uniformly affected by changes in the phase of the eardrum reflectance.

[0044] As described above, the acoustic device 100 can estimate optimal characteristics of the eardrum portion by adjusting the energy reflectance and phase characteristics of the eardrum portion. The procedure for such adjustment processing will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the procedure for adjustment processing according to the embodiment.

[0045] First, the acoustic device 100 arbitrarily sets initial values ​​of the reflectance and phase to be set for the eardrum portion (step S101). For example, the acoustic device 100 sets the initial values ​​to "0" for the phase and "80%" for the reflectance.

[0046] Next, the acoustic device 100 executes an acoustic simulation including outputting and measuring a test sound (step S102).The acoustic device 100 then compares the resonance peak frequency obtained by the simulation with the value of the actual measurement data, and determines whether there is a discrepancy (step S103).

[0047] If there is a deviation (step S103; Yes), the acoustic device 100 adjusts the phase characteristics at the frequency where the deviation occurs (step S104). For example, if the resonance peak frequency in the simulation is higher than the actual measurement value, the acoustic device 100 adjusts the phase characteristics at that frequency to be smaller. Also, if the resonance peak frequency in the simulation is lower than the actual measurement value, the acoustic device 100 adjusts the phase characteristics at that frequency to be larger.

[0048] If the deviation is eliminated after repeating this adjustment (step S103; No), the acoustic device 100 determines the phase characteristics of the frequency after the adjustment (step S105). Using the determined values, the acoustic device 100 further performs an acoustic simulation (step S106).

[0049] The acoustic device 100 compares each resonance peak level obtained by the simulation with the value of the actual measurement data, and determines whether or not there is a discrepancy (step S107).

[0050] If there is a deviation (step S107; Yes), the acoustic device 100 adjusts the reflectance characteristics at the frequency where the deviation occurred (step S108). For example, if the resonance peak level in the simulation is higher than the actual measurement value, the acoustic device 100 adjusts the reflectance characteristics at that frequency to decrease. Also, if the resonance peak level in the simulation is lower than the actual measurement value, the acoustic device 100 adjusts the reflectance characteristics at that frequency to increase.

[0051] If the deviation disappears after repeating this adjustment (step S107; No), the acoustic device 100 determines the reflectance at the frequency after adjustment as the reflectance absolute value characteristic (step S109).In this way, the acoustic device 100 obtains the reflectance at the eardrum (step S110).

[0052] However, the simulation for performing the above adjustments may require a large number of trials, potentially requiring an enormous amount of calculation time. Therefore, the acoustic device 100 may employ a method for reducing the number of trials. For example, as shown in FIGS. 7 and 8 , the resonant frequency and peak level change monotonically as the reflectance of the eardrum increases or decreases. Therefore, instead of the adjustment process shown in FIG. 9 , the acoustic device 100 can calculate the energy reflectance and phase from the actually measured peak level and frequency using, for example, an interpolation method. This allows the acoustic device 100 to more efficiently determine the optimal characteristics of the eardrum.

[0053] Furthermore, acoustic device 100 may calculate the reflectance characteristics for the entire frequency band by further using interpolation, using the energy reflectance and phase at each frequency determined by optimization based on the frequency and level of each resonance peak of the measured data (the adjustment process shown in FIG. 9 ) or the interpolation method described above. It is empirically known that in the low frequency range and in the frequency band above 20 kHz, the eardrum reflectance is close to 100%, and the phase in the low frequency range is close to 0. Therefore, acoustic device 100 may perform interpolation after setting the reflectance at 0 Hz and 24,000 Hz to "1" and the phase at 0 Hz to "0."

[0054] The characteristics obtained by this method will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a diagram showing an example of reflectance characteristics obtained from a subject by estimation processing. Fig. 11 is a diagram showing an example of phase characteristics obtained from a subject by estimation processing.

[0055] Graph 350 in Fig. 10 shows an example of the relationship between eardrum reflectance and frequency. The "x" symbols in graph 350 indicate eardrum reflectance values ​​calculated based on actual resonance characteristics. The solid line in graph 350 indicates the reflectance characteristics for the entire frequency band calculated, for example, by interpolation.

[0056] Graph 370 in Fig. 11 shows an example of the relationship between the eardrum portion, phase characteristics, and frequency. The "x" in graph 370 indicates a phase characteristic value calculated based on the actually measured resonance characteristics. The solid line in graph 370 indicates the phase characteristic for the entire frequency band calculated, for example, by interpolation.

[0057] The characteristics of the eardrum portion obtained in this manner will be described with reference to FIG. 12 . FIG. 12 is a diagram showing ear canal transfer characteristics. The solid line in graph 380 of FIG. 12 indicates the ear canal transfer characteristics obtained when an acoustic simulation is performed after setting the characteristics (impedance) of the eardrum portion. Furthermore, the thick dotted line in graph 380 of FIG. 12 indicates the ear canal transfer characteristics based on actual measurement data. As shown in graph 380, the ear canal transfer characteristics obtained by applying the estimated eardrum impedance are consistent with the actual measurement data.

[0058] Continuing the explanation, returning to Figure 2, the acoustic device 100 applies the tympanic membrane impedance obtained by the above processing to the 3D model 10 including the head, pinna, and tympanic membrane. That is, the acoustic device 100 sets the tympanic membrane impedance estimated to match the ear canal resonance of the actual measurement data as the estimated value of the tympanic membrane impedance of the user, and as the sound absorption characteristics of the tympanic membrane portion.

[0059] The acoustic device 100 then performs an acoustic simulation at the measurement point at the entrance of the ear canal in the 3D model 10 (step S25). After HRTF post-processing (step S26), the acoustic device 100 calculates the open HRTF 50 applied to the user.

[0060] The optimization of eardrum impedance and acoustic simulation described above may be performed on the cloud. Furthermore, the acoustic device 100 may perform multiple simulations using different eardrum impedance values ​​and different ear canal shapes to generate training data showing the relationship between them. This allows the acoustic device 100 to construct a machine learning model that inputs measured ear canal transfer characteristics and parameters describing the shape of the ear canal, such as the length of the ear canal, and outputs the eardrum impedance. The acoustic device 100 can then estimate the eardrum impedance using the training model.

[0061] (1-3. Second Method According to the Embodiment) Next, the second method (step S21 in FIGS. 1 and 2) will be described. The second method is a method that uses ear canal impedance.

[0062] Conventionally, there are known methods for acoustically measuring impedance in the ear canal. In these methods, a sound source that emits a measurement signal and a microphone that measures sound pressure characteristics are placed in the ear canal. Specifically, in these methods, earplugs with earphones and a sound guide tube for the microphone fixed thereto are inserted into the user's ear canal. Alternatively, in these methods, the measurement is performed while the user wears in-ear headphones equipped with a microphone. Then, in these methods, the ear canal impedance is obtained based on the output characteristics of the measured ear canal sound pressure characteristics and the impedance of the measurement device obtained in advance by calibration.

[0063] However, with this method, the impedance is measured at the position of the installed microphone, which is usually near the entrance of the ear canal and away from the eardrum, so the measured value cannot be directly reflected in the eardrum part of the simulation model.

[0064] To overcome this drawback, the acoustic device 100 according to the present disclosure employs a method for simulating the open-circuit HRTF using the impedance measured at the entrance of the ear canal. That is, the acoustic device 100 creates a virtual sound-absorbing surface at the measurement point (e.g., the entrance of the ear canal) and sets the measured impedance as the boundary condition of that surface, thereby simulating a state in which the ear canal and eardrum are actually present behind the virtual sound-absorbing surface.

[0065] For example, in this method, to create a 3D model for simulation, a user estimates the position of the tip of the probe microphone tube within the ear canal and then creates a 3D model cut at that position along a plane perpendicular to the ear canal. This method reduces the burden on the user because even if 3D data up to the eardrum cannot be obtained, it is sufficient to obtain open-circuit HRTFs as long as the shape around the entrance of the ear canal can be acquired. The shape around the entrance of the ear canal can be acquired using a dedicated 3D scanner or based on depth data obtained by a depth sensor-equipped camera on the user's smartphone.

[0066] For example, the acoustic device 100 acquires the measurement results (sound pressure, etc.) obtained by measuring a measurement signal output by a user from a smartphone using earphones attached to the user's ear canal (step S22). Furthermore, the acoustic device 100 calculates the impedance at the entrance of the ear canal based on the obtained measurement results (step S23). The acoustic device 100 obtains a 3D model 20 in which the impedance at the entrance of the ear canal is set. Subsequently, the acoustic device 100 performs acoustic simulation and HRTF post-processing using the 3D model 20, thereby calculating the open HRTF 50.

[0067] Here, Fig. 13 and Fig. 14 show impedance characteristics measured at the entrance of the ear canal using the second method. Fig. 13 is a diagram (1) showing impedance characteristics measured at the entrance of the ear canal using the second method. Fig. 14 is a diagram (2) showing impedance characteristics measured at the entrance of the ear canal using the second method. Graph 390 shown in Fig. 13 and graph 400 shown in Fig. 14 show the relationship between impedance and frequency. Fig. 15 is a diagram showing an example of an HRTF that reflects the impedance characteristics measured at the entrance of the ear canal. Graph 410 shown in Fig. 15 shows an example of an open HRTF obtained by acoustic simulation that reflects the impedance characteristics measured at the entrance of the ear canal.

[0068] As an alternative to the above-described simulation, the acoustic device 100 can also perform calculations using a machine learning model. For example, the acoustic device 100 acquires, as training data, a large number of open HRTFs obtained by simulation using the above-described method, as well as images (depth data) obtained by a depth sensor and signals measured by a microphone. The acoustic device 100 then uses the training data to train a model that receives the images and measurement signals as input and outputs open HRTFs. This allows the acoustic device 100 to obtain a machine learning model that can estimate open HRTFs.

[0069] In the second method, a user may also send depth data and audio signals acquired by a smartphone to the cloud and have the above processing executed on the cloud. By downloading the open HRTF generated on the cloud, the user can easily obtain the open HRTF optimized for the user.

[0070] Furthermore, the acoustic device 100 may calculate, by simulation, average HRTFs for use in the third and fourth methods described below. That is, in services that provide HRTFs optimized for individuals based on photographs, videos, body measurement data, and the like, as described above, ear canal information for the user cannot be obtained from such data. Therefore, it is necessary to first personalize the closed HRTFs, which do not include ear canal information, and then add information for correcting them to open HRTFs. The acoustic device 100 obtains the information used in these correction processes by simulation.

[0071] For example, to determine the average ear canal characteristics, the acoustic device 100 acquires an average ear canal shape 30 and an average eardrum impedance 31. For example, the average ear canal shape 30 is acquired by calculating the spatial average from ear canal shape data of multiple people.

[0072] Next, the acoustic device 100 synthesizes the average ear canal shape with the individual's head and pinna data (step S27). Furthermore, the acoustic device 100 performs an acoustic simulation using this data (step S28), and after HRTF post-processing (step S29), obtains an HRTF containing information about the average ear canal (referred to as "HRTF with average ear canal 32"). Note that in this simulation, the sound absorption characteristics used at the eardrum position can be the average eardrum impedance calculated from the average energy reflectance and the average reflectance phase, as in the above-mentioned method.

[0073] In this way, the acoustic device 100 can store in the database 41 the average ear canal HRTF 32 obtained from a large amount of data collected in advance. The acoustic device 100 also stores in the database 41 the average occlusion HRTF 40 obtained in advance from a simulation of a case in which the ear canal is blocked. By averaging these data differentially and performing calculations using a learning model (step S41), the acoustic device 100 can obtain a correction filter 42 for correcting the occlusion HRTF to the open HRTF. This allows the acoustic device 100 to easily calculate the open HRTF without incurring the cost of running a large number of simulation trials for each user.

[0074] In the averaged data, the head and auricle parts other than the ear canal are completely the same, so the difference in characteristics can be considered to be almost entirely due to the influence of the average ear canal. In this regard, the difference between the open HRTF and the closed HRTF is expressed as the difference in sound pressure characteristics between the open and closed ear canals (PD, Pressure Division). As mentioned above, the ear canal characteristics are independent of direction, so the PD characteristics calculated from the HRTFs of different sound source directions are considered to be constant up to approximately 16 kHz.

[0075] Therefore, the acoustic device 100 can calculate the correction filter characteristics of the average ear canal characteristics by first calculating the average PD for multiple sound source directions for each subject and then averaging the PD characteristics for multiple people. FIG. 16 is a diagram showing the average PD calculated from the HRTFs simulated for multiple people. The thick dotted line in graph 420 of FIG. 16 indicates the average PD at each frequency. The acoustic device 100 uses this average characteristic as the correction filter 42 shown in FIG. 2. FIG. 17 is a diagram showing an example of an HRTF using the correction filter 42. As shown in graph 430 of FIG. 17, the difference between the simulated average ear canal HRTF (solid line) and the HRTF (dotted line) obtained by processing the simulated occlusion HRTF with the correction filter 42 is extremely small.

[0076] The acoustic device 100 may perform machine learning using the open HRTFs and occlusion HRTFs with average ear canals stored in the database 41 as training data, and estimate a correction filter for correcting the occlusion HRTFs to the open HRTFs. Alternatively, the acoustic device 100 may perform training to directly derive the open HRTFs from the occlusion HRTFs, and calculate the open HRTFs using the trained model.

[0077] (1-4. Third Method According to the Embodiment) Next, a third method (step S31 in FIGS. 1 and 2) will be described. In the third method, the acoustic device 100 calculates the open-circuit HRTF using the user's personal HpTF (step S31 in FIG. 2).

[0078] For example, if the acoustic device 100 can obtain the HpTF of the individual user with the ear canal in an open state by measurement (step S32), the open HRTF can be personalized by the ear canal correction process described above.

[0079] First, the acoustic device 100 extracts ear canal resonance characteristics using the HpTF obtained by actual measurement (step S33). Here, FIG. 18 shows the ear canal resonance characteristics of the HpTF. FIG. 18 is a diagram illustrating an example of the relationship between the PD and the HpTF. As shown in graph 440 in FIG. 18, the PD and the HpTF have similar peaks and depressions (peak notch characteristics) in the relationship between frequency and level. The peak notch characteristics that match between the HpTF and the PD (e.g., notches around 3-4 kHz and 8 kHz in the example of FIG. 18) are thought to be due to ear canal resonance. Therefore, the acoustic device 100 can estimate the resonance characteristics of the HpTF by detecting the PD in the HpTF.

[0080] Although the frequency of each resonance notch varies slightly due to the impedance phase characteristics of the eardrum, the ear canal characteristics are similar to the resonance of the tube, and therefore they are roughly odd-numbered. Furthermore, the characteristics of the HpTF due to the influence of the pinna typically occur in the frequency band of 5 kHz or higher. Therefore, notches below 5 kHz in the HpTF can be considered to be due to the influence of the ear canal. Therefore, the acoustic device 100 can detect the notch with the lowest frequency as the first resonance, and the notches closest to frequencies 3, 5, and 7 times the notch frequency as the first resonance.

[0081] Using the detected notch frequency and gain (level) parameters, the acoustic device 100 can obtain a personalized ear canal correction filter 35 using several methods (step S35).

[0082] For example, the acoustic device 100 can obtain an ear canal correction filter that matches the resonance and anti-resonance frequencies of the individual ear canal obtained from the measured HpTF by fitting the average ear canal correction filter with a PEQ (Parametric Equalizer) or the like and adjusting the frequency, notch frequency, and gain of the PEQ.

[0083] Alternatively, acoustic device 100 may select the PDs of multiple individuals obtained from the simulation described above that have the closest resonant frequency and gain (step S34). Alternatively, acoustic device 100 may employ a method in which, for example, labeled training data of measured HpTFs and PDs is prepared, and a machine-learned model is constructed to predict individual PDs from the measured HpTFs.

[0084] The measurement of the HpTF and the personalization of the PD compensation filter can be performed by the user themselves without special equipment, for example, by playing back a measurement signal from a smartphone owned by the user, converting it into an acoustic signal using headphones worn by the user, and recording the signal using a microphone attached to the headphones. In this method, the smartphone owned by the user may calculate the HpTF by signal processing and generate a compensation filter for personalization from the calculated HpTF. Alternatively, the user may send a signal to the cloud and obtain the compensation filter processed and generated on the cloud.

[0085] The acoustic device 100 can obtain the open HRTF 50 by applying the correction filter 35 to the ear canal occlusion HRTF 45. When the acoustic device 100 convolves the open HRTF obtained by the correction process with a sound source to reproduce a binaural signal, it is necessary to correct the HpTF when the user listens to the sound source. When using the above-mentioned personal PD correction filter, the acoustic device 100 can correct the HpTF by generating an inverse filter using the same actually measured HpTF.

[0086] (1-5. Fourth Method According to the Embodiment) Next, the fourth method (step S40 in FIGS. 1 and 2) will be described.

[0087] It can be assumed that the difference between the resonance characteristics of the average ear canal and the individual ear canal contained in the HRTF is equal to the difference between the resonance characteristics of the average ear canal and the individual ear canal contained in the HpTF. Thus, if the individual HpTF cannot be obtained and the individual ear canal resonance characteristics are unknown, the acoustic device 100 can attempt correction (hereinafter referred to as "HP correction") using a representative HpTF that contains the average ear canal characteristics (step S40 in FIG. 2).

[0088] First, to obtain a representative HP correction, the acoustic device 100 averages the measured HP corrections of multiple subjects. Because the measured HP corrections include the ear canal resonances of each subject, the acoustic device 100 must correct them to obtain a representative ear canal resonance characteristic. For example, the acoustic device 100 adds a resonance characteristic difference to align the individual ear canal resonance characteristics included in the HpTF with the resonance characteristics of the average ear canal. The resonance difference can be calculated from the difference between the HRTF obtained by the individual ear canal simulation and the HRTF obtained by the simulation using the average ear canal, as described above. In other words, the difference between the HRTF obtained by the individual ear canal simulation and the average ear canal HRTF is only the ear canal in the 3D model, and therefore the difference between these HRTFs can be considered to be primarily the difference in resonance due to the ear canal.

[0089] Fig. 19 shows an example of the characteristics of the HpTF corrected by the difference in the resonance characteristics of the HRTFs described above. Fig. 19 is a diagram showing an example of the characteristics of the HpTF corrected by the difference in the resonance characteristics of the HRTFs. As shown in graph 450 in Fig. 19, the HpTF corrected by the difference in the resonance characteristics has a shift in the notch frequency around 8 to 9 kHz, and a characteristic is obtained that matches the ear canal resonance characteristics included in the PD of the average ear canal.

[0090] The acoustic device 100 can average the HpTFs with average ear canals obtained by the above method for multiple people and create a representative HP correction characteristic from the average. For example, when using the average ear canal open HRTF obtained using the correction filter 42 shown in Figure 2, the acoustic device 100 uses the representative HP correction characteristic. This method allows the user to experience sound source playback using the open HRTF without measuring the individual HpTF.

[0091] (1-6. Configuration of the Acoustic Device According to the Embodiment) Next, the configuration of the acoustic system 1 including the acoustic device 100 according to the embodiment will be described with reference to Fig. 3A. Fig. 3A is a diagram showing an example of the configuration of the acoustic system 1 according to the embodiment.

[0092] As shown in FIG. 3A , the audio system 1 includes an audio device 100 , a user terminal 200 , and headphones 250 .

[0093] The user terminal 200 is an information processing terminal owned by a user, such as a smartphone. The user terminal 200 can perform processes such as outputting a measurement signal, recording the output signal, and transmitting the recorded signal to the acoustic device 100. The user terminal 200 may also have various imaging functions such as a camera or a depth sensor. This allows the user to capture an image of their ear by photographing their ear, and to obtain depth data such as the shape of the entrance to the ear canal.

[0094] The headphones 250 are an audio output device used by a user. The headphones 250 may include a microphone for noise cancellation in addition to an audio output unit. If the headphones 250 have an information processing function, the headphones 250 may perform standalone processing such as outputting and inputting measurement signals, similar to the user terminal 200 described above.

[0095] The acoustic device 100 is an information processing device that executes acoustic processing according to the embodiment. In the embodiment, the acoustic device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The acoustic device 100 may also include an input unit (e.g., a touch panel) that accepts various operations from a user operating the acoustic device 100, and a display unit (e.g., a liquid crystal display) that displays various information.

[0096] The communication unit 110 is realized by, for example, a network interface card (NIC) or a network interface controller. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from external devices such as the user terminal 200 and headphones 250 via the network N. The network N is realized by, for example, a wireless communication standard or method such as Bluetooth (registered trademark), the Internet, Wi-Fi (registered trademark), UWB (Ultra Wide Band), or LPWA (Low Power Wide Area).

[0097] The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0098] The storage unit 120 stores various types of information related to acoustic processing, such as data related to the HRTF shown in FIG. 2 , data related to the 3D model, and various types of data such as filters used in correction processing.

[0099] The control unit 130 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing a program (for example, an audio program according to the present disclosure) stored inside the audio device 100 using a RAM or the like as a work area. The control unit 130 is a controller, and may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0100] 3A, the control unit 130 has an acquisition unit 131, a calculation unit 132, an output unit 133, and a measurement unit 134, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3A, and may be any other configuration as long as it performs the information processing described below.

[0101] The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires user information, which is information about a user, from the user who is the subject of open HRTF measurement.

[0102] For example, in a first technique, the acquisition unit 131 acquires actual measurement data relating to the shape of the user's ear canal, such as 3D model data including the user's head and the inside of the ear canal measured by MRI or the like.

[0103] The acquisition unit 131 also acquires actual measurement data of the ear canal transfer characteristics of the user based on the difference between the frequency characteristics of sound measured by a microphone placed at the entrance of the ear canal and a microphone placed at the eardrum. When the user performs the actual measurement, for example, the user obtains the actual measurement data by inserting a spring microphone or a probe microphone into the ear canal.

[0104] Furthermore, when employing a method for estimating eardrum impedance using a learning model, the acquisition unit 131 acquires multiple pieces of actual measurement data related to the shape of the ear canal and the impedance of the eardrum portion applied to multiple ear canals. By using this as learning data, the subsequent calculation unit 132 can generate a learning model that receives as input parameters describing the ear canal transfer characteristics and the shape of the ear canal, such as the length of an arbitrary user's ear canal, and outputs the eardrum impedance.

[0105] Furthermore, the acquiring unit 131 may acquire actual measurement data of the ear canal transfer characteristics measured by the user from the user terminal 200 via communication using the measurement signal output from the user terminal 200. That is, the data acquired by the acquiring unit 131 may be data measured and acquired by an external device such as the user terminal 200 or headphones 250, instead of by the measuring unit 134 described later.

[0106] In the second technique, the acquisition unit 131 acquires a 3D model of the shape of the entrance of the user's ear canal as user information. That is, in the second technique, the acoustic device 100 can obtain the open HRTF of the user only using easily obtainable data, rather than using data that is difficult for the user to obtain, such as 3D data of the inside of the ear canal or the eardrum. For example, the acquisition unit 131 may acquire shape data up to the measurement point (the installation position of the probe microphone described above) when measuring the ear canal impedance of the user.

[0107] The acquisition unit 131 may also acquire open HRTFs obtained by acoustic simulation using a general-purpose 3D ear canal model. The open HRTFs obtained based on such simulations can be used for various purposes, such as generating correction filters, which will be described later.

[0108] When acquiring the shape of the user's ear canal, the acquisition unit 131 may acquire a 3D model of the shape of the entrance of the user's ear canal based on depth data acquired by a depth sensor included in the user terminal 200. This allows the user to easily enjoy the acoustic processing according to the present disclosure without using dedicated facilities, tools, or the like.

[0109] In the third technique, the acquisition unit 131 acquires the user's personal HpTF as user information, and the acoustic device 100 calculates the open-circuit HRTF of the user using the personal HpTF.

[0110] In a fourth technique, the acquisition unit 131 acquires, as user information, a general-purpose HpTF obtained from the HpTFs of any multiple users. In the fourth technique, the acoustic device 100 can obtain the open HRTF of a target user by applying a general-purpose HpTF (e.g., a representative HP correction characteristic) when using a general-purpose HRTF (e.g., an average ear canal open HRTF).

[0111] The calculation unit 132 calculates the open HRTF to be applied to the user by performing acoustic simulation to obtain the open HRTF of the user or by correcting the occluded HRTF based on the user information. The acoustic simulation method corresponds to the first and second methods. The correction method corresponds to the third and fourth methods.

[0112] In the first method, the calculation unit 132 uses actual measurement data regarding the shape of the user's ear canal to estimate the impedance to be set in the user's eardrum portion through acoustic simulation, and calculates the open HRTF to be applied to the user based on the estimated impedance.

[0113] Specifically, the calculation unit 132 regards the closed end of a 3D model generated based on actual measurement data regarding the shape of the user's ear canal as the user's eardrum, sets the impedance of the eardrum at the closed end, and performs an acoustic simulation based on the set impedance.

[0114] More specifically, the calculation unit 132 adjusts the impedance so that the transfer characteristics in the acoustic simulation based on the impedance set for the eardrum portion approach the actually measured data of the ear canal transfer characteristics of the user. Then, the calculation unit 132 estimates the adjusted impedance as the eardrum impedance and calculates the open HRTF to be applied to the user.

[0115] In the adjustment process, for example, as shown in Fig. 9, the calculation unit 132 adjusts the reflectance and phase characteristics set for the eardrum portion so that the resonance frequency in the transfer characteristics in the acoustic simulation and the measurement level at that resonance frequency approach the actual measurement data of the transfer characteristics of the user's ear canal. Through this process, the calculation unit 132 can estimate the eardrum impedance that will obtain a result closer to the actual measurement data.

[0116] The calculation unit 132 may estimate the tympanic membrane impedance using a machine learning technique. That is, the calculation unit 132 may estimate the tympanic membrane impedance applied to the user who is the measurement target using a prediction model trained using a combination of multiple pieces of actual measurement data and the impedance of the tympanic membrane as training data.

[0117] In the second method, the calculation unit 132 calculates the open-circuit HRTF to be applied to the user by acoustic simulation using a 3D model of the shape of the entrance of the user's ear canal.

[0118] For example, the calculation unit 132 measures the impedance at the entrance of the ear canal and sets the measured impedance as the boundary condition of a predetermined surface inside the 3D ear canal model. Then, the calculation unit 132 executes an acoustic simulation to calculate sound pressure characteristics at an observation point near the predetermined surface, thereby calculating the open-circuit HRTF to be applied to the user.

[0119] The calculation unit 132 may generate a correction filter based on the difference between an open HRTF using a general-purpose ear canal 3D model and a occlusion HRTF corresponding to the open HRTF through acoustic simulation. The occlusion HRTF corresponding to the open HRTF may be, for example, an occlusion HRTF obtained from the same user who obtained the open HRTF (in other words, under the same simulation conditions), or an occlusion HRTF obtained from a user similar to the user who obtained the open HRTF. This is because the difference between the open HRTF and the occlusion HRTF obtained from the same user can be considered to be due only to the influence of the general-purpose ear canal. The calculation unit 132 may then use the correction filter to calculate the open HRTF to be applied to the user from the occlusion HRTF of the user obtained by a simple method (e.g., estimated from image data obtained by capturing an image of the user's ear).

[0120] In the above process, the calculation unit 132 may calculate an open HRTF to be applied to a user from the user's occluded HRTF using a learning model that has learned the characteristics of the difference between an open HRTF using a general-purpose 3D model of the ear canal and an occluded HRTF corresponding to the open HRTF.

[0121] In the third technique, the calculation unit 132 calculates an open HRTF to be applied to a user from the closed HRTF of the user using a correction filter generated based on characteristics related to the ear canal.

[0122] Specifically, the calculation unit 132 adjusts the frequency and gain to be applied to the correction filter based on the frequency and sound pressure level identified by comparing the user's HpTF characteristics with the difference in sound pressure characteristics between a general-purpose closed HRTF and an open HRTF. That is, as shown in FIG. 18 , the calculation unit 132 extracts the frequency and sound pressure level based on the influence of the ear canal from the difference between the personal HpTF and the PD. Then, the calculation unit 132 generates a correction filter (e.g., frequency shift or gain adjustment to increase or decrease the sound pressure level) to reflect the influence of the ear canal. This method allows the user to obtain the open HRTF that applies to them without the effort of acquiring a 3D model of the ear canal or measuring the impedance at the entrance of the ear canal.

[0123] In addition, in the fourth technique, the calculation unit 132 calculates an open HRTF to be applied to a user by correcting the closed HRTF of the user based on characteristics related to the ear canal derived from a general-purpose HpTF.

[0124] Specifically, when the calculation unit 132 acquires the HpTFs of any multiple users, it corrects the ear canal characteristics of each user included in the multiple HpTFs based on the average resonance characteristics obtained from the difference between the average closed HRTF and the open HRTF. That is, the calculation unit 132 corrects the ear canal characteristics of the multiple HpTFs to average data (i.e., aligns them to the resonance characteristics of the average ear canal), thereby obtaining a general-purpose correction filter (correction filter 42 in this embodiment) that is not affected by a specific individual. Furthermore, when using the average open ear canal HRTF obtained using such a correction filter, the calculation unit 132 uses the representative HP correction characteristic. This allows the calculation unit 132 to correct the closed HRTF to a natural open HRTF.

[0125] The output unit 133 outputs various information. For example, the output unit 133 outputs the open HRTF calculated by the calculation unit 132 to the user terminal 200. By applying the open HRTF to a music playback app or the like, the user can enjoy stereophonic sound reproduced with higher accuracy.

[0126] The measurement unit 134 controls measurement processing for obtaining various data according to the present disclosure. Fig. 3B shows an example of the measurement processing executed by the measurement unit 134. Fig. 3B is a diagram showing an example of a measurement system 260 according to an embodiment.

[0127] 3B shows an example in which the measurement system 260 measures ear canal impedance at the entrance of the user's ear canal. As shown in FIG. 3B, the measurement unit 134 of the acoustic device 100 outputs a predetermined stimulus sound (e.g., a TSP (Time Stretched Pulse) signal) from a driver (signal output unit) via an audio interface. Note that a synchronization click sound loop is executed within the audio interface to synchronize the audio.

[0128] The audio signal is output from a driver in an earplug placed in the user's ear canal. The impedance of the measurement device, including the earplug in which the driver and microphone are installed, is measured in advance through calibration using a metal tube. The metal tube is designed as a tube whose impedance can be theoretically calculated, and multiple lengths may be used as appropriate.

[0129] A probe microphone installed inside the earplug picks up the output signal, and the audio interface returns the picked-up signal to the acoustic device 100. Through this measurement, the acoustic device 100 can obtain information (the output signal and the signal transmitted inside the ear canal) for calculating the impedance of the user's ear canal.

[0130] As described above, the impedance of the earplug (measuring device) including the driver and microphone is measured in advance, so the acoustic device 100 can calculate the ear canal impedance by removing the influence of the impedance of the device itself from the measured ear canal sound pressure characteristics.

[0131] Furthermore, the measurement shown in Figure 3B is an example, and the measurement unit 134 can control and execute measurements to obtain various data required for the acoustic processing related to the present disclosure based on various known methods.

[0132] (1-7. Modifications of the Embodiment) The information processing according to the embodiment described above may be modified in various ways. Modifications of the embodiment will be described below.

[0133] In the above embodiment, an example was shown in which the acoustic device 100 acquires various information and calculates the open HRTF.

[0134] However, the acoustic processing according to the embodiment does not necessarily have to be performed by the acoustic device 100. For example, the acoustic processing according to the embodiment may be performed by the user terminal 200 or the headphones 250, as long as the device is capable of executing the acoustic program according to the present disclosure.

[0135] (2. Other Embodiments) The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0136] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. Furthermore, the information, including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings, can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0137] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0138] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0139] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0140] (3. Effects of the Acoustic Device According to the Present Disclosure) As described above, the acoustic device according to the present disclosure (acoustic device 100 in the embodiment) includes an acquisition unit (acquisition unit 131 in the embodiment) and a calculation unit (calculation unit 132 in the embodiment). The acquisition unit acquires user information, which is information about a user, from a user who is a target of measurement of an open-ear-canal head-related transfer function, which is a head-related transfer function in an open ear canal state. The calculation unit calculates the open-ear-canal head-related transfer function to be applied to the user, based on the user information, by performing an acoustic simulation to obtain the open-ear-canal head-related transfer function of the user, or by correcting the closed-ear-canal head-related transfer function, which is a head-related transfer function in an closed ear canal state.

[0141] In this way, the acoustic device according to the present disclosure calculates open HRTFs applicable to the user by either simulation or correction, depending on information obtained from the user. This allows the acoustic device to provide the user with open HRTFs that more accurately reflect the transmission of sound to the user, rather than the conventionally commonly used occlusion HRTFs calculated when the ear canal is closed. In other words, the acoustic device can calculate HRTFs that can reproduce stereophonic sound with higher accuracy.

[0142] The acquisition unit acquires measured data on the shape of the user's ear canal. The calculation unit estimates an impedance to be set at the eardrum of the user through acoustic simulation using the measured data on the shape of the user's ear canal, and calculates an open-ear head-related transfer function to be applied to the user based on the estimated impedance.

[0143] Specifically, the calculation unit generates a 3D model based on actual measurement data regarding the shape of the user's ear canal, regards the closed end of the 3D model as the user's eardrum, sets the impedance of the eardrum at the closed end, and performs an acoustic simulation based on the set impedance.

[0144] In this way, the acoustic device performs acoustic simulation by setting the eardrum impedance, and therefore can execute a simulation that accurately reflects the vibration of the eardrum, the sound absorption characteristics when sound propagates from the eardrum to the middle ear, etc. This allows the acoustic device to calculate open-circuit HRTFs with higher accuracy.

[0145] The acquisition unit acquires actual measurement data of the ear canal transfer characteristics of the user based on the difference between the frequency characteristics of sound measured by a microphone placed at the entrance of the ear canal and a microphone placed at the eardrum. The calculation unit adjusts the impedance so that the transfer characteristics in the acoustic simulation based on the impedance set at the eardrum come closer to the actual measurement data of the ear canal transfer characteristics of the user, and calculates an ear canal open head related transfer function to be applied to the user based on the adjusted impedance.

[0146] The calculation unit also estimates the impedance to be set in the eardrum portion by adjusting the reflectivity and phase characteristics to be set in the eardrum portion so that the resonance frequency in the transfer characteristics in the acoustic simulation and the measurement level at that resonance frequency approach the actual measurement data of the transfer characteristics of the user's ear canal.

[0147] In this way, the acoustic device can estimate the eardrum impedance by adjusting the reflectance and phase characteristics based on actual measurement data, thereby more accurately reproducing the transfer characteristics inside the user's ear, which are difficult to measure.

[0148] The acquisition unit acquires a plurality of pieces of actual measurement data relating to the shape of the ear canal and an impedance of the eardrum portion applied to the plurality of ear canals. The calculation unit estimates the impedance of the eardrum portion applied to the user who is the measurement target, using a prediction model trained using a combination of the plurality of actual measurement data and the impedance of the eardrum portion as training data.

[0149] In this way, the acoustic device can estimate the eardrum impedance using a learning model, thereby reducing the processing load and performing highly accurate estimation processing.

[0150] In addition, the acquisition unit uses a measurement signal output from an output unit provided in a terminal device used by the user (in this embodiment, the user terminal 200) to acquire actual measurement data of the ear canal transfer characteristics measured by the user from the terminal device via communication.

[0151] In this way, the acoustic device may perform the acoustic processing according to the present disclosure using a signal measured using a device used by the user, such as a smartphone, thereby allowing the user to perform the acoustic processing according to the present disclosure without having to go to a special facility or the like.

[0152] The acquisition unit acquires a 3D model of the shape of the ear canal entrance of the user as user information, and the calculation unit calculates an ear canal open head-related transfer function to be applied to the user through acoustic simulation using the 3D model of the shape of the ear canal entrance of the user.

[0153] For example, the calculation unit measures the impedance at the entrance of the ear canal, sets the measured impedance as the boundary condition of a specified surface inside the 3D ear canal model, and performs an acoustic simulation to calculate the sound pressure characteristics of an observation point near the specified surface, thereby calculating the ear canal open head-related transfer function to be applied to the user.

[0154] In this way, the acoustic device can perform the acoustic processing according to the present disclosure by setting the ear canal impedance even when it is difficult to obtain 3D model data of the user's ear canal using MRI, etc. In other words, the acoustic device can provide the user with highly accurate open HRTFs while reducing the burden on the user.

[0155] The acquisition unit acquires an open-ear-canal head-related transfer function obtained by acoustic simulation using a general-purpose 3D ear canal model, and the calculation unit calculates the open-ear-canal head-related transfer function to be applied to the user from the closed-ear-canal head-related transfer function of the user using a correction filter generated based on a difference between the open-ear-canal head-related transfer function obtained using the general-purpose 3D ear canal model and the closed-ear-canal head-related transfer function corresponding to the open-ear-canal head-related transfer function.

[0156] In this way, the acoustic device can generate a correction filter for correcting the closed HRTF to the open HRTF through acoustic simulation, and therefore can easily calculate the open HRTF without actual measurement or the like.

[0157] The acquisition unit acquires an open-ear-canal head-related transfer function obtained by acoustic simulation using a general-purpose 3D ear canal model, and the calculation unit calculates the open-ear-canal head-related transfer function to be applied to the user from the closed-ear-canal head-related transfer function of the user using a learning model that has learned features of the difference between the open-ear-canal head-related transfer function obtained using the general-purpose 3D ear canal model and the closed-ear-canal head-related transfer function corresponding to the open-ear-canal head-related transfer function.

[0158] In this way, the acoustic device can easily and quickly calculate the open HRTF through processing that utilizes machine learning.

[0159] The acquisition unit also acquires a 3D model of the shape of the entrance of the user's ear canal based on depth data obtained by a depth sensor included in the terminal device used by the user.

[0160] In this way, the acoustic device can acquire the shape based on depth data obtained by a smartphone or the like, and therefore can collect the information necessary for processing without requiring dedicated equipment or the like.

[0161] The acquisition unit acquires the user's actually measured headphone characteristics as the user information. The calculation unit calculates an ear-canal open-ear related transfer function to be applied to the user from the ear-canal closed-ear related transfer function using a correction filter generated based on ear-canal characteristics derived from the user's headphone characteristics.

[0162] For example, the calculation unit adjusts the frequency and gain to be applied to the correction filter based on the frequency and sound pressure level identified based on a comparison between the user's headphone characteristics and the sound pressure characteristic difference between the ear canal closed head related transfer function and the ear canal open head related transfer function.

[0163] In this way, even if it is difficult for the user to measure the ear canal impedance, the acoustic device can calculate an open HRTF with high accuracy by using the user's personal headphone characteristics. In other words, the user can obtain an open HRTF that is optimized for them without having to take the time and effort to measure or prepare the necessary equipment or devices for the measurement.

[0164] The acquisition unit acquires general-purpose headphone characteristics obtained from the headphone characteristics of any of a plurality of users, and the calculation unit applies the general-purpose headphone characteristics when an ear-canal open-ear related transfer function is calculated from an ear-canal occlusion head-related transfer function of the user using a correction filter generated based on ear-canal characteristics derived from the headphone characteristics of the plurality of users.

[0165] For example, when the calculation unit acquires the headphone characteristics of any multiple users, it calculates the difference between the average and each individual user in ear canal resonance from the difference between the open ear canal head-related transfer function in the ear canal of each individual user and the open ear canal head-related transfer function in a general-purpose ear canal, corrects the characteristics related to the ear canal of each individual user included in each of the multiple headphone characteristics based on the calculated difference, and then calculates the general-purpose headphone characteristics.

[0166] In this way, the audio device can easily calculate the open HRTF by adding the average ear canal characteristics to the closed HRTF without obtaining personalized headphone characteristics, allowing the user to easily enjoy an acoustic experience based on the open HRTF.

[0167] (4. Hardware Configuration) Information devices such as the acoustic device 100 according to each of the above-described embodiments are realized by a computer 1000 configured as shown in FIG. 20 . The following description will be given using the acoustic device 100 according to the embodiments as an example. FIG. 20 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the acoustic device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The various components of the computer 1000 are connected by a bus 1050.

[0168] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0169] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0170] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an audio program according to the present disclosure, which is an example of program data 1450.

[0171] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0172] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.

[0173] For example, when the computer 1000 functions as the acoustic device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes an acoustic program loaded onto the RAM 1200 to realize the functions of the control unit 130, etc. The acoustic program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0174] The present technology can also be configured as follows: (1) An acoustic device comprising: an acquisition unit that acquires, from a user who is a target for measurement of an open-ear-canal head-related transfer function, which is a head-related transfer function in an open ear canal state, user information that is information about the user; and a calculation unit that calculates an open-ear-canal head-related transfer function to be applied to the user by performing an acoustic simulation to obtain the open-ear-canal head-related transfer function of the user based on the user information, or by correcting an closed-ear-canal head-related transfer function, which is a head-related transfer function in an closed ear canal state. (2) The acoustic device described in (1), wherein the acquisition unit acquires actual measurement data regarding the shape of the user's ear canal, and the calculation unit estimates an impedance to be set at the user's eardrum by the acoustic simulation using the actual measurement data regarding the shape of the user's ear canal, and calculates the open-ear-canal head-related transfer function to be applied to the user based on the estimated impedance. (3) The acoustic device according to (1) or (2), wherein the calculation unit regards a closed end of a 3D model generated based on actual measurement data regarding the shape of the user's ear canal as the user's eardrum, sets an impedance of the eardrum at the closed end, and performs the acoustic simulation based on the set impedance. (4) The acoustic device according to (3), wherein the acquisition unit acquires, as the actual measurement data, actual measurement data of ear canal transfer characteristics of the user based on a difference between frequency characteristics of sound measured by a microphone placed at the entrance of the ear canal and a microphone placed in the eardrum, and the calculation unit adjusts the impedance so that the transfer characteristics in the acoustic simulation based on the impedance set in the eardrum are closer to the actual measurement data of the ear canal transfer characteristics of the user, and calculates an ear canal open head related transfer function to be applied to the user based on the adjusted impedance. (5) The acoustic device according to (4), wherein the calculation unit estimates the impedance to be set in the eardrum portion by adjusting the reflectance and phase characteristics to be set in the eardrum portion so that a resonance frequency in the transfer characteristics in the acoustic simulation and a measurement level at the resonance frequency approach actual measurement data of the ear canal transfer characteristics of the user.(6) The acoustic device according to any one of (2) to (5), wherein the acquisition unit acquires a plurality of pieces of actual measurement data relating to the shape of the ear canal and impedances of the eardrum portion applied to the plurality of ear canals, and the calculation unit estimates the impedance of the eardrum portion applied to the user who is the measurement target using a prediction model trained using a combination of the plurality of actual measurement data and the impedance of the eardrum portion as training data. (7) The acoustic device according to any one of (2) to (6), wherein the acquisition unit acquires, via communication, actual measurement data of ear canal transfer characteristics measured by the user from a terminal device used by the user, using a measurement signal output from an output unit included in the terminal device. (8) The acoustic device according to any one of (1) to (7), wherein the acquisition unit acquires, as the user information, a 3D model of the shape of the entrance of the ear canal of the user, and the calculation unit calculates an ear canal open head-related transfer function applied to the user by the acoustic simulation using the 3D model of the shape of the entrance of the ear canal of the user. (9) The acoustic device according to (8), wherein the calculation unit measures an impedance at the entrance of the ear canal, sets the measured impedance as a boundary condition of a predetermined surface inside the ear canal 3D model, and calculates an open ear canal head-related transfer function to be applied to the user by executing an acoustic simulation that calculates sound pressure characteristics of an observation point near the predetermined surface. (10) The acoustic device according to (9), wherein the acquisition unit acquires the open ear canal head-related transfer function obtained by the acoustic simulation using the general-purpose ear canal 3D model, and the calculation unit calculates the open ear canal head-related transfer function to be applied to the user from the open ear canal head-related transfer function of the user using a correction filter that is generated based on a difference between the open ear canal head-related transfer function using the general-purpose ear canal 3D model and an open ear canal head-related transfer function corresponding to the open ear canal head-related transfer function.(11) The acoustic device according to (9) or (10), wherein the acquisition unit acquires an open ear canal head-related transfer function obtained by the acoustic simulation using the general-purpose 3D ear canal model, and the calculation unit calculates an open ear canal head-related transfer function applied to the user from the closed ear canal head-related transfer function of the user using a learning model that has learned features of a difference between the open ear canal head-related transfer function using the general-purpose 3D ear canal model and an closed ear canal head-related transfer function corresponding to the open ear canal head-related transfer function. (12) The acoustic device according to any of (8) to (11), wherein the acquisition unit acquires a 3D model of the shape of the entrance of the ear canal of the user based on depth data obtained by a depth sensor included in a terminal device used by the user. (13) The acoustic device according to any one of (1) to (12), wherein the acquisition unit acquires, as the user information, actually measured headphone characteristics of the user, and the calculation unit calculates, from the ear canal occlusion head related transfer function of the user, an ear canal open head related transfer function to be applied to the user, using a correction filter generated based on ear canal characteristics derived from the user's headphone characteristics. (14) The acoustic device according to (13), wherein the calculation unit adjusts a frequency and a gain to be applied to the correction filter, based on a frequency and a sound pressure level identified based on a comparison between the user's headphone characteristics and a sound pressure characteristic difference between the ear canal occlusion head related transfer function and the ear canal open head related transfer function. (15) The acoustic device according to any one of (1) to (14), wherein the acquisition unit acquires general-purpose headphone characteristics obtained from the headphone characteristics of any of a plurality of users, and the calculation unit applies the general-purpose headphone characteristics when an ear canal open head related transfer function is calculated from an ear canal occlusion head related transfer function of the user using a correction filter generated based on ear canal characteristics derived from the headphone characteristics of the plurality of users.(16) The acoustic device according to (15), wherein the calculation unit, when acquiring headphone characteristics of any of the plurality of users, calculates a difference between an average and an open-ear head-related transfer function in ear canals of each user from a difference between the open-ear head-related transfer function in the ear canal of each user and the open-ear head-related transfer function in a general-purpose ear canal, corrects the characteristics related to the ear canals of each user included in each of the plurality of headphone characteristics based on the calculated difference, and then calculates the general-purpose headphone characteristics. (17) An acoustic method including a computer acquiring user information that is information about a user from a user who is a target for measurement of the open-ear head-related transfer function, which is a head-related transfer function in an open ear canal state, based on the user information, and calculating the open-ear head-related transfer function to be applied to the user by performing an acoustic simulation to obtain the open-ear head-related transfer function of the user or correcting the closed-ear head-related transfer function, which is a head-related transfer function in an closed ear canal state, based on the user information. (18) An acoustic program for causing a computer to function as an acoustic device comprising: an acquisition unit that acquires user information, which is information about a user who is a target for measurement of an open-ear ear canal head-related transfer function, which is a head-related transfer function in an open ear canal state, from the user; and a calculation unit that calculates the open-ear ear canal head-related transfer function to be applied to the user by performing an acoustic simulation to obtain the open-ear ear canal head-related transfer function of the user based on the user information, or by performing correction of the closed-ear ear canal head-related transfer function, which is a head-related transfer function in an closed ear canal state.

[0175] REFERENCE SIGNS LIST 100 Acoustic device 110 Communication unit 120 Storage unit 130 Control unit 131 Acquisition unit 132 Calculation unit 133 Output unit 134 Measurement unit 200 User terminal 250 Headphones

Claims

1. An acoustic device comprising: an acquisition unit that acquires user information, which is information about a user, from a user who is to be measured for an open-ear-canal head-related transfer function, which is a head-related transfer function in an open ear canal state; and a calculation unit that calculates the open-ear-canal head-related transfer function to be applied to the user based on the user information by performing an acoustic simulation to obtain the open-ear-canal head-related transfer function of the user, or by performing a correction of a closed-ear-canal head-related transfer function, which is a head-related transfer function in a closed ear canal state.

2. The acoustic device of claim 1, wherein the acquisition unit acquires actual measurement data regarding the shape of the user's ear canal, and the calculation unit uses the actual measurement data regarding the shape of the user's ear canal to estimate an impedance to be set in the user's eardrum portion by the acoustic simulation, and calculates an ear canal open head-related transfer function to be applied to the user based on the estimated impedance.

3. The acoustic device of claim 1, wherein the calculation unit regards a closed end of a 3D model generated based on actual measurement data regarding the shape of the user's ear canal as the user's eardrum, sets an impedance of the eardrum part at the closed end, and performs the acoustic simulation based on the set impedance.

4. The acoustic device of claim 3, wherein the acquisition unit acquires as the actual measurement data actual measurement data of the ear canal transfer characteristics of the user based on the difference in frequency characteristics of sound measured by a microphone placed at the entrance of the ear canal and a microphone placed in the eardrum portion, and the calculation unit adjusts the impedance so that the transfer characteristics in the acoustic simulation based on the impedance set in the eardrum portion approaches the actual measurement data of the ear canal transfer characteristics of the user, and calculates an ear canal open head related transfer function to be applied to the user based on the adjusted impedance.

5. The acoustic device of claim 4, wherein the calculation unit estimates the impedance to be set in the eardrum portion by adjusting the reflectance and phase characteristics to be set in the eardrum portion so that the resonant frequency in the transfer characteristics in the acoustic simulation and the measurement level at the resonant frequency approach the actual measured data of the transfer characteristics of the user's ear canal.

6. The acoustic device of claim 2, wherein the acquisition unit acquires a plurality of actual measurement data relating to the shape of the ear canal and the impedance of the eardrum portion applied to the plurality of ear canals, and the calculation unit estimates the impedance of the eardrum portion applied to the user who is the subject of measurement using a prediction model trained on a combination of the plurality of actual measurement data and the impedance of the eardrum portion as training data.

7. The acoustic device according to claim 2, wherein the acquisition unit acquires actual measurement data of the ear canal transfer characteristics measured by the user from a terminal device used by the user via communication using a measurement signal output from an output unit provided in the terminal device.

8. The acoustic device of claim 1, wherein the acquisition unit acquires a 3D model of the shape of the entrance of the ear canal of the user as the user information, and the calculation unit calculates an ear canal open head-related transfer function to be applied to the user by the acoustic simulation using the 3D model of the shape of the entrance of the ear canal of the user.

9. The acoustic device of claim 8, wherein the calculation unit measures the impedance at the entrance of the ear canal, sets the measured impedance as a boundary condition of a specified surface inside the 3D model of the ear canal, and calculates an ear canal open head-related transfer function to be applied to the user by performing an acoustic simulation to calculate sound pressure characteristics of an observation point in the vicinity of the specified surface.

10. The acoustic device of claim 9, wherein the acquisition unit acquires an ear canal open head transfer function obtained by the acoustic simulation using the general-purpose ear canal 3D model, and the calculation unit calculates an ear canal open head transfer function to be applied to the user from the ear canal occlusion head transfer function of the user using a correction filter generated based on the difference between the ear canal open head transfer function using the general-purpose ear canal 3D model and the ear canal occlusion head transfer function corresponding to the ear canal open head transfer function.

11. The acoustic device of claim 9, wherein the acquisition unit acquires an ear canal open head transfer function obtained by the acoustic simulation using the general-purpose ear canal 3D model, and the calculation unit calculates an ear canal open head transfer function to be applied to the user from the ear canal occlusion head transfer function of the user using a learning model that has learned the characteristics of the difference between the ear canal open head transfer function using the general-purpose ear canal 3D model and the ear canal occlusion head transfer function corresponding to the ear canal open head transfer function.

12. The acoustic device according to claim 8, wherein the acquisition unit acquires a 3D model of the shape of the entrance of the user's ear canal based on depth data acquired by a depth sensor provided in a terminal device used by the user.

13. The acoustic device of claim 1, wherein the acquisition unit acquires the user's actually measured headphone characteristics as the user information, and the calculation unit calculates an ear canal open head related transfer function applied to the user from the ear canal occlusion head related transfer function of the user using a correction filter generated based on characteristics related to the ear canal derived from the user's headphone characteristics.

14. The acoustic device of claim 13, wherein the calculation unit adjusts the frequency and gain to be applied to the correction filter based on a frequency and sound pressure level identified based on a comparison between the user's headphone characteristics and a sound pressure characteristic difference between an ear canal closed head related transfer function and an ear canal open head related transfer function.

15. The acoustic device of claim 1, wherein the acquisition unit acquires generic headphone characteristics obtained from the headphone characteristics of any of a plurality of users, and the calculation unit applies the generic headphone characteristics when an ear canal open head related transfer function is calculated from an ear canal occlusion head related transfer function of the user using a correction filter generated based on characteristics related to the ear canal derived from the headphone characteristics of the plurality of users.

16. The acoustic device described in claim 15, wherein the calculation unit, when acquiring headphone characteristics of any of the multiple users, calculates a difference between the individual user and the average in ear canal resonance from the difference between the ear canal open head related transfer function in the ear canal of each user and the ear canal open head related transfer function in a generic ear canal, corrects the characteristics related to the ear canal of each user included in each of the multiple headphone characteristics based on the calculated difference, and then calculates the generic headphone characteristics.

17. An acoustic method comprising: a computer acquiring user information, which is information about a user, from the user who is to be measured for an open-ear-canal head-related transfer function, which is a head-related transfer function in an open ear canal; and calculating the open-ear-canal head-related transfer function to be applied to the user based on the user information by performing an acoustic simulation to obtain the open-ear-canal head-related transfer function of the user, or by performing correction of the closed-ear-canal head-related transfer function, which is a head-related transfer function in a closed ear canal.

18. An acoustic program for causing a computer to function as an acoustic device comprising: an acquisition unit that acquires user information, which is information about a user, from a user who is to be measured for an open-ear ear canal head transfer function, which is a head transfer function in an open ear canal state; and a calculation unit that calculates the open-ear ear canal head transfer function to be applied to the user based on the user information by performing an acoustic simulation to obtain the open-ear ear canal head transfer function of the user, or by performing correction of the closed-ear ear canal head transfer function, which is a head transfer function in a closed ear canal state.

Images