Acoustic signal generation device, acoustic signal generation method, and program

Abstract

One aspect of the present invention is an acoustic signal generation device (1) that comprises an open headphone (20) that includes a first speaker and is worn by a user, a speaker unit (30) that includes a second speaker and is positioned in the surroundings of the user, and a processing unit (10) that includes an acoustic signal acquisition unit (111) that acquires a first acoustic signal that is a first sound source as reproduced from a second position that corresponds to a first position of the first speaker and captured at a third position that corresponds to the initial position of the head of the user and a second acoustic signal that is the first sound source as reproduced from a fifth position that corresponds to a fourth position of the second speaker and captured at the third position, a first correction unit (112) that corrects the first acoustic signal on the basis of the first acoustic signal and the second acoustic signal to generate a third acoustic signal, and a second correction unit (114) that corrects the third acoustic signal and the second acoustic signal on the basis of the initial position, the current position of the head of the user, and the fourth position.

Description

Acoustic signal generating device, acoustic signal generating method, and program

[0001] One aspect of the present invention relates to an audio signal generating device, an audio signal generating method, and a program.

[0002] In recent years, content that provides a high level of immersion using a head-mounted display (HMD), such as virtual reality (VR) or augmented reality (AR), has become widespread. To provide a content user with a more immersive experience, it is necessary to provide an experience that is comparable to an experience in real space. To achieve this, it is necessary to reproduce a certain acoustic space in a different space. Known methods for reproducing an acoustic space include, for example, a method using headphones and a method using a speaker array consisting of multiple speakers.

[0003] In the headphone-based method, for example, an HMD equipped with headphones or an HMD with built-in speakers that do not block the ears is used. In these methods using headphones or speakers placed near the head, sound images can be localized by using a head-related transfer function. A head-related transfer function is a signal containing sound propagation information, and is capable of reproducing the volume and phase difference of sound between the two ears. However, since head-related transfer functions vary from person to person, there are problems such as incorrectly determining the front or back of the sound image position or the sound image being localized inside the head.

[0004] On the other hand, speaker array techniques include surround systems using multiple speakers arranged around the listener, and wave field synthesis systems that physically reproduce a sound field. These techniques can accurately localize a sound image for the listener. However, they have known issues, such as being dependent on the size of the device and having a limited listening position (sweet spot).

[0005] Non-Patent Document 1 describes a method for correcting signals entering both ears when a speaker and a full open-air headphone are used in combination.

[0006] Ikuichiro Kinoshita et al., "Compensation for interaural differences in two-channel reproduction using both loudspeakers and full open-air headphones," Journal of the Acoustical Society of Japan, Vol. 61, No. 4, 2005, pp. 179-191

[0007] However, the method described in Non-Patent Document 1 assumes that the listener is stationary, and it has been confirmed that errors become larger when the listener's head actually moves. This makes it difficult to respond in real time. In order to combine virtual space and real space using content using an HMD, such as VR or AR, an audio reproduction technology that responds to the listener's movements in real time is required.

[0008] This invention was made with the above-mentioned circumstances in mind, and provides an acoustic signal generation technology for reproducing sound in real time in response to the listener's head movements, using open headphones worn by the listener and speakers placed around the listener.

[0009] In order to solve the above problem, an acoustic signal generating device according to one aspect of the present invention includes open-type headphones that include a first speaker and are worn by a user; a speaker unit that includes a second speaker and is arranged around the user; an acoustic signal acquisition unit that acquires a first acoustic signal in which a first sound source is reproduced from a second position corresponding to a first position of the first speaker and picked up at a third position corresponding to an initial position of the user's head, and a second acoustic signal in which the first sound source is reproduced from a fifth position corresponding to a fourth position of the second speaker and picked up at the third position; a first correction unit that generates a third acoustic signal by correcting the first acoustic signal based on the first acoustic signal and the second acoustic signal; and a second correction unit that corrects the third acoustic signal and the second acoustic signal based on the initial position, the current position of the user's head, and the fourth position.

[0010] An acoustic signal generation method according to one aspect of the present invention is an acoustic signal generation method executed by an acoustic signal generation device including open-type headphones worn by a user, the open-type headphones including a first speaker, a speaker unit including a second speaker and arranged around the user, and a processing unit, the method comprising the steps of: acquiring a first acoustic signal in which a first sound source is reproduced from a second position corresponding to a first position of the first speaker and picked up at a third position corresponding to an initial position of the user's head; and acquiring a second acoustic signal in which the first sound source is reproduced from a fifth position corresponding to a fourth position of the second speaker and picked up at the third position; generating a third acoustic signal by correcting the first acoustic signal based on the first acoustic signal and the second acoustic signal; and correcting the third acoustic signal and the second acoustic signal based on the initial position, the current position of the user's head, and the fourth position.

[0011] According to one aspect of the present invention, an acoustic signal can be generated to reproduce sound in real time in response to the listener's head movements using open headphones worn by the listener and speakers placed around the listener.

[0012] FIG. 1 is a schematic diagram showing an example of the configuration of an acoustic signal generation device according to an embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of an acoustic signal generation device according to an embodiment. FIG. 3 is a block diagram showing an example of the functional configuration of an acoustic signal generation device according to an embodiment. FIG. 4 is a diagram showing an example of a method for collecting an acoustic signal using a dummy head. FIG. 5 is a diagram showing another example of a method for collecting an acoustic signal using a microphone. FIG. 6 is a diagram showing another example of a method for collecting an acoustic signal using a dummy head. FIG. 7 is a diagram showing a situation when the head of a listener moves in the acoustic signal generation device according to an embodiment. FIG. 8 is a flowchart showing an example of the operation of the acoustic signal generation device according to an embodiment.

[0013] Hereinafter, embodiments will be described with reference to the drawings. In the following description, components having substantially the same functions and configurations will be designated by the same reference numerals. When particularly distinguishing between elements having similar configurations, different letters or numbers may be added to the end of the same reference numerals.

[0014] 1. Embodiments An acoustic signal generating device according to an embodiment will be described below. In the following, an acoustic signal generating device including open headphones worn by a listener (user) and speakers arranged around the listener will be described as an example of the acoustic signal generating device.

[0015] 1.1 Configuration 1.1.1 Configuration of the Acoustic Signal Generation Device The configuration of the acoustic signal generation device according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of the configuration of the acoustic signal generation device.

[0016] The acoustic signal generating device 1 is a device that generates at least two corrected acoustic signals by correcting at least two acoustic signals acquired from the outside. Hereinafter, the at least two acoustic signals acquired from the outside by the acoustic signal generating device 1 will be referred to as "acoustic signal P ＨＬ，ＨＲ (t)" and "acoustic signal P Ｌｉ (t)" (i is an integer equal to or greater than 1). t represents time. ＨＬ，ＨＲ (t) is the acoustic signal P ＨＬ (t) and the acoustic signal P ＨＲ (t). The acoustic signal P ＨＬ，ＨＲ The acoustic signal obtained by correcting (t) is called "acoustic signal P~ ＨＬ，ＨＲ (t)". ​​The acoustic signal P~ ＨＬ，ＨＲ (t) is the acoustic signal P ＨＬ (t) and the acoustic signal P ＨＲ (t). The acoustic signal P Ｌｉ The acoustic signal obtained by correcting (t) is called "acoustic signal P~ Ｌｉ In the example of FIG. 1, the acoustic signal P Ｌｉ (t) is the acoustic signal P Ｌ１ (t), P Ｌ２ (t), P Ｌ３ (t), .... The acoustic signal P Ｌｉ (t) is the acoustic signal P Ｌ１ (t), P~ Ｌ２ (t), P~ Ｌ３ (t), .... The listener receives the acoustic signal P~ ＨＬ，ＨＲ (t) and P Ｌｉ (t) is inserted.

[0017] As shown in FIG. 1 , the acoustic signal generating device 1 includes a processing unit 10 , headphones 20 , and a speaker unit 30 .

[0018] The processing unit 10 is, for example, a computer. The processing unit 10 is configured to be able to communicate with the headphones 20, the speaker unit 30, and other external devices. For example, the processing unit 10 is connected to the headphones 20, the speaker unit 30, and other external devices wirelessly.

[0019] The processing unit 10 receives an audio signal P ＨＬ，ＨＲ (t) and P Ｌｉ The processing unit 10 acquires the initial head position PS0 of the listener U from the headphones 20. Ｔ and current position PS Ｃ (t) = [x Ｃ , y Ｃ , z Ｃ ] Ｔ The processing unit 10 acquires the speaker SP from the speaker unit 30. Ｌｉ Position PS Ｌｉ The processing unit 10 acquires the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t), initial position PS0 and current position PS Ｃ (t), and speaker SP Ｌｉ Position PS Ｌｉ Based on this, the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ The processing unit 10 corrects the acoustic signal P ＨＬ The acoustic signal P obtained by correcting (t) ＨＬ (t) to the speaker SP of the headphone 20 ＨＬ The processing unit 10 transmits the acoustic signal P ＨＲ The acoustic signal P obtained by correcting (t) ＨＲ (t) to the speaker SP of the headphone 20 ＨＲ The processing unit 10 transmits the acoustic signal P Ｌｉ The acoustic signal P obtained by correcting (t) Ｌｉ (t) is the speaker SP of the speaker unit 30 Ｌｉ Send to.

[0020] The headphones 20 are, for example, open type headphones. The headphones 20 include a mounting part AP, a speaker unit 24, and a sensor 25. The mounting part AP is a main body part that is mounted on the head of a listener U. The speaker unit 24 includes a speaker SP ＨＬ and SP ＨＲ Includes speaker SP ＨＬ is the speaker for the left ear. ＨＲ is the speaker for the right ear. ＨＬ and SP ＨＲ is attached to the mounting unit AP so as not to block the ears of the listener U. The sensor 25 detects the position of the head of the listener U. The sensor 25 is disposed on the mounting unit AP, for example.

[0021] The headphones 20 detect the initial position PS0 and the current position PS of the head of the listener U detected by the sensor 25. Ｃ (t) is transmitted to the processing unit 10. The speaker SP of the headphone 20 ＨＬ and SP ＨＲ Each of the signals P is output from the processing unit 10 as an audio signal P ＨＬ，ＨＲ (t) and the acoustic signal P ＨＬ，ＨＲ (t) is output (played back).

[0022] The speaker unit 30 includes one or more speakers SP Ｌｉ Includes speaker SP Ｌｉ is the acoustic signal P~ Ｌｉ (t). That is, the speaker SP Ｌ１ is the acoustic signal P~ Ｌ１ (t) Speaker SP Ｌ２ is the acoustic signal P~ Ｌ２ (t) Speaker SP Ｌ３ is the acoustic signal P~ Ｌ３ In other words, the speaker unit 30 outputs the acoustic signal P Ｌｉ (t) Ｌｉ One or more speakers SP Ｌｉ The speaker unit 30 is arranged, for example, around the listener U. Ｌ１ For example, the position PS Ｌ１ = [x Ｌ１ , yＬ１ , z Ｌ１ ] Ｔ Speaker SP Ｌ２ For example, the position PS Ｌ２ = [x Ｌ２ , y Ｌ２ , z Ｌ２ ] Ｔ Speaker SP Ｌ３ For example, the position PS Ｌ３ = [x Ｌ３ , y Ｌ３ , z Ｌ３ ] Ｔ will be placed in.

[0023] Speaker SP of speaker unit 30 Ｌｉ is output from the processing unit 10 as an acoustic signal P Ｌｉ (t) and the acoustic signal P Ｌｉ (t) is output (played back).

[0024] The listener U receives the acoustic signal P~ output from the headphones 20. ＨＬ，ＨＲ (t), and the acoustic signal P~ output from the speaker unit 30 Ｌｉ Listen to (t).

[0025] 1.1.2 Hardware Configuration of Acoustic Signal Generation Device The hardware configuration of the acoustic signal generation device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the acoustic signal generation device 1.

[0026] (Processing Unit 10) The following describes the hardware configuration of the processing unit 10. As shown in FIG.

[0027] The processor 11 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), etc. The processor 11 executes, for example, an acoustic signal acquisition process, a first correction process, a position information acquisition process, and a second correction process. The acoustic signal acquisition process is performed by acquiring an acoustic signal P ＨＬ，ＨＲ(t) and P Ｌｉ The first correction process is a process for obtaining the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) is corrected to produce an acoustic signal P^ ＨＬ，ＨＲ (t) and P^ Ｌｉ (t) is generated. ＨＬ，ＨＲ (t) is the acoustic signal P^ ＨＬ (t) and the acoustic signal P^ ＨＲ (t). ＨＬ (t) is the acoustic signal P ＨＬ The acoustic signal P^(t) is obtained by correcting the acoustic signal P^(t). ＨＲ (t) is the acoustic signal P ＨＲ The position information acquisition process is carried out by correcting the initial position PS0 and the current position PS1 of the head of the listener U. Ｃ (t), and the speaker SP of the speaker unit 30 Ｌｉ Position PS Ｌｉ The second correction process is a process for obtaining the acoustic signal P^ ＨＬ，ＨＲ (t) and P^ Ｌｉ The corrected acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ This is a process for generating the acoustic signal P~(t). ＨＬ (t) is the acoustic signal P^ ＨＬ The acoustic signal P is obtained by correcting (t). ＨＲ (t) is the acoustic signal P^ ＨＲ The acoustic signal is obtained by correcting (t). The acoustic signal acquisition process, the first correction process, the position information acquisition process, and the second correction process will be described in detail later.

[0028] The memory 12 includes, for example, a read-only memory (ROM) and a random access memory (RAM). The ROM stores, for example, programs for causing the processor 11 to execute the acoustic signal acquisition process, the first correction process, the position information acquisition process, and the second correction process. The RAM is used as a working area for the processor 11. The RAM temporarily stores, for example, the above-mentioned programs executed by the processor 11, as well as data obtained when the acoustic signal acquisition process, the first correction process, the position information acquisition process, and the second correction process are executed.

[0029] The communication interface 13 controls communication between the processing unit 10 and the headphones 20, the speaker unit 30, and other external devices. ＨＬ，ＨＲ (t) and P Ｌｉ The communication interface 13 receives the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) to the processor 11. The communication interface 13 receives the initial position PS0 and the current position PS1 of the head of the listener U from the headphones 20. Ｃ The communication interface 13 receives the initial position PS0 and the current position PS Ｃ The communication interface 13 transmits the signal (t) to the processor 11. Ｌｉ From speaker SP Ｌｉ Position PS Ｌｉ The communication interface 13 receives the position PS Ｌｉ The communication interface 13 transmits the acoustic signal P~ to the processor 11. ＨＬ，ＨＲ (t) and P Ｌｉ The communication interface 13 receives the acoustic signal P ＨＬ，ＨＲ (t) to the headphones 20. The communication interface 13 transmits the acoustic signal P Ｌｉ (t) is transmitted to the speaker unit 30.

[0030] (Headphones 20) The following describes the hardware configuration of the headphones 20. As shown in FIG.

[0031] The processor 21 is, for example, a CPU, an MPU, a GPU, an FPGA, etc. The processor 21 executes, for example, a position information acquisition process. The position information acquisition process acquires the initial position PS0 and the current position PS1 of the head of the listener U. Ｃ The location information acquisition process will be described in detail later.

[0032] The memory 22 includes, for example, a ROM and a RAM. The ROM stores, for example, a program for causing the processor 21 to execute the location information acquisition process. The RAM is used as a work area for the processor 21. The RAM temporarily stores, for example, the above-mentioned program executed by the processor 21 and data during execution of the location information acquisition process.

[0033] The communication interface 23 controls communication between the headphones 20 and the processing unit 10. The communication interface 23 receives the acoustic signal P~ from the processing unit 10. ＨＬ，ＨＲ The communication interface 23 receives the acoustic signal P ＨＬ，ＨＲ (t) to the processor 21. The communication interface 23 receives the initial position PS0 and the current position PS of the head of the listener U from the processor 21. Ｃ The communication interface 23 receives the initial position PS0 and the current position PS Ｃ (t) is sent to the processing unit 10.

[0034] The sensor 25 is, for example, a head tracker. The sensor 25 includes, for example, an acceleration sensor and a gyro sensor.

[0035] The headphones 20 may include a display unit (for example, a display).

[0036] (Speaker unit 30) The speaker unit 30 includes one or more speakers SP. Ｌｉ Includes speaker SP Ｌｉincludes a GNSS (Global Navigation Satellite System) device (not shown). Ｌｉ is the speaker SP acquired by the GNSS device Ｌｉ Position PS Ｌｉ is transmitted to the processing unit 10.

[0037] 1.1.3 Functional Configuration of the Acoustic Signal Generating Device The functional configuration of the acoustic signal generating device 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the acoustic signal generating device 1. Fig. 3 also shows the communication interface 13 of the processing unit 10, the communication interface 23 of the headphones 20, the speaker unit 24, the sensor 25, and the speaker unit 30.

[0038] (Processing Unit 10) The following describes the functional configuration of the processing unit 10. As shown in Fig. 3, the processing unit 10 includes, as functional blocks, an acoustic signal acquisition unit 111, a first correction unit 112, a position information acquisition unit 113, and a second correction unit 114. The processor 11 of the processing unit 10 functions as the acoustic signal acquisition unit 111, the first correction unit 112, the position information acquisition unit 113, and the second correction unit 114.

[0039] (Acoustic Signal Acquisition Unit 111) The acoustic signal acquisition unit 111 executes an acoustic signal acquisition process. Specifically, the acoustic signal acquisition unit 111 receives an acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ The acoustic signal acquisition unit 111 acquires the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) is transmitted to the first correction unit 112.

[0040] (First Correction Unit 112) The first correction unit 112 executes a first correction process. The first correction process is a process of correcting an acoustic signal at the initial position PS0 of the head of the listener U. The first correction process will be described in detail below.

[0041] The first correction unit 112 receives the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) is received.

[0042] The first correction unit 112 corrects the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t), the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) are corrected to produce the acoustic signal P^ ＨＬ，ＨＲ (t) and P^ Ｌｉ Specifically, the first correction unit 112 generates the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t), the acoustic signal P ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ A correction factor t that corrects the time delay between (t) Ｌ，Ｒ , and the acoustic signal P ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ (t) and the correction coefficient c Ｌ，Ｒ The first correction unit 112 calculates the correction coefficient t Ｌ，Ｒ and c Ｌ，Ｒ Based on this, the acoustic signal P̂ ＨＬ，ＨＲ (t) and P^ Ｌｉ (t) is generated.

[0043] First, the speaker unit 30 is a single speaker SP Ｌｉ (SP Ｌ１ ) Correction coefficient t Ｌ，Ｒ and c Ｌ，Ｒ The calculation of the speaker unit 30 is explained below. Ｌｉ The acoustic signal generating device 1 in the case where the acoustic signal generating device 1 includes, for example, the acoustic signal generating device 1 shown in FIG. Ｌ２ , SP Ｌ３ , ... have been eliminated.

[0044] Speaker SP of headphone 20 ＨＬ and SP ＨＲ , and the speaker SP of the speaker unit 30 Ｌｉ When the same sound source is played simultaneously from the speaker SP ＨＬ and SP ＨＲ Acoustic signal P from ＨＬ，ＨＲ Since the sound signal P (t) reaches the listener U first, the sound image may be localized at the position of the headphones 20 due to the precedence effect. ＨＬ，ＨＲ (t) and the acoustic signal PＬｉ (t) and the time delay between them. Ｌ，Ｒ As a method for determining the above, for example, the following method using a dummy head DH can be considered.

[0045] 4 is a diagram showing an example of a method for collecting an acoustic signal using a dummy head DH. As shown in FIG. 4, for example, when the initial position PS0 of the head of a listener U is [0,0,0], Ｔ Position PS0' corresponding to [x0, y0, z0] Ｔ The dummy head DH is a recording device that mimics the shape of a human head. The dummy head DH is connected to the microphone MP Ｌ and MP Ｒ Includes microphone MP Ｌ is provided at a position corresponding to the left ear of the dummy head DH. Ｒ is provided at a position corresponding to the right ear of the dummy head DH.

[0046] For example, in a space where a dummy head DH is arranged as shown in FIG. Ｌｉ '=[x Ｌｉ ',y Ｌｉ ', z Ｌｉ '] Ｔ , position PS Ｈ１ '=[x Ｈ１ ',y Ｈ１ ', z Ｈ１ '] Ｔ , and position PS Ｈ２ '=[x Ｈ２ ',y Ｈ２ ', z Ｈ２ '] Ｔ The same sound source SS is played back simultaneously from the position PS Ｌｉ ' denotes the speaker SP of the speaker unit 30. Ｌｉ Position PS Ｌｉ = [x Ｌｉ , y Ｌｉ , z Ｌｉ ] Ｔ The position corresponds to the position PS Ｈ１ ' is the speaker SP of the headphone 20 ＨＬ Position PS Ｈ１ = [x Ｈ１ , y Ｈ１ , zＨ１ ] Ｔ The position corresponds to the position PS Ｈ２ ' is the speaker SP of the headphone 20 ＨＲ Position PS Ｈ２ = [x Ｈ２ , y Ｈ２ , z Ｈ２ ] Ｔ That is, the position of the speaker SP relative to the initial position PS0. ＨＬ , SP ＨＲ , and SP Ｌｉ The positional relationship of the position PS Ｈ１ ', position PS Ｈ２ ', and position PS Ｌｉ '. When collecting sound with the dummy head DH and when the listener U listens, the speaker SP ＨＬ , SP ＨＲ , and SP Ｌｉ Although the absolute position of the microphone MP1 does not change, in this specification, for ease of understanding, the positions where the sound is picked up by the dummy head DH or the microphone MP1 described later are indicated with "'". When the sound source SS is played back simultaneously, the dummy head DH placed at the position PS0' (more precisely, the microphone MP1 at the position PS0') Ｌ and MP Ｒ at the position of the microphone MP Ｌ and MP Ｒ The acoustic signal is picked up by the speaker SP. ＨＬ and SP ＨＲ It is played from the microphone MP Ｌ and MP Ｒ The acoustic signal collected by the ＨＬ，ＨＲ (t). Speaker SP Ｌｉ It is played from the microphone MP Ｌ and MP Ｒ The acoustic signal collected by the Ｌｉ Let (t) be the time.

[0047] The collected acoustic signal P ＨＬ，ＨＲ (t), and P Ｌｉ(t) is acquired by the acoustic signal acquisition unit 111 via the communication interface 13 and transmitted from the acoustic signal acquisition unit 111 to the first correction unit 112 .

[0048] The first correction unit 112 corrects the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ The correction coefficient t is used to maximize the cross-correlation of (t). Ｌ，Ｒ For example, the first correction unit 112 determines the correction coefficient t that maximizes the cross-correlation function R using the following equation (1): Ｌ，Ｒ As a result, the acoustic signal P ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ (t) can be corrected.

[0049] Also, the speaker SP of the headphone 20 ＨＬ and SP ＨＲ and the speaker SP of the speaker unit 30 Ｌｉ When the sound signal P is reproduced using different devices, the volume differs for each device, and the sound image is localized to the device with the larger volume. ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ The first correction unit 112 corrects the amplitude difference between the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ The correction coefficient c is set so that the maximum amplitude value of (t) coincides. Ｌ，Ｒ For example, the first correction unit 112 determines the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ The correction coefficient c that makes the maximum amplitude value of (t) coincide Ｌ，Ｒ As a result, the acoustic signal P ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ (t) can be corrected. Here, max||P Ｌｉ || is the acoustic signal P Ｌｉ represents the maximum amplitude value of (t). max∥P ＨＬ，ＨＲ || is the acoustic signal P ＨＬ，ＨＲ represents the maximum amplitude value of (t).

[0050] The first correction unit 112 calculates the correction coefficient t Ｌ，Ｒ and c Ｌ，Ｒ Based on this, the acoustic signal P ＨＬ，ＨＲ (t) is corrected to produce an acoustic signal P^ ＨＬ，ＨＲ (t) to generate the corrected acoustic signal P̂ ＨＬ，ＨＲ (t) is expressed as the following equation (3).

[0051] The speaker unit 30 is a single speaker SP Ｌｉ If the acoustic signal P Ｌｉ (t) is the acoustic signal P^ Ｌｉ That is, the acoustic signal P^ (t) generated (output) by the first correction unit 112 is Ｌｉ (t) is the acoustic signal P Ｌｉ It is the same as (t).

[0052] Next, the speaker unit 30 is composed of I speakers SP (I is an integer of 2 or more). Ｌｉ Correction coefficient t when (1≦i≦I) is included Ｌ，Ｒ and c Ｌ，Ｒ The calculation of is explained below.

[0053] A plurality of speakers SP are placed around the listener U. Ｌｉ When the speakers SP are arranged, the acoustic signal received by the listener U is Ｌｉ The acoustic signal P Ｌｉ However, the microphone MP of the dummy head DH needs to be corrected. Ｌ and MP Ｒ Therefore, the first correction unit 112 corrects the sound pressure and phase difference of the I speakers SP so that the sound pressure and phase difference are the same at the initial position PS0 of the head of the listener U. Ｌｉ The acoustic signal output from the input terminal is corrected in the same manner as in equations (1) and (2).

[0054] 5 is a diagram showing another example of a method for collecting an acoustic signal using a microphone MP1. As shown in FIG. 5, for example, when the initial position PS0 of the head of a listener U is [0,0,0], Ｔ Position PS0' corresponding to [x0, y0, z0] Ｔis determined, and microphone MP1 is placed at position PS0'.

[0055] For example, in a space where a microphone MP1 is arranged as shown in FIG. 5, each speaker SP of the speaker unit 30 Ｌｉ (SP Ｌ１ , SP Ｌ２ , SP Ｌ３ , ...) Ｌｉ '(PS Ｌ１ ', P.S. Ｌ２ ', P.S. Ｌ３ The same sound source SS is simultaneously reproduced from the microphones MP1, MP2, and MP3. Ｓｉ (t) (P Ｓ１ (t), P Ｓ２ (t), P Ｓ３ (t), ...). The first correction unit 112 corrects the collected sound signal P Ｓｉ (t) (P Ｓ１ (t), P Ｓ２ (t), P Ｓ３ (t), ...), P Ｓｉ (t) is corrected to produce an acoustic signal P^ Ｓｉ (t), i.e., the acoustic signal P Ｌｉ (t) is corrected to produce an acoustic signal P^ Ｌｉ Specifically, the first correction unit 112 generates the acoustic signal P Ｓｉ (t) to P Ｍａｘ (t), and for the time delay, similarly to equation (1), the correction coefficient t that maximizes the cross-correlation function R is calculated using the following equation (4): Ｓｉ (t Ｓ１ , t Ｓ２ , t Ｓ３ , ...) are calculated. Ｓｉ The time delay between (t) can be corrected.

[0056] Similarly to equation (2), the first correction unit 112 also calculates the amplitude difference by using the following equation (5) to obtain the acoustic signal P Ｓｉ (t) and P Ｍａｘ The correction coefficient c that makes the maximum amplitude value of (t) coincide Ｓｉ (c Ｓ１ , c Ｓ２ , cＳ３ , ...) are calculated. Ｓｉ The amplitude difference between (t) can be corrected. Here, max||P Ｍａｘ || is the acoustic signal P Ｍａｘ represents the maximum amplitude value of (t). max∥P Ｓｉ || is the acoustic signal P Ｓｉ represents the maximum amplitude value of (t).

[0057] The first correction unit 112 calculates the correction coefficient t Ｓｉ and c Ｓｉ Based on this, the acoustic signal P Ｌｉ (t) is corrected to produce an acoustic signal P^ Ｌｉ (t) is generated. Ｍａｘ Speaker SP from which (t) is played Ｌｉ The speaker SP is located closer to the initial position PS0 than the speaker SP Ｌｉ The picked-up sound signal P Ｓｉ (t) is corrected. The corrected acoustic signal P^ Ｌｉ (t) is expressed as the following equation (6).

[0058] 6 is a diagram showing another example of a method for collecting an acoustic signal using a dummy head DH. The dummy head DH is disposed in the same manner as in FIG.

[0059] acoustic signal P^ Ｌｉ After generating (t), the speaker SP of the speaker unit 30 is placed in the space where the dummy head DH is placed as shown in FIG. Ｌｉ (SP Ｌ１ , SP Ｌ２ , SP Ｌ３ , ...) Ｌｉ '(PS Ｌ１ ', P.S. Ｌ２ ', P.S. Ｌ３ ', ...) Ｌｉ (t) are simultaneously played back. ＨＬ Position PS Ｈ１ Position PS corresponding to Ｈ１ ', and the speaker SP of the headphone 20 ＨＲPosition PS Ｈ２ Position PS corresponding to Ｈ２ At this time, the first correction unit 112 simultaneously reproduces the same sound source SS from the microphone MP Ｌ and MP Ｒ By using equations (1) and (2) for an acoustic signal picked up by ＨＬ，ＨＲ That is, when the speaker unit 30 includes I units (I is an integer equal to or greater than 2), the first correction unit 112 generates the acoustic signal P ＨＬ，ＨＲ (t) and the acoustic signal P Ｌｉ In the correction based on (t), the correction coefficient t Ｓｉ and c Ｓｉ The acoustic signal P̂ Ｌｉ (t), and the acoustic signal P ＨＬ，ＨＲ (t), the acoustic signal P ＨＬ，ＨＲ (t) is corrected to produce an acoustic signal P^ ＨＬ，ＨＲ (t) is generated. ＨＬ，ＨＲ (t) and a plurality of acoustic signals P Ｓｉ (t) (acoustic signal P Ｌｉ (t)) can be corrected for time delay and amplitude differences between them.

[0060] (Position Information Acquisition Unit 113) The position information acquisition unit 113 executes a position information acquisition process. Specifically, the position information acquisition unit 113 receives the initial position PS0 and the current position PS1 of the head of the listener U from the position information acquisition unit 211 of the headphones 20 via the communication interface 13. Ｃ The position information acquisition unit 113 acquires the initial position PS0 and the current position PS Ｃ The position information acquisition unit 113 transmits the position information (t) of the speaker SP of the speaker unit 30 via the communication interface 13. Ｌｉ Position PS Ｌｉ The position information acquisition unit 113 acquires the position information of the speaker SP Ｌｉ Position PS Ｌｉ to the second correction unit 114.

[0061] (Second Correction Unit 114) The second correction unit 114 executes the second correction process. The second correction process is a process for correcting the acoustic signal during head movement of the listener U. The second correction process will be described in detail below.

[0062] FIG. 7 shows the head of the listener U moving from the initial position PS0 to the current position PS Ｃ (t) = [x Ｃ , y Ｃ , z Ｃ ] Ｔ This is a diagram showing the situation when the camera moves to the left.

[0063] The second correction unit 114 receives the acoustic signal P^ from the first correction unit 112. ＨＬ，ＨＲ (t) and P^ Ｌｉ The second correction unit 114 receives the initial position PS0 and the current position PS1 of the head of the listener U from the position information acquisition unit 113. Ｃ (t), and speaker SP Ｌｉ Position PS Ｌｉ Receive.

[0064] The second correction unit 114 calculates the initial position PS0 and the current position PS1 of the head of the listener U. Ｃ (t), and speaker SP Ｌｉ Position PS Ｌｉ Based on this, the acoustic signal P̂ ＨＬ，ＨＲ (t) and P^ Ｌｉ (t) are corrected to produce the acoustic signal P ～ ＨＬ，ＨＲ (t) and P ～ Ｌｉ Specifically, the second correction unit 114 generates a value (t) based on the initial position PS0 and the current position PS Ｃ (t), and position PS Ｌｉ Based on this, the amplitude coefficient A of the acoustic signal Ｌｉ (t), time difference TD ｉ (t), and the time difference TD Ｈ (t) is calculated. Ｌｉ (t) is the acoustic signal P ～ Ｌｉ (t) (the reproduced audio signal) and the audio signal P^ Ｌｉ (t) is a correction coefficient for correcting the amplitude difference between the time difference TD ｉ (t) is the acoustic signal P ～Ｌｉ (t) and the acoustic signal P^ Ｌｉ (t) is a correction coefficient for correcting the time difference between the Ｈ (t) is the acoustic signal P ～ ＨＬ，ＨＲ (t) (the reproduced audio signal) and the audio signal P^ ＨＬ，ＨＲ The second correction unit 114 corrects the time difference between the amplitude coefficient A Ｌｉ (t) and time difference TD ｉ (t), the acoustic signal P ～ Ｌｉ The second correction unit 114 generates the time difference TD Ｈ (t), the acoustic signal P ～ ＨＬ，ＨＲ (t) is generated.

[0065] The second correction unit 114 calculates the amplitude coefficient A based on the assumption that the sound pressure is inversely proportional to the distance from the sound source. Ｌｉ (t) is calculated by the following formula (7). Here, d ｉ (t) is the speaker SP Ｌｉ represents the distance from the initial position PS0 to the ｉ (t) = ||PS Ｌｉ -PS0||. d Ｈｉ (t) is the speaker SP Ｌｉ From current location PS Ｃ represents the distance to (t), and d Ｈｉ (t) = ||PS Ｌｉ -PS Ｃ (t)||.

[0066] The second correction unit 114 corrects the acoustic signal P ～ Ｌｉ Time difference TD of (t) ｉ (t) is calculated using the following equation (8). Here, max(d Ｈｉ (t)) is the distance d Ｈｉ (t) represents the maximum value of the frequency component, c represents the speed of sound, and fs represents the sampling frequency.

[0067] The second correction unit 114 also corrects the acoustic signal P ～ ＨＬ，ＨＲTime difference TD of (t) Ｈ (t) is calculated by the following equation (9). Here, d ｉｍａｘ is max(d Ｈｉ The speaker SP selected in (t) Ｌｉ represents the distance from the initial position PS0 to the

[0068] The second correction unit 114 calculates the time difference TD Ｈ (t), the acoustic signal P^ ＨＬ，ＨＲ (t) is corrected to produce an acoustic signal P ～ ＨＬ，ＨＲ (t) to generate the corrected acoustic signal P ～ ＨＬ，ＨＲ (t) is expressed as the following equation (10).

[0069] The second correction unit 114 calculates the amplitude coefficient A Ｌｉ (t) and time difference TD ｉ (t), the acoustic signal P^ Ｌｉ (t) is corrected to produce an acoustic signal P ～ Ｌｉ (t) to generate the corrected acoustic signal P ～ Ｌｉ (t) is expressed as the following equation (11).

[0070] The second correction unit 114 corrects the acoustic signal P ～ ＨＬ，ＨＲ (t) to the headphones 20 via the communication interface 13. The second correction unit 114 corrects the acoustic signal P ～ Ｌｉ (t) is transmitted to the speaker unit 30 via the communication interface 13 .

[0071] (Headphones 20) The following describes the functional configuration of the headphones 20. As shown in Fig. 3, the headphones 20 include, as a functional block, a position information acquisition unit 211. The processor 21 of the headphones 20 functions as the position information acquisition unit 211.

[0072] (Position Information Acquisition Unit 211) The position information acquisition unit 211 executes a position information acquisition process. Specifically, the position information acquisition unit 211 acquires the initial position PS0 and the current position PS1 of the head of the listener U from the sensor 25. Ｃ The position information acquisition unit 211 transmits the initial position PS0 and the current position PS1 to the position information acquisition unit 113 of the processing unit 10 via the communication interface 23. Ｃ (t) is transmitted.

[0073] 1.2 Operation The following describes the operation of the acoustic signal generation device 1. Fig. 8 is a flowchart showing an example of the operation of the acoustic signal generation device 1.

[0074] First, the acoustic signal acquisition unit 111 of the processing unit 10 receives an acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) is acquired (S101). ＨＬ，ＨＲ (t) and P Ｌｉ (t) is an acoustic signal acquired by, for example, the sound collection method shown in FIGS.

[0075] Next, the first correction unit 112 of the processing unit 10 executes the first correction process described above (S102). ＨＬ，ＨＲ (t) is corrected to produce an acoustic signal P^ ＨＬ，ＨＲ (t), and the acoustic signal P Ｌｉ (t) is corrected to produce an acoustic signal P^ Ｌｉ (t) is generated.

[0076] Next, the position information acquisition unit 113 of the processing unit 10 executes the above-mentioned position information acquisition process (S103). As a result, the initial position PS0 and the current position PS1 of the head of the listener U are obtained. Ｃ (t), and the speaker SP of the speaker unit 30 Ｌｉ Position PS Ｌｉ is obtained.

[0077] Next, the second correction unit 114 of the processing unit 10 executes the second correction process described above (S104). ＨＬ，ＨＲ The corrected acoustic signal P ＨＬ，ＨＲ (t), and the acoustic signal P̂ Ｌｉ The corrected acoustic signal P Ｌｉ(t) is generated.

[0078] Next, the second correction unit 114 of the processing unit 10 corrects the acoustic signal P ＨＬ，ＨＲ (t) is transmitted to the headphones 20, and the acoustic signal P Ｌｉ (t) is transmitted to the speaker unit 30 (S105).

[0079] 1.3 Effects of the Present Embodiment In the acoustic signal generating device 1 according to the present embodiment, the speaker SP ＨＬ (and SP ＨＲ ), headphones 20 (open type) worn by a listener U, and a speaker SP Ｌｉ The audio system includes speaker units 30 arranged around a listener U, and a processing unit 10. The processing unit 10 includes an acoustic signal acquisition unit 111, a first correction unit 112, and a second correction unit 114.

[0080] The acoustic signal acquisition unit 111 receives the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t) is acquired. ＨＬ，ＨＲ (t) is the speaker SP of the headphone 20 ＨＬ (and SP ＨＲ ) position PS Ｈ１ (and P.S. Ｈ２ ) corresponding to the position PS Ｈ１ '(and P.S. Ｈ２ The sound source SS is reproduced from the position PS0′, and the sound signal is picked up at the position PS0′ corresponding to the initial position PS0 of the head of the listener U. Ｌｉ (t) is the speaker SP of the speaker unit 30 Ｌｉ Position PS Ｌｉ Position PS corresponding to Ｌｉ The sound source SS is reproduced from the position PS0′, and the sound signal is picked up at the position PS0′ corresponding to the initial position PS0 of the listener U's head.

[0081] The first correction unit 112 corrects the acoustic signal P ＨＬ，ＨＲ (t) and P Ｌｉ (t), the acoustic signal P ＨＬ，ＨＲ (t) is corrected to produce an acoustic signal P^ ＨＬ，ＨＲ (t) is generated. This generates the signal from the speaker SP of the headphone 20. ＨＬ (and SP ＨＲ), and the speaker SP of the speaker unit 30 Ｌｉ The acoustic signals reproduced from each of the speakers and heard by the listener U can be corrected according to the position of the speaker and the type of device.

[0082] The second correction unit 114 calculates the initial position PS0 and the current position PS1 of the head of the listener U. Ｃ (t), and speaker SP Ｌｉ Position PS Ｌｉ Based on this, the acoustic signal P̂ ＨＬ，ＨＲ (t) and P Ｌｉ (t) is corrected. As a result, the speaker SP of the headphone 20 ＨＬ (and SP ＨＲ ), and the speaker SP of the speaker unit 30 Ｌｉ The acoustic signals reproduced from each of the speakers and heard by the listener U can be corrected in real time according to the movements of the listener U's head.

[0083] Therefore, according to this embodiment, an acoustic signal can be generated to reproduce sound in real time in response to the listener's head movements using open headphones worn by the listener and speakers placed around the listener.

[0084] 2. Modifications, etc. In the above embodiment, an example is shown in which the sensor 25 is integrated with the headphones 20, but the sensor 25 may be separate from the headphones 20 as long as it can be worn on the head of the listener U.

[0085] In the above embodiment, the speaker SP Ｌｉ Position PS Ｌｉ is acquired by the GNSS device, but the speaker SP Ｌｉ Position PS Ｌｉ is the speaker SP Ｌｉ Position PS Ｌｉ As long as the above information can be obtained, it may be acquired by a method other than acquisition by a GNSS device.

[0086] Furthermore, in the flowcharts described in the above embodiments, the order of the processes can be changed as much as possible.

[0087] It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by selecting and combining the multiple disclosed constituent elements.

[0088] REFERENCE SIGNS LIST 1 Acoustic signal generating device 10 Processing unit 11 Processor 12 Memory 13 Communication interface 20 Headphones 21 Processor 22 Memory 23 Communication interface 24 Speaker unit 25 Sensor 30 Speaker unit 111 Acoustic signal acquiring unit 112 First correction unit 113 Position information acquiring unit 114 Second correction unit 211 Position information acquiring unit

Claims

1. An acoustic signal generating device comprising: open-type headphones including a first speaker and worn by a user; speaker units including a second speaker and arranged around the user; an acoustic signal acquisition unit that acquires a first acoustic signal in which a first sound source is reproduced from a second position corresponding to a first position of the first speaker and picked up at a third position corresponding to an initial position of the user's head, and a second acoustic signal in which the first sound source is reproduced from a fifth position corresponding to a fourth position of the second speaker and picked up at the third position; a first correction unit that generates a third acoustic signal by correcting the first acoustic signal based on the first acoustic signal and the second acoustic signal; and a second correction unit that corrects the third acoustic signal and the second acoustic signal based on the initial position, the current position of the user's head, and the fourth position.

2. The acoustic signal generating device according to claim 1, wherein the correction by the first correction unit includes: calculating, based on the first acoustic signal and the second acoustic signal, a first correction coefficient that corrects a time delay between the first acoustic signal and the second acoustic signal, and a second correction coefficient that corrects an amplitude difference between the first acoustic signal and the second acoustic signal; and generating the third acoustic signal by correcting the first acoustic signal based on the first correction coefficient and the second correction coefficient.

3. The acoustic signal generating device according to claim 1, wherein the correction by the second correction unit includes: calculating a third correction coefficient for correcting the time difference between the third acoustic signal based on the initial position, the current position, and the fourth position; and correcting the third acoustic signal based on the third correction coefficient.

4. The acoustic signal generating device of claim 1, wherein the correction by the second correction unit includes: calculating a fourth correction coefficient for correcting a time difference with the second acoustic signal and a fifth correction coefficient for correcting an amplitude difference with the second acoustic signal based on the initial position, the current position, and the fourth position; and correcting the second acoustic signal based on the fourth correction coefficient and the fifth correction coefficient.

5. The acoustic signal generating device according to claim 1, wherein the speaker unit further includes a third speaker arranged at a sixth position closer to the initial position than the second speaker, the acoustic signal acquiring unit further acquires a fourth acoustic signal collected at the third position when the first sound source is reproduced from a seventh position corresponding to the sixth position, and the first correcting unit further generates a fifth acoustic signal by correcting the fourth acoustic signal based on the second acoustic signal and the fourth acoustic signal.

6. The acoustic signal generating device according to claim 5, wherein the correction based on the first acoustic signal and the second acoustic signal by the first correction unit is performed based on the first acoustic signal and the fifth acoustic signal.

7. An acoustic signal generation method executed by an acoustic signal generation device including open-type headphones that include a first speaker and are worn by a user, speaker units that include a second speaker and are arranged around the user, and a processing unit, the acoustic signal generation method comprising: acquiring a first acoustic signal in which a first sound source is reproduced from a second position corresponding to a first position of the first speaker and picked up at a third position that corresponds to an initial position of the user's head, and a second acoustic signal in which the first sound source is reproduced from a fifth position corresponding to a fourth position of the second speaker and picked up at the third position; generating a third acoustic signal by correcting the first acoustic signal based on the first acoustic signal and the second acoustic signal; and correcting the third acoustic signal and the second acoustic signal based on the initial position, the current position of the user's head, and the fourth position.

8. A program for causing a processor included in an acoustic signal generating device to execute processing by the acoustic signal generating device according to any one of claims 1 to 6.

Images